Should I allow Windows Explorer to connect to the Internet?

Yes, Windows Explorer is practically just another instance of Internet Explorer. We recommend you to allow this activity.