I don't use IE, but I don't use Chrome either and refuse to use Chrome. Why I refuse to use Chrome isn't the point of this response, the point is there are reasons why I refuse to run Chrome and if you make it so your site is only accessible via Chrome then I won't be visiting your site.
What I just said about Chrome could be said for any browser, a lot of people have different reasons for not running certain browsers on their system. To generalize it's a bad idea to make your site accessible only via a specific browser.