https://gds.blog.gov.uk/2013/10/21/how-many-people-are-missi...
1.1% of users don't see JavaScript enhancements. Most because it doesn't load, not because they've disabled it. Also, some people use browsers that don't support JavaScript, or their ISP filters JavaScript, or a whole bunch of other things.
I'm not endorsing making menus in CSS. I'm just saying that it isn't "accessibility vs. people who turn JS off".
As the link says: progressive enhancement is the solution to this.