A significant number of people have modern browsers but browse with JS disabled (eg NoScript). If your layout relies on JS, you may be making your site inaccessible to them.
Can you cite?
Whether that's significant or not is a matter of opinion. Most estimates I've seen are 3-6%, but should be taken with a big grain of salt. For sites I've worked on I tend to see about 2-4% of users using Google Chrome as their browser, for comparison.
I realize this doesn't help much.
I was aware that most web stats packages use javascript to report stats - when we did the exercise of checking for disabled JS, that's where we looked.