At least when I went there it wasn't very crowded. There was a group of people at the summit (with vending machine present, of course), but otherwise hardly any people. However it was a slightly foggy day, so you couldn't see much from the summit. It is possible that on a clear day there are crowds trying to see Mount Fuji or something.

Perhaps it was this one. http://jihan.30maps.com/map/54837

Takao can get too crowded. Head to Jinba 陣馬 which is just a few train stations away for fun and good food on top.