Don't know much about music schools, but I'd think Juilliard to be much more elite than Berklee, and closer to the "Harvard of music" that OP mentioned.
You're right about that. And same for Curtis Institute of Music and probably a dozen other conservatories as well. But those are more focused on "art music" / classical music.