Either you're interviewing bottom-tier candidates or haven't paid any attention to schools that are out there. If you're in the US, you should be recognize the names of the top 25 CS schools. You might not know their rank, but there aren't any obscure names on that list, and they graduate a lot of students.
Thankfully my co-workers who graduated from MIT and Stanford didn't know that, or otherwise they would have thrown away my resume.
In the other extreme I've even heard people here in HN argue that MIT and Stanford graduates are often mediocre, which is very different from my experience.
Hiring is hard and we all do it differently. YMMV.
Unless you are in the top 3 the distinction is mostly academic.
> At that point I might as well memorize them all.
Read through the list. You'll probably recognize a lot of the names. Out of 50, if you pay much attention to colleges in the US, I bet you'll recognize 40 of them.