Ranking of CS departments based on theory papers | Hacker News Reader

Ranking of CS departments based on theory papers

projects.csail.mit.edu

41 pointskenjackson52 comments

It is hard for me to believe that uchicago.edu should outrank illinois.edu. Illinois.edu is a famous top-tier CS program; uchicago.edu is a top-tier university with a relatively small CS program.

Using papers to rank reduces the impact of reputation, although of course doesn't fully remove it. Also isn't UIUC more of a systems school?

Still, this seems a bit like Harvard outranking MIT, or Penn outranking CMU.

I looked at the Chicago vs UIUC site. Chicago lists 8 theory professors. UIUC lists 7 theory professors.

I don't think Chicago Theory > UIUC Theory seems absurd. Certainly not like Penn > CMU.

That doesn't mean UIUC outclasses Chicago in any given subfield within CS.

UChicago undergrad is ranked #4 in the United States. It's not that surprising.

Undergrand ranking is different from PhD. For instance, yale undergrad >> UCBerkely; however, UCBerkely PhD >>> yale phd program.

At U Chicago many people believe in studying only academic theory. The attitude is that if you're smart enough, you can learn everything practical in industry... if you're one of those unworthy people who wants to go into industry. This attitude affects undergraduates the most, but permeates the whole academic atmosphere. It's not unexpected that a top-tier school that focuses only on academia would outperform in producing academic papers.

Regarding undergraduate and masters' education, U Chicago degrees have top-tier status despite much lower selectivity than comparable schools, so maybe that attitude is worth something.

This is analogous to why Princeton is so high up there as well. The school has a tremendous focus on theory even in the engineering disciplines.

(I studied electrical engineering there and as much as I hate to admit, because I wasn't motivated enough to seek out attainment of more industry-ready skills, it actually set me back significantly as an engineer)

Lower selectivity? That may have been true a decade or two ago but their accept rate is under 10% now.

Seriously? When I went there it higher than 30%. That was less than 10 years ago.

Yup - it has decreased dramatically. Its not much different from Duke now.

https://collegeadmissions.uchicago.edu/apply/class-profile

I can't say whether the metric in the article is good or not. What I can say is that Chicago being comparable to UIUC in CS theory is not an unreasonable position. Across CS it would be false, but Chicago's theory faculty are considered quite good. For example, a few years ago their department housed the only known credible effort regarding P!=NP https://news.ycombinator.com/item?id=1586091. Also, if you look at the two most recent theory hires of the respective departments you'll find them to be of similarly impressive credentials (IAS postdoc, formidable publication record). I don't have the qualifications to really speak more on the subject, but I don't object to the claim.

It's been a bunch of hours since I raised this objection and a lot is going on here right now but let me just raise a belated "I stand totally corrected".

Think of the difference as being like the difference between teaching future engineers and future scientists.

I think many CS programs are, or should be, how to engineer software. Of course, any engineer must study the science behind the field.

It should be amusing that the two professors that developed this system, Saeed and Mohammad, are both at MIT and University of Maryland, and this ranking bumps UMD up from like #17 in CS to #7. Quite handy...

UMD is a state university--and basically, if you're not UCB, imo state schools suffer from the whole "ivy bias" that society tends to have.

I'm not sure that's really true in CS. If you're not UCB, Stanford, CMU, or MIT, you're not a "top" CS dept whether public or private. Georgia Tech, U of Washington, UC San Diego, UIUC, UCLA, UT Austin are all credibly somewhere in the top 20.

As someone who spent time at MIT and Georgia Tech, MIT may be a bit better but the difference in quality of labs, resources and faculty is pretty small, with lots of cross pollination between the schools (exe: Thad Starner jumping ship from Media Lab to GT). I don't think you could credibly call GT anything but top tier unless your only resource is Hacker News or you're focusing on one of the few sub-disciplines the College of Computing (yes CS is its own independent college, not merged with EE) doesn't have great coverage of.

Now if you combine EE+CS, I might believe you.

In terms of actual quality, that's probably true. I've worked with people who absolutely won't hire anyone who didn't go to one of the "top 4", and that's the sort of mindset I was addressing. It's not an "ivy only" mindset, it's a "Stanford-Berkeley-MIT-CMU only" (and sometimes not even CMU and Berkeley) mindset.

Ah, the hiring equivalent of "nobody ever got fired for buying IBM". I've seen the same with people only hiring Harvard Business School grads. I figure school-based hiring practices like that are a pretty good indication of poor management quality.

Dear MIT : one day, you will receive your deserved comeuppance. Best regards, CMU.

This is elitism in the bad sense of the word.

Meh, reads like harmless humor to me. -no particular allegiance, perhaps Georgia Tech

What's the good sense?

You're entitled to that opinion, but it was intended in good fun.

And Berkeley will just be waiting in the wings for the battle to bring you both down. Muahaha.

Dear CMU : We'll get there someday, but I'm not holding my breath. - NCSU

Quantity is a pretty poor metric. I wonder how closely it corresponds with faculty budget.

In general I would normally agree with this sentiment, but they're only looking at top tier conference venues. It's very difficult to pass lackluster work through these places, so it really isn't that terrible of a metric. I would be more interested to see what would happen if they only included double-blind conferences, as name recognition in peer review can help make the rich richer.

As another commenter said, there are few reliable signals that one could really use for this in general, let alone to get a current ranking. Citations are a fine metric but typically it will take a few years for those to really build up on a paper. I would be very interested to see them expand these rankings beyond theoretical CS and into the other disciplines... particularly robotics, where US News doesn't offer a ranking.

> It's very difficult to pass lackluster work through these places

Not really that hard. It's difficult to pass outright bad work through a top-tier conference, but above a quality threshold, it's basically a crapshoot; the reviewing process doesn't reliably separate middling from great papers. The very low acceptance rates (~10-15%) and very high inter-reviewer variance [1] combine to make it pretty random whether a given paper, of sufficiently "ok" quality, gets accepted or not. So the main successful strategy to accumulate a lot of papers in top-tier conferences is simply to submit a lot of decent papers to top-tier conferences. How many such papers a given professor can churn out depends largely on how many students, post-docs, and research scientists they can hire, which depends on funding.

[1] NIPS, one of the top-tier machine learning conferences, did a very interesting experimental study of this. http://blog.mrtz.org/2014/12/15/the-nips-experiment.html

"We certainly do not claim that publication record is the best measure, or a complete measure. However, it is the record that everyone sees and compares in CV, for jobs, grants, reports, etc." > That always makes me really sad.

Signals are hard to come by in acedemic CS.

There are no signals of value under socialism. It's a well-known problem in economics.

Yes, academic CS is socialism---I am a CS grad student, I know. CS academics are just sucking the public teets instead of trading value for value.

I'm an ex-CS grad student; they even gave me a piece of paper called a PhD for my trouble.

Academics isn't socialism. There is definitely an aristocracy in there and whether your work is accepted (not just published, but appreciated) or not depends on how you fit into that. The aristocracy serves on grant and program committees; yes they direct money from the state, but that is where the socialism ends and the rat race begins.

You'll learn that eventually :)

Sounds like as a CS person you've peripherally processed politic science and come up with binary answers to complex ill-posed questions. Academia is not socialism and even communism is not completely lacking in value signals (otherwise it wouldn't be able to deliver excellent medical care, for example).

If you're so convinced academia has no value signals it seems rather foolish (or at minimum hypocritical) of you to pursue academic credentials.

Just to be clear, just because something isn't optimal doesn't mean its not worth being a part of. You can be critical and not leave at the same time. Actually, the ones who stay are the critical ones who have hopes (founded or not) that the system can improve.

Having no value signals is a lot worse than being sub-optimal. Everything is sub-optimal.

Huh? So they are just counting papers published in conferences by faculty members as ranking function? That's nuts in 2015. Exactly the reason why this people couldn't have invented PageRank.

Is it the case almost none offer a degree online? Regardless of how effective online courses become I worry most top tier schools will always opt out to try and avoid diluting the brand. Scarcity and exclusivity use to be a resource problem that resulted in prestige. Now it seems the prestige must be artificially maintained by limiting the supply to anyone not happy with just a certificate.

Georgia Tech, #8, offers an online Master's of CS. I'm enrolled now and it's quite challenging and rewarding.

Same here. Highly recommended. p.s. the tuition for the whole degree is around $7,000. Stanford and Berkley offer an online program for $60K and $50K (the latter is for a data science program).

Would you mind describing a bit about what you learn or study in the program?

I'm not an academic, but I don't understand the point of ranking both by "1" and "1 + 1/2". 21 schools were ranked identically by each method. 18 were one rank different. At this point we seem to be straining at gnats. Was this really just to mess with Texas A&M?

I'm just surprised my CS department made the list. That's nice at least.

Even though we are at the bottom...

Is it NC State? I found the instruction there excellent--at least for theory classes like Automata, Decision And Complexity Theory as well as Data Structures. They offer a real distance education that is run just like an in-person class, with videotaped lectures, projects, and proctored exams. They are one the few universities that offer the full repertoire of bachelor CS classes online, a little known secret. Even though their distance education isn't well marketed, you could complete in theory a BS computer science remotely just through NC State--something that you can't quite yet do with MITx or Udacity--and in addition, their online CS classes do not have an asterisk on your transcript indicating they were taken online, since you are often lumped into a section with the in-person class.

The ranking here is about research output, not quality of teaching.

This list isn't all that meaningful because they have a narrow definition of "CS theory."

It's perfectly meaningful of course if you want a ranking based on that narrow definition.

That's true, but I do think its disappointing we rank so far behind our neighbors (Duke, UNC) in CS in it seems every ranking.

How do Canadian universities compare?

The title needs "in the US" at the end to be more descriptive.

Cal >> Stanfurd.

Reply on news.ycombinator.com