39 karma · joined February 9, 2010
Your point is still valid, but perhaps we approach a time when over-reliance is better than all but the best human pilots (Sully, perhaps).
My other concern was the answer to #1, namely, "4m + 5b" vs "5m + 4b". I feel like if you know it is in that form (sum rather than product) the two answer choices are just intended to trip people up who are moving through the test at speed.
In fact I would say an important "mistake reviewers make" is ... not actually doing much work. I've seen some appalling 2-3 line comments like "seems fine", even from senior academics. That's not even to talk about problems with misunderstood p-values, not reading the algorithm closely, not walking through the proof manually, and so on.
I did not explicitly look at size as this was not one of the original research questions, but a good point. I suspect older systems will tend to be larger (in the domains we studied, anyway). And your point about arch choices being great "early on" can, I think, be captured in the "system age" variable. I guess I'm trying to think of a system that might be young and yet quite large in LOC.. would be an interesting outlier to look at.
[1] http://neilernst.net/2013/01/25/teaching-advanced-software-e...
see http://forums.macrumors.com/showthread.php?t=1446182&pag...
Like any profession, academia is about networking. Social media can really help here, and I think academics are slowly understanding this. There are two problems: blogging/tweeting are lower priority than grant-writing, teaching, marking, and paper-writing; and academics are usually older and (in my experience) older people don't 'do' social media as much.
But in software engineering, at least, there is a good community of people on social media. For example, the Never Work In Theory blog (of which I'm an author), and various academics on twitter, like @profserious or @avandeursen.