Hacker News Reader

Hindsight is 20/20. You don't get to reinterpret statements made earlier just because he got the general gist.

Like he said, his specificity screwed him (laptops, xml, etc) - He's totally not being too hard on himself. He's being fair and honest in his evaluation.

It would be super easy to predict trends if you are nonspecific: I predict new and shiny will win in 10 years. Bleh that's a non-statement.

> You don't get to reinterpret statements made earlier just because he got the general gist.

Of course someone can. You can do whatever you want too. Predictions are hard and any correlation in 11 years is impressive, as are these. The question raised is not "is this fair or accurate according to my standards" but moreso, "who cares?"

> It would be super easy to predict trends if you are nonspecific

Otherwise known as the Nostradamus principle.

Agree. It's an interesting post, and I don't mean any criticism toward him (the thought process was good), but in practice I'm not sure he truly got any of them right.

"a new internet community hangout will appear" is not really a prediction. People have been "hanging out" online since BBS's and AOL. Would anyone bet against the same prediction looking from today forward?

The only other one where he gave himself full marks is "someone will make a lot of money by hosting open-source web applications." It's true of course, but PaaS is not specific to open source applications. If you take away the open source part, what he got right was that someone would make money by hosting webapps. And that's been happening in some form since at least the early 2000's when AWS launched.

That probably sounds more critical than I meant it. My hats off to him hitting a lot of them pretty well. But I wouldn't say he is going easy on himself with the scoring.

AWS didn't come out until 2006, and it was considered pretty revolutionary at the time.

The way you hosted a website in 2004 was that you signed up with a web hosting provider and either rented a shared webhost for $10/month or paid for a dedicated server for about $60/month. There were little or no provisions for being able to grow your hosting needs at the click of a button. When you read stories of how eg. HotOrNot scaled, it involved a lot of panicked calls to investors, and on-the-phone negotiations with hosting providers where they were like "We need more servers now, we'll make sure we can get you the money within 30 days", often followed by a tech going down to the datacenter and physically installing a new server in the colo cage.

The other revolutionary part of his prediction is "Someone will make a lot of money". ISPs & hosting in 2004 were a very competitive commodity business, filled with overcapacity from the dot-com boom.

> AWS didn't come out until 2006, and it was considered pretty revolutionary at the time.

He did work at Amazon until 2005, so he likely saw the internal development that resulted in what would become the publicly-available AWS. And some of his other predictions strike me as being heavily influenced by what he saw working at Amazon. Companies like Amazon and Google tend to encounter problems a few years before the rest of the world realizes that those are even concerns. So making predictions about the industry at large based on the cutting edge technologies used inside those companies is actually a pretty reasonable assumption.

For example, according to Wikipedia, Dynamo and BigTable were developed in 2004. So while NoSQL would have seemed revolutionary to almost everyone outside of Amazon and Google in 2004, to people working inside those companies, that prediction was basically predicting that their ideas would work and become very successful. I'd also bet there were a ton of Google engineers that saw Map Reduce before it was made public that could have predicted in 2003 that computationally-heavy tasks would be primarily distributed in a few years. And there's probably a bunch of Intel employees who could have predicted in 2004 that the chips Intel would be selling a decade later would run at the same GHz as their current offering since 2005 was about the time that they switched to adding more cores.

When you're ahead of the curve, you can see what's coming around the corner before the rest of us.

How specific do you need to be? Do you need to provide implementation details? I give full marks for the XML Database prediction because he began with this statement: "Nobody likes to do O/R mapping; everyone just wants a solution." And then: "XML is the most popular vehicle for representing loosely-structured, heirarchical data."

I could quite happily interpret that as "schemaless databases (using the most popular vehicle for representing loosely-structured data) will surpass relational databases in popularity by 2011". I'll take marks off for 2011 being too early, but not for the datatype choice.

It's unfortunate that he went into more detail by describing XPath and current support for XML operations, but I feel that's just justification for the specificity of the datatype choice in the title.

I totally agree with you regarding laptops and apple though. "New and shiny" is hardly a prediction, and the specificity is relevant to the prediction.

> I'll take marks off for 2011 being too early, but not for the datatype choice.

I would. If he had said something like "XML is most popular... but really meant for documents not data, so something similar will probably replace XML" then I would have given full marks, even if he didn't specify JSON or BSON explicitly.

(I would give ~50% though, not 20%)

XML databases actually got to be amazing, if that was his prediction he would have been totally right. The fact is that they just got amazing a little bit too late. No one is going to consider an XML database because XML got a bad reputation (and rightly so).

But XML database technology actually has nothing to do with XML. The idea is that you store nested data nodes. Imagine you have a mongodb collection, and in it you have documents, and each of those documents would also be a mongodb collection in which you could insert more documents. You can build full graphs like that, and if you allow references to nodes those graphs can be cyclic too. Pretty powerful right?

Now you might say yeah of course, but that's got to be crazy slow! And at the time Yegge made his prediction you would have been right. But then academics came into the scene, they developed data structures and algorithms that allow you to store these recursively nested documents in column stores (think SQL table with 2 columns, a primary key and a piece of data) in such a genius way that you can make whatever query you can think of (recursive, joins, whatever) and execute them almost as fast as any indexed search query.

That magic happens, and is implemented for example in MonetDB/XQuery which sadly got abandoned a few years ago.

I still think if a crazy person sets off to go build a database company (crazy people like that seem to be quite popular in SV these days) they could make a lot of money forking MonetDB/XQuery, and search-replaceing every instance of X with J and every instance of XML with JSON. Crazier still, you can even let MonetDB/XQuery results be formatted as json, even before MongoDB became popular I think.

Isn't that just the Object databases from 15 years ago? I remember using one from Versant that seems to be what you're describing.

Could you go into a bit more detail about the data structures and algorithms used in doing recursive queries on nested data in a column store? Pointers to relevant papers/documentation would be great.

This paper summarises the MonetDB/XQuery work, I'm on my phone and couldn't find a free PDF but I bet there's one somewhere. I read this paper while I was an undergraduate, it's very accessible.

http://dl.acm.org/citation.cfm?id=1142527

OpenLink Virtuoso implements a lot of this functionality (xml storage, column store, graphs, rdf, json), optimized for speed and open source. I know because I was lucky enough to have had conversations about it with the guy responsible for it, Orri Erling http://sites.computer.org/debull/A12mar/p3.pdf

Not everyone has a "horse" today. You're thinking of the developed world! (I remember a WIRED article back in 1999 where a panel discussed which technologies would have the biggest impact in the 21st century, and someone suggested the telephone, only then penetrating third world markets; the same person pointed out that the most impactful technology of the 20th century was probably the bicycle.)

I get your point, but the nuclear bomb by preventing major countries from going to war with each other is a good contender for most impactful technology of the 20th century.

Oh for sure (on the nuclear bomb) -- it's really impossible to compare these things meaningfully. Nuclear bomb? Modern agriculture? Penicillin? Vaccines? It's really impossible to judge.

Which countries did the nuclear bomb prevent going to war? There was never any serious prospect of the US invading the USSR or vice versa but MUAD certainly didn't stop them from involving themselves in more wars (generally supporting opposing sides) in the late twentieth century than in the previous four, India and Pakistan haven't exactly softened their stances on Kashmir since acquiring deadly weapons, and the UK even had part of its territory briefly annexed by a foreign power.

Tell that to the NATO and Warsaw Pact armies and all of the plans, materials, and infrastructure they built around fighting the Next Big European/Asian war. Huge amounts of money and piles of weapon systems were developed by both sides with an eye towards fighting that war.

It's easy to say that in hindsight, but given all of the tension around the Iron Curtain and all of the times the US and USSR came very close indeed to going to war despite how likely total nuclear destruction was, it seems short-sighted to just brush off the idea of that war actually happening if neither side had any nuclear weapons.

I think it was more luck than judgement that we escaped annihilation. There were any number of times where a wrong call would have meant Game Over. And the probability would have been much higher if someone other than Gorbachev had been the last Soviet president.

History isn't over yet. The risk is probably higher now than at any time since it peaked in the mid-80s.

Depends, personally I'd see the number of times the US and USSR nearly went to war (or actually did, but indirectly) despite having nuclear weapons as pretty good indication that lack of any feasible strategy for inflicting a decisive military defeat on the other party was a much more significant factor in the Cold War not turning hot than any nuke-induced desire for peace

Ultimately, one naval officer's minority vote[1] prevented a Soviet sub from launching a tactical nuclear weapon at US shipping. Several thousand miles of ocean prevented the Soviet Union from feasibly achieving decisive military victory against the United States. [1]https://en.wikipedia.org/wiki/Vasili_Arkhipov

It's true that an invasion of the US mainland was never in the cards, given the situation during the cold war. Pretty much nobody was seriously worried about that. What NATO was very worried about was a Soviet invasion of Western Europe, which is a very reasonable worry considering the number and detail of Soviet plans for doing just that.

For their part, the Soviets were very worried about a combined NATO invasion, which seems quite reasonable considering that Germany alone, while also fighting/occupying most of the rest of Europe, managed to get to the suburbs of Moscow, and kill millions of Russians along the way.

Now if the Soviets did manage to invade and conquer Western Europe and maintain control over it, then they might be in a rather better position to consider direct attacks against the US. Which is a good part of the reason why the US invested so much in keeping them out of Western Europe.

> the nuclear bomb by preventing major countries from going to war with each other is a good contender for most impactful technology of the 20th century.

It didn't prevent the superpowers from having wars with each other. It just shifted the battle fronts, so now the superpowers fight through proxy wars, and the deaths all happen in distant, "developing" countries.

It's speculation, right? Either it had an overwhelming effect OR it had little if any effect. We'll never know -- thank goodness. We do know that penicillin, clean drinking water, vaccines, and the bicycle had enormous effects all over the world.

> It's speculation, right? Either it had an overwhelming effect OR it had little if any effect. We'll never know -- thank goodness.

By that logic, we can 'never know' anything about historical effects, ever, and it's pointless to discuss the consequences of any historical events, including the others you mentioned. But really, the effect of the nuclear bomb in shifting the battlefronts of wars to proxy wars in developing nations under hegemonic control is as clear and as accepted by historians as we can expect.

It may not be quite on par with the "theory" of evolution being all-but-proven fact, but it's in that ballpark.

It's not pointless but we can tell for certain that some things had a certain direct impact. It's very hard to claim that the psychological impact of nuclear weapons was X as a matter of fact, and it's nothing at all like the theory of evolution which is confirmed by ridiculous amounts of direct and indirect evidence.

In a world without nuclear weapons, the USSR would have invaded Europe (and the Middle East) and the entire continent would have been laid waste. They came ridiculously close to doing it anyway despite nuclear weapons. If you think that such a European war would have meant that conflicts in other parts of the world would not have happened, I think you're being a bit naive.

> most impactful technology of the 20th century was probably the bicycle

The smallpox vaccine is a strong contender. Smallpox killed 300m people in the 20th century, more than all wars combined. And we wiped it out.

The smallpox vaccine would be a candidate for most impactful invention of the 18th century (the first smallpox vaccination, named for the fact that it was derived from a cowpox blister -- vacca being Latin for cow -- was administered in 1796).

Yes but we're talking about the century in which the technology had an impact, not the century in which it was invented. The telephone is a 19th century invention which is only reaching the majority of the world's population in the 21st. Likewise the bicycle was a 19th century invention that really hit home in the 20th.

The form of the vaccine that allowed us to undertake global eradication was much newer:

" In the late 1940s and early 1950s, Leslie Collier, an English microbiologist working at the Lister Institute of Preventive Medicine, developed a method for producing a heat-stable freeze-dried vaccine in powdered form.[50][51] Collier added 0.5% phenol to the vaccine to reduce the number of bacterial contaminants but the key stage was to add 5% peptone to the liquid vaccine before it was dispensed into ampoules. This protected the virus during the freeze drying process. After drying the ampoules were sealed under nitrogen. Like other vaccines, once reconstituted it became ineffective after 1–2 days at ambient temperatures. However, the dried vaccine was 100% effective when reconstituted after 6 months storage at 37 °C (99 °F) allowing it to be transported to, and stored in, remote tropical areas. Collier's method was increasingly used and, with minor modifications, became the standard for vaccine production adopted by the WHO Smallpox Eradication Unit when it initiated its global Smallpox Eradication Campaign in 1967 " -- https://en.wikipedia.org/wiki/Smallpox_vaccine

If "horse" is a stand-in for non-human-powered transportation, then many people in the developed world don't have one. Plenty of cities (outside of stereotypical US suburbia) are perfectly livable only owning a bicycle because of their density, decent public transportation for when you have guests, and car sharing.

That speaker was right: don't underestimate the impact of bicycles.

I consider him much less correct with his mobile phone prediction. "At least 5 years out" is completely, totally wrong. If you made a business decision with that prediction you'd have felt a severe sting.

I agree with you about the Apple and XML ones though.

Not really. Did you see Dan Luu's original post at http://danluu.com/yegge-predictions/ ? Even though the iPhone was released in 2007, smartphone adoption didn't really start exploding until 2009.

Sure, but Yegge's original post wasn't cautioning about speed of adoption being slow, it was ridiculing the notion that companies were even close to knowing what consumers wanted and suggesting there were huge collaboration and content problems, as well as technical issues, questioning the idea of a $500 price point for ordinary consumers and suggesting that smartphones and tablets people actually wanted to buy weren't going to happen for a "long time".

In practice, the ultimate form of today's smartphone was starting to take shape in secret in Cupertino pretty much as he wrote his article, the now ubiquitous design went on sale within 2 and a half years, the App store came the following year and Apple alone had already shipped 30M units before the "at least" 5 years were up. The release of the iPad also took less than six years.

Yegge didn't see significant enough innovations taking place within 5 or so years to make an impact. In fact, all the innovations that lead to over a billion people owning smartphones today took place in that time frame, with only incremental improvements since then.

I think it's pretty generous of Steve to give himself more than half marks for that!

The first iphone didn't have apps.

> If you made a business decision with that prediction you'd have felt a severe sting.

I somehow feel that this would depend on the decision you made.

If the decision was "it's not too late to make it big in this market if we start NOW", I think you would have done okay.

Like BlackBerry tried?

He is being perfectly fair and accurate, which is probably a first for anyone making futurist predictions, so kudos to him.

This is exactly why all futurists are full of shit (see Kurzweil) and predicting the future is impossible. Getting some aspects of your prediction right is virtually guaranteed if you generalize when you re-interpret later.

You'd have to make spectacularly bad predictions that are overly detailed and specific NOT to do half decently if we're going to allow post-hoc retrofitting of which broad trends kind of fit the original prediction.

How did iPads "disrupt" the laptop model?

That's just the bubble you're living in talking, as in most of the world, iPad sales have been declining, with the Mac revenue surpassing them again. And besides the iPad, the industry for tablets is basically non-existent. Really, our world is weirder and weirder.