Steve Yegge's predictions from 2004 | Hacker News Reader

Steve Yegge's predictions from 2004

steve.yegge.googlepages.com

117 pointsgozzoo72 comments

You guys seem to be saying he was wrong on most of them. I strongly disagree. Sure, he didn't nail every one exactly, because he's not psychic, but for all of them, he correctly saw the trend, and he's very in the ballpark.

1. This is happening, just not as quickly as he thought, and with JSON. (XML Databases)

2. If you actually read what he wrote, this is obviously happening. EC2, Github, Heroku. (Open Source Hosting)

3. Using processes instead of threads is absolutely rising in popularity.

4. Off on the timing, but we are in the middle of an explosion of other JVM languages. The JVM has clearly established itself as the best way to launch a new language.

5. Off on the timing, possibly off completely. Clojure is gaining in popularity, which ties in with #4. (Lisp in the Top 10)

6. Facebook. This one was easy. (Community Hangout)

7. He was actually late on this one. iPhone obviously. (Mobile Computing 5 Years Out)

8. Google Apps. (I'll Pay Google)

9. Wrong literally, but if you take out netbooks, which weren't around, Apple sells the large majority of $1000+ laptops, and they obviously sell a ton of iPhones, which are just small computing devices. (Apple Selling Laptops)

In fact, he was the first to point out that most of them were wrong. But he did get the general concept right in most cases.

Re #2, I don't think its fair to count EC2, Github, or Heroku, unless you're also counting any shared hosting that lets you run PHP.

I think what Steve was referring to was people paying for hosted versions of specific pieces of open source software, e.g., a hosted version of SugarCRM, not the ability to run open source software in general.

It was still prescient though, because we're starting to see that. For instance, you can do this with Azure + SugarCRM.

He's still wrong on most of them, whether you strongly disagree or not.

1. This one is so mushy, it's wrong just because of it. Substituting your favourite representation format (JSON instead of XML) doesn't make it any better or really anything to do with databases.

2. It's happening if you think EC2, Github and Heroku are somehow comparable. Also which one of these is making 'a lot of money' as predicted?

3. Your response doesn't actually address the prediction. Also, Clojure, Erlang, blah blah.

4. Just hasn't happened.

5. See above.

6. And Twitter and 'a new internet community hangout will appear' in 5 years is the most solid bet you can make on the internet.

7. It happened to be 5 years out or under. Oh well.

8. Nobody, statistically, pays Google for anything. Other than adwords.

9. Straight out wrong.

I think that easily qualifies for 'wrong on most of them'.

Regarding 1:

* XML is the most popular vehicle for representing loosely-structured, heirarchical data. Relational databases can't do this very well. ...XML can be mapped straightforwardly to the objects and data structures of modern programming languages, so it works better than relational modeling for serializing things like object graphs.*

While his estimate of market share is off (he suggested >50%), he was right about the future popularity of document databases. The only thing he got wrong was the specific serialization format.

So he was wrong on the market share, the format, and the everything else. I think that's still a solid 'completely wrong'. He is, of course, right, about every single thing you choose to read into the statements that he doesn't actually make. I don't see him predicting the 'future popularity of document databases'.

From his article: Many common types of data are intrinsically heirarchical and loosely structured, so storing them in an RDBMS can be difficult. Examples: books, articles, and most other written documentation. HTML. Search indexes. Filesystems. Game worlds. Classical music scores. It's just easier to model some things this way.

I don't see how to interpret this as anything other than a document database. Could you explain to me?

Sure. Let's take another look at the article -

"Prediction #1: XML databases will surpass relational databases in popularity by 2011."

It's still completely wrong. I don't see how to interpret it as anything other than wrong. Yes we can sift carefully through it to find some way in which it's kind of, maybe right, sort of. We're discussing Steve Yegge's actual predictions not whether filesystems or the Library of Congress are really a special case of document databases.

You are arguing syntax, I'm arguing semantics. {"foo" : 25, "bar" : 32} is just not significantly different from <foo>25</foo><bar>32</bar> in my view (and I imagine in his view as well).

I think we should just agree to disagree on this one.

Its kind of like astrology - they are almost all right if you want them to be right.

3. Not really. What is rising in popularity is making multi threaded programming easier. I'm thinking about STM and lock free structures.

In summary, he nailed it as far as where the the wind is blowing but was wrong on timing; on all accounts except the last one. This is no bad feat as nailing both the direction and timing is nearly impossible.

Compare with, for instance, Bill Gates, who has been claiming that speech recognition as a primary means to control computers is just five years away, for over twenty-five years now.

He nailed it but was wrong on timing? That's like really nailing it on flying cars and the return of the Messiah. Maybe in a few centuries we'll be reaping the benefits of XML databases and salute Steve Yegge's prophetic vision. Come on.

Flying cars are nowhere to be seen, neither is the return of messiah.

The point is that Steve's predictions are actually observable, though some of them are advancing at a slower pace than he thought (and a tiny bit different, like JSON instead of XML, which IMHO is irrelevant).

There are a few people trying [struggling] to make flying cars too!

(but I agree with you :))

They are 'observable' if you bend your critical faculties to a point where small airplanes are flying cars and the Messiah is your favourite politician. Most of Steve's predictions are nonsense. Most predictions are, so it's hardly Steve's fault. But going through great logical contortions to defend some predictions the guy was either brave or foolish enough to commit to paper in 2004? I'll say again, come on.

Small airplanes already existed in 2004, and politician and son-of-god is quite a bit different than XML and JSON which are basically interchangeable serialization formats for the purposes they are both used.

Remind me again what that has to do with the point at hand, please.

Sorry, but there's a saying in investing: "Right but early is wrong. Right but late is wrong."

Any fool can make a very long list of predictions that will all turn out to be correct, even in the stock market, as long as he doesn't have to give them a timeline. Plenty of people saw the housing crash, for example, and plenty of them understood that certain banks were underwater, and regardless of their actions, they were going to go under. However, just knowing that isn't actually worth anything -- a short position carries a heavy premium in interest, and in the famous words of Keynes: "The markets can remain irrational longer than you can remain solvent."

In your 'sayings' early and late are ambiguous. As far as investing is concerned being right in a prediction means you profited, so the early-late error bar is defined by the time in which you could profit by your prediction. Applying this same rationale here the early-late error bar is harder to define, since profit is not clear and somewhat subjective. So taking some liberties with early-late and right-wrong is I feel appropriate.

Taking into consideration that these predictions are 5+ years old I think they are pretty good. He clearly spotted several trends whose future significance was at the time not obvious. If someone started a business believing one of these trends had promise it is possible that the business would have been successful. I do feel that for what he was trying to accomplish being early or late by a couple of years or off on a detail but correct in spirit is not significant.

Also, I think it is important to add that making predictions like these is very important becuase the future develops from our perceptions of what it will be.

Security prices have two basic outcomes: "up" and "down." Usually they fluctuate, so almost any call will "eventually" be proven correct. Sporting events are also mostly binary outcomes, so "the Cubs will reach the World Series" will probably be right if given 200 years.

But with technology predictions, there's an almost infinite set of outcomes, so being right in direction but not timing is significant. For instance, someone who says "Little Rock, Arkansas will win the Super Bowl within 5 years" will look pretty smart even if it takes 6 years for Little Rock to (very unexpectedly) get an NFL franchise and 9 years to reach the Super Bowl.

Granted, technology predictors can use also use some tricks to narrow the possibility space in their favor.

Hey, hey, speech recognition is really here. In fact, I ham dict eating too the come pewter write now.

It's like an automatic mondegreen generator.

I launched Dragon Naturally Speaking and said, "I ham dict eating too the come pewter write now", which it transcribed as "I am dictating to the computer right now."

It's important to note here that JSON is essentially XML without the boilerplate - and in fact was invented at least partly to address this very XML pain point.

So it is essentially XML except for all of the things that make XML any different than csv or some other text-based representation?

I don't think your comparison is fair. CSV has only one structure (tabular data) and only one datatype (string). JSON supports multiple datatypes and data structures (objects, lists, hash tables).

Absolutely not.

XML has its roots in document creation for large organizations, and has a structure appropriate for that -- mostly an ordered graph of text nodes, which do not need to be declared (and indeed are not consistent between parsers), with provision for special node types with their own contents. But early in its history XML got hijacked by a lot of comp-sci theorists who wanted to create highly abstract notions of messaging, separating message validity and provenance from application logic. This gave the world things like XML Schemas and SOAP and worse.

Meanwhile, JSON is a straight representation of common scripting language data structures. Arrays, dictionaries. Simple. Node boundaries are syntactically explicit. In this philosophy is taken for granted that message validity will be ascertained at the application level. There are no fantasies of doing so at the document parsing level.

re: 1. "JSON databases" are starting to displace Relational Databases?

Your terms - json databases - do not have enough search volume to show graphs. http://www.google.com.au/trends?q=json+databases

OODB were all the rage a decade or two ago, and they solved the O/R impedance mismatch - but they didn't become mainstream. What has changed, that will make "JSON databases" succeed?

EDIT Actually, hierarchical databases were popular before Relational DB were invented, and then everyone switched. Why? Normalization of RDB is said to be important, but it seems little used in practice. It seems more a selling point, but it usually takes more than a good pitch to motivate deep organizational change (maybe it was different in those early days, when new installations quickly outnumbered old ones?)

It is because databases often outlive the application that created them. They also often serve more than one application, for more than one purpose (e.g. operations vs. business intelligence). For this, you need a data representation that is in terms of the data itself rather than the use. The problem with data structures in a programming language, such as objects is that they are designed in terms of their use; a data representation is chosen that makes sense for what you want to do with the data.

The O/R impedance mismatch is not just between "media" (objects vs. tables), but also between the specific representations chosen within that media. If you store your objects directly in a DB, your data is closely coupled to your use; the same problem occurs with OR mappers. I surmise that the advantage of relational over hierarchical databases was that they could better represent the data in terms of itself.

JSON databases will have the same problem - the data representation will be designed in terms the application that generated them, and this close-coupling will make it difficult to interoperate with a later application, and with other applications, with other purposes. The data representation will not be designed in terms of itself.

I used to be a 'wizard' (player admin) for Yegge's GMUD, Wyvern (he mentions it occasionally in his newer world-facing blog; it can be found at www.cabochon.com). He doesn't just make predictions like this, he actually bets on them.

For example, he bet that people would want to play a game with deep and intriguing gameplay regardless of the graphics. Then he bet that people would want to play it on a handheld device (specifically, he targeted the Sharp Zaurus, but the game is compatible with some other palmtops of the same era). I think he's right on the first count, but too early on the second (targeting the iPhone, for instance, would probably have been more successful).

There are a number of other reasons the game didn't take off, but the point is that he's not an armchair pundit, shouting that we should all listen to him and take action based on his foresight. If anything, he's taking action, and just logging the predictions that prompted him.

Wyvern on an iPhone. That's exactly what I want to see, and what the other games I've been playing don't quite match up to. It has a low demand on the graphics and network side of things, but has a rich and rewarding game play experience. For me, obviously.

I haven't played that for a few years. I'm going to have to start again.

I don't know exactly what it was, but the game lost a lot of magic for me at some point. Now the aesthetic is totally different, and I just can't get back into it.

TL;DR:

Prediction #1: XML databases will surpass relational databases in popularity by 2011.

Prediction #2: Someone will make a lot of money by hosting open-source web applications.

Prediction #3: Multi-threaded programming will fall out of favor by 2012.

Prediction #4: Java's "market share" on the JVM will drop below 50% by 2010.

Prediction #5: Lisp will be in the top 10 most popular programming languages by 2010.

Prediction #6: A new internet community-hangout will appear. One that you and I will frequent.

Prediction #7: The mobile/wireless/handheld market is still at least 5 years out.

Prediction #8: Someday I will voluntarily pay Google for one of their services.

Prediction #9: Apple's laptop sales will exceed those of HP/Compaq, IBM, Dell and Gateway combined by 2010.

Prediction #10: In five years' time, most programmers will still be average.

OK, I'll take a stab at them. (Yes, I did read his Important Note, but why not?) Status:

1: No

2: No (I think). I don't know if Google App Engine is making money for them, and Github doesn't host apps, and Amazon Web Services doesn't (only) host open-source web apps.

3: Hopefully.

4: Doubtful, but one can dream =)

5: No.

6: HN? Is Steve Yegge here?

7: Yes. I think it's valid to put the iphone's success in 2009 (rather than earlier) due to the large number of apps and growing number of competitors in that year.

8: Well, considering he's working there, I think he's being paid instead =)

9: Probably.

10: Yes, although it's not a tautology. (You could have asymmetric heavy tails which would make most programmers below or above the mean.)

If you take prediction 1 as key-value stores (which I think is more-or-less where stevey was going) - he's spot on.

"surpass relational databases in popularity by 2011"

Are you really sure about that? AFAIK the use case for key-value stores isn't one that a lot of people are going to suffer (seriously high-volume).

I think there was a lot of excitement about XML databases around that time, because they would let you store more complex structures in the DB, not less complex (like key-value). Somehow XML did not take off, not sure why.

#1 is kinda sorta like the NoSQL movement (hierarchical versus relational modeling, do away with much of the O-R impedance mismatch).

#2 Github?

#3 The evils of threads, and how to replace them with something else to use multi-cores, is very high on the list of blogged about programming topics. Several new languages have staked their claim to fame largely on how they handle concurrency. So, yes, this one was understated, if anything.

#4 Not there yet, but might be on that trajectory if there is some "tipping point" of Java programmers en masse getting fed up with Java's limitations. There are certainly enough alternative JVM languages to choose from.

#5 Does Clojure make the top 10? Certainly not in number of users, but maybe in most popular to blog about.

#6 Facebook.

#7 iPhone was released June, 2007, beating his prediction by a couple of years.

#8 Maybe if you buy an Android phone from them?

#9 Apple laptops are immensely popular, but the scale of this prediction was a little ridiculous. Between iPhone and iPod, it's possible that there are more people who own an Apple product than own a product from all those companies combined.

#10 Now that's just insulting. Who's he calling average?

#10 Now that's just insulting. Who's he calling average?

The average programmer.

As if that weren't bad enough, at least half of them are still below average.

Only if it's a standard distribution (eg bell curve). It's quite possible with a non-standard distribution to have much more than half above average as well. Now median on the other hand, by definition, separates the distribution in half... </pedantic>

Or perhaps much more than half below average.

Given that there’s no reasonable way to quantify programmers, and we could arbitrarily change the scale however we like even if there were, usage like “average programmer” is generally understood to mean the same thing that “median programmer” would mean, if we had such a rubric (since we don’t, “average” is a fine term). Your pedantry is misplaced.

Emacs Lisp seems to be the 11th most popular language on github. Common lisp, clojure and scheme seem to be 19,20 and 21 respectively. If we count them together, they might make it to top 10.

It depends on what else we count together.

#2 Heroku

Yes, #10 is a bit offensive, implying that he is in the top but you probably aren't. Also, it is rather unlikely that you can attain a huge productivity edge over others in everyday tasks like "take an old buggy codebase and add a simple feature here and there". This might work in more research-oriented programming, where some can create things that others didn't even think possible; but I don't think that's what he means.

I didn't find it offensive. It just means you have to think twice before using fancy stuff in code other people will be maintaining.

#8 Google Apps

The evils of threads, and how to replace them with something else to use multi-cores, is very high on the list of blogged about programming topics.

They may be high on the list of blogged about topics, but in places where performance really matters threads are used more and more. If you for example look at the article http://lwn.net/Articles/357658/ you'll see that Google uses thousands of threads on 16-32 core machines and in fact is making changes to the Linux kernel to make it support a larger number of threads.

"Prediction #10: In five years' time, most programmers will still be average."

In order to sleep through the night, I'll just assume that he's not talking about me.

#1: XML DB has nothing to do with NoSQL. Even if there is some external similarity with document stores actually the main point of the author is that people don't want to perform conversions. This was not the reason NoSQL started at all and not our main problem with SQL in the latest years.

#2: This has nothing to do with Github or hosting of source code. Here the author was referring to sell the service of running a web application for you, like a forum or a wiki, and so forth. something like vanilla forums or wordpress.org are doing indeed. IMHO this prediction was more or less totally wrong, as the latest five years showed us that web applications are a big business but companies will try to rewrite from scratch the product they are offering in order to take it closed source, to offer more in terms of usability, and probably even for scalability concerns.

Actually the reverse happened, for instance Reddit instead to use a Digg clone wrote the code, and later released it as open source.

The important thing to remember when grading predictions is that most predictions are about trends. So the important thing isn't whether "The Year of MobilePhone Computing" was 2008 or 2009 or 2010, the important thing is that it was still a few years off from 2004. It's the same thing as the "Singularity" guy's predictions. When you're predicting things 6 months in advance, you should get graded on a lot harder of a curve than predicting things 6 years in advance or 16 years in advance.

I think he nailed it with prediction #7. Many of you are saying he was late because the iPhone was released in 2007. Well, the iPhone SDK wasn't released until 2008 and the industry didn't step up to the game until 2009 when a bunch of Android handsets started to come out.

I do think he was wrong on most of the others, though.

It sucks he didn't put a date on that post. Kind of an egregious oversite, right.

Not sure if you're being sarcastic, but the date is right at the bottom of the post: "(Published Nov 10th, 2004)"

I think #10 is pretty much a truism (but still important to remember!).

So basically he didn't get anything but #6 (facebook) and #10 (which is more of an insider joke than a prediction) right. Remember this next time you read Yegge's rants.

facebook was already out in 2004, and everyone was on myspace.

Good point. That wasn't hard to predict.

Remember this next time you read Yegge's rants.

Well, even deep understanding doesn't imply the ability to predict, unless chance plays little role -- a stipulation which is manifestly not the case, here.

Edit: grammar

Deep understanding should discourage one from making predictions, though.

Depends on your goal. If your goal is to avoid making statements that are wrong, then sure. If your goal is to be interesting and say provocative things, maybe not so much. :)

I'm surprised to see that I'm getting a negative response to this comment. Would someone care to explain what makes it so controversial?

Well, you're committing an ad homenim, for one thing... this article could be completely, 100% wrong, and it has no affect on the correctness of his next one.

But I also didn't downvote you, so I can't speak for those that did.

I don't think he offended Yegge in any way. It's a valid point to make when someone goes to make inaccurate predictions: you have to take that into consideration for future articles. But well, this is Hacker News these days.

> It's a valid point to make when someone goes to make inaccurate predictions: you have to take that into consideration for future articles.

Nope, that's a logical fallacy. Lots of people do it, yes. I'm guilty of it myself. But that doesn't mean that it's correct.

http://en.wikipedia.org/wiki/Ad_hominem

It is not a logical fallacy nor ad hominem to take into account the number of times someone has been incorrect about a specific subject matter in order to estimate how much one should weigh the probability of said individual being correct on future related matters. That is, unless one has complete and correct information about the future.

I agree with you, it certainly makes sense in my opinion to question the credibility of predictions based on a history of bad predictions.

For example, I am ridiculously bad at estimating how much time implementing features / fixing bugs would take. Most of the time I'm really off, at least by a factor of 2. Surely my manager should take this fact into consideration when asking me how much time something would take me when planning the timeline/schedule (he does, in fact).

I wouldn't call this an ad hominem attack. You've got to be able to say that someone is wrong (even that someone is often wrong) without being considered to be attacking them personally.

You can say someone is wrong, sure. That's not the issue. The issue is disregarding what someone says because of previous things they've said.

> So basically he didn't get anything but #6 (facebook) and #10 (which is more of an insider joke than a prediction) right

Okay, you're good...

> Remember this next time you read Yegge's rants.

Logical fallacy.

Why logical fallacy? I didn't say "ergo Yegge is stupid" or "don't trust a word the man says". I just meant that it's worth keeping this in mind when he engages in future-telling again. FWIW I do enjoy reading his essays, even when I think they're wrong.

Reply on news.ycombinator.com