https://4.bp.blogspot.com/-GLc5ve5_djc/V-ysZgW6uDI/AAAAAAAAD...
The worst case scenario was 5x, which was a factor of 10 off. If you are that far off when you do your capacity planning, you can be pretty sure you've got problems throughout your entire stack.
That said, what you said is still true. If you're thinking you're going to have 100k users, you might be willing to allow a lot more data to be transmitted and/or processed than if you had 10m users. Just looking at tracker alone, having to transmit and measure distance between a dozen points and ordering them is a lot more work than checking whether a dozen coordinates are within a range and listing them in any order.
I don't believe this for a second. Nintendo knows where its goldmines are. IMHO it is far more likely that they steered clear of mobile because they didn't want to cannibalize their Game Boy sales. It's a classic case of a big corporation being slow to react to change or even trying to stop it because they hold a dominant position in the old system.
Nintendo would much prefer if "mobile gaming" still meant Game Boys but the world has already made their decision and they can't stop it. They can't ignore the sales figures on what would have otherwise been an obscure goofy spinoff fitness app on the Game Boy. They've ignored the mobile market for too long and now there is pent up demand. Without Pokemon Go I doubt you would have seen the sudden scramble to develop and release a Mario game for iPhone.
I'm not sure this is entirely accurate. The Nintendo 3DS (I assume what you mean by "game boy"), has been a very successful product for Nintendo. And sales of the 3DS have actually spiked as a result of Pokemon GO.
The reason Nintendo has avoided mobile gaming is probably down to fear of piracy. The DS was ruined (from Nintendo's POV) by flash carts, and with the changes they made on the next generation the barrier to entry for copying games is now lower on Android/iOS than on the 3DS. I think that when they do finally go mobile, the games will have a lot of server-side checks going on. The games will be a pain in the neck to play even if you are paying. For example, always-on internet required will only work on non-rooted devices, that sort of thing.
The Niantic team did incredible things given the instant historic success that became Pokemon Go.
The generic 1X, 5X and 50X are hard to understand in this context I think. Pokemon is so popular it's very difficult in imaging what the real numbers actually are. For instance on launch day in America and Asia I would have expected insane numbers (many hundreds of million).
I also feel like the 1X, 5X and 50X number placeholders are useless in this conversation because it doesn't give a sense of scale at all.
Given no app had ever done this in history over the first few weeks, yet alone on launch day with no marketing, I doubt it.
But in all seriousness this is Pokemon; they sold over 3 million copies of a remake of a game on a niche portable console in just 3 days; elevating its brand to an open platform for FREE that can be downloaded and installed on theoretically, what, a billion or more devices? Seems incredibly doable for Pokemon. Few other brands could do the same. Even Mario wouldn't be able to come close to competing with Pokemon's brand power.
I would be very interested in the real numbers :)
If this doesn't clearly show how crazy this growth was compared to what anyone was expecting, I'm not sure we're operating in the same plane.
"Many hundreds of millions" implies at least three of them.
I would not expect the entire population of the US to be playing on day one, especially the roughly 50% of them that don't even own a phone.
Sigh. This would have been easier if we had real numbers. Let us all test our hypothesis. Oh well.
The 2 lines can be thought of as ceilings or upper bounds, hence why they are static - this are the numbers that traffic was expected to eventually reach at peak.So you can think of it as, "we thought we'd be looking at graphs that had this line as the top and traffic would be some curve underneath.
Obviously from the graph shown here, we/they needed a tall graph.
I think this is the disconnect. This wasn't not obvious to me. It looked like everyone was expecting flat growth at either 1X or 5X. If there was a line showing what they thought would be the traffic that goes up (which would be expected) in addition to the ceiling lines then I think there would have been a lot less confusion.
I wonder if a single bar graph would have illustrated it better with overlaying colors for each ceiling.
Question: When should we start checking around for posts containing
A) High-level technical overviews with some basic implementational detail
B) In-depth analyses of the stack you built, the challenges you faced, what improvements you folded back into various open-source components, what you'd have done differently, etc etc
?
I'm thinking in terms of timescales - like n months or so. I suspect (A) will be a little easier (and quicker?) to publish than (B).
The article says (in one of the only bits of real information) that they blasted past their estimates with only Australia and NZ just 15m after launch.
Whoever came up with those numbers must have had some serious methodology flaws. I know they couldn't predict that it would become the biggest online game ever for a while, but the initial demand prediction was clearly way off even before it started growing like a rocket.
Pokemon Go became the most-downloaded app in history within the first few days.
Are you saying that's the bar that people should estimate at launch?
>Throughout my career as an engineer, I’ve had a hand in numerous product launches that grew to millions of users. User adoption typically happens gradually over several months, with new features and architectural changes scheduled over relatively long periods of time.
When it comes down to it, the game is closer to a fitness app than a Pokemon game and can be described as bare bones. The fancy accessory wasn't even available at launch. Many of the players are people who have not bought a Pokemon game in over a decade, or ever in some cases.
If there is one area I can solidly criticize Niantic for it is rushing the other region releases so quickly. It's clear the servers were already badly overloaded and they just started adding countries left and right. I know it sucks for people who live in those countries to have to sit out while the rest of the world is having fun, but it's really not much more fun to sit at the eternal loading screen because the backend is entirely on fire.
And you're sitting there as the engineering lead or staff going, "How do I even feel about this? Fortunately I have no time to feel because I am off to fight fires." I didn't go home for 2 days, I worked 82 hours that week and >70 the next.
Complaining that the estimates are bad for a product that literally broke everything we know about how to build a successful mobile game and has scaled to a truly unprecedented level is meaningless. Obviously no one expected this. Obviously the engineers wouldn't have wanted it. Obviously the world will respond the way it will to our work.
Show some compassion. But also some humility. None of us are qualified to make projections in the face of phenomena like this.
You can't predict it will be the biggest game ever, that's not possible. But I feel like their initial estimates were still too low and you could predict it would have been higher. I don't know if it was based on how well Ingres did but...
Pokemon is a huge property. It's had tons of games, movies, sleeping bags, an incredibly successful trading card game, etc. Just having the Pokemon brand on something makes it VERY big.
In the game, you live out the Pokemon dream. This isn't just Pokemon Puzzle League. This isn't Pokemon Mystery Dungeon where you navigate cute little Pokemon around and play a top-down rogue like. You FIND AND CATCH Pokemon in the wild. It's exactly what Ash did in the TV show or comics.
Also, Pokemon are cute as hell. That plus the novelty of the AR stuff meant this had a lot of potential. "Look, I found a cute Eevee over here on my potted plant!" Those pictures were EVERYWHERE. That's tons of viral advertising.
But there is also the in-person effect. You want to compare what Pokemon you have with other people, and that encourages you to get your friends into it. But people were walking outside with their phones playing the game, and they quickly got spotted by people asking "What are you doing?"
All these things make it clear to me the this game had a high chance of success.
There's no way to know it would go to 50x what they guessed or would top the charts. Given their numbers I wonder if the expected should have been closer to 7x and the worst case at 20-25x.
The popularity they got would have taken basically anyone down. That was going to happen. I'm just surprised the estimates weren't much higher.
What numbers? We know that their estimates and their realities were quite different, but not knowing the real numbers we have no way of even beginning to judge what's reasonable and unreasonable here. For all we know, they modeled directly after the most successful game in the Android market at that date as a baseline and then said, "At the worst case we'd expect 5x THAT."
You can write a ton of paragraphs about how cute Pokemon are, but the truth is that the Pokemon AR game was a massive risk. AR games have had extremely limited update. It seems incredibly unreasonable for me to expect that those folks should have realized apriori that they were about to release the most popular mobile game ever created.
I, for one, will not throw stones. I don't get why you feel the need to assert that you (or anyone) could have done a better job by setting a 20x or 25x target. Or that you could have not only forseen it was necessary, but convince everyone around you that the capex was justified.
Why are you so keen on assigning blame and shame in this scenario? Some of our peers made history. Can we be happy for them for 6 months before immediately backseat driving about how much better we all are in our armchairs?
A mobile game, available to be downloaded onto hundreds of millions of phones, that is also FREE? I feel like they broke 100 million in the first day at the latest (counting totals from each region's first day). It wouldn't surprise me if it's significantly higher than that.
Media, users; everyone has been BEGGING Nintendo to release IP to mobile devices but they have kept it locked up in their own hardware. If no one had even a rough idea of the possible popularity they most certainly had a very, very wrong methodology.
> In response, Google CRE seamlessly provisioned extra capacity on behalf of Niantic to stay well ahead of their record-setting growth.
Anyone who played that game within the first week of launch would certainly not use "seamless" to describe the experience.
You could argue that Google was providing GCE and therefore GCE couldn't scale, but GCE is really just hosted Kubernetes and it's scaling limits were known in advance. Luckily GCP was able to push a quick version update and migrate the cluster but that took considerable time, coordination and engineering effort that couldn't be done "seamlessly".
This is a clear situation where the software Niantic chose couldn't handle the resources that were available. It turns out a lot of the choices you made for your worst case capacity aren't necessarily adequate for 50x that amount :)
(disclaimer I work at Google but not on GCP)
GCE - Google Compute Engine. VMs in the Cloud.
GKE - Google Container Engine. Hosted Kubernetes. Runs on top of GCE.
(I work on GCE)
I think a large part of that was due to the non-Google login from Pokemon Trainer Club (which must have been handled by Nintendo's servers, AFAIK), as opposed to the Google OAuth. Both groups of users had problems, but people who used their pre-existing, Nintendo-issued logins had much more problems (and it took a lot longer for those to get fixed).
There was a period of about a week or so during which time people who used the Pokemon Trainer Club accounts were still having just as much trouble logging in and staying logged in, but people who used their Google accounts were fine.
I wouldn't be surprised if providing multiple login options made it harder for them to properly separate their login servers from their game servers, and if this coupling meant that millions of users (effectively) DDoSing the PTC servers ended up impacting the game's uptime more than simply doing all authentication themselves would have.
I can't think of anything further from the problems this game had at launch than the login. So few people actually had pokemon logins this comment is honestly comical to me.
"reliable"
"low-latency"