Apple now have their own mapping effort similar to Google and I remember watching a video at a Google sponsored event where it was mentioned that for self-driving cars to be viable, you first need a mapping solution which Google had been building for several years as Google Maps.
Anyway, I wouldn't count them out just yet. It took them 5-6 years to catch up to Google Docs and these days, I often consider using Office 365 because it's local AND syncs to the cloud automatically.
The lithium ion battery problems are mostly under control now.
The reality is Google Fiber exists to scare the existing players away from squeezing too hard, but not as an actual plan to create a large profitable business to deliver actual competition. Google has orders of magnitude more capital than they would need to do that if they wanted.
Also another reason they may worry, is that maybe phased-array antennas may become competitive soon, so investing on fiber for the long term might be risky.
On the other hand, stuff like self-driving cars should have a different competitive/economic story, so scaling may look different.