Visible signs left after Google Fiber abandons Louisville
gizmodo.com
gizmodo.com
"No", he said. "We'll sue them for every mile of fiber they lay down".
I can understand why they would make that play. It's kind of the only play they have. AT&T and Verizon innovate enough to remain...okay...but they're not exactly Google. The only way they would win is to get dirty and turn fiber laying into Google's very own Stalingrad. And between starting a new, massive, low-margin (by advertising's standards), and capital-intensive business, and keeping salaries high and employees happy, Google chose the latter and bowed out because honestly, dying on this hill isn't a smart move.
I doubt Google wants to do this. Pulling out at this stage, when Fiber hadn't actually provided the "nudge" to internet speeds and throughput that would make Google and Internet companies more powerful, would not place Google at a place of negotiating strength. It makes Google look bad and cements the ISP monopolies' notion that Google pulled out because they physically can't bear the pain and so they remain safe, not because Google chose to, can do this whenever they want, and have the ISPs at their mercy. It probably also burned a few bridges, if there were bridges at all, and makes weighing favors in a (hopefully not long-term) post-net-neutrality world swing towards the other major power players like Amazon or Facebook.
Is there enough demand already generated in industries requiring significant network throughput (e.g. e-sports, industrial IoT, or elsewhere) that ISPs will shoot themselves in the foot if they collapse the market? I personally don't use enough data to justify a Gigabit connection. My friends who stream video and game online might. Hopefully the latter will pull us into the future, but I don't know for sure.
I can answer that: they won't. ISPs showed they have the capacity to expand and improve and bring better service to people when they're challenged to do so, but it is equally obvious that they give not two fucks about anywhere they have no competition, which is basically everywhere.
The Google of the past that did cool stuff was our one ticket to disrupting this bullshit and getting it fixed, and Google the corporate big business machine of today killed that hope.
And if you think it'll get better as other parts of the world continue to pass us by, all you have to do is look at our health care system to see how that works out.
That doesn't seem to mean anything when all the company cares about is next quarter's stock performance.
When I moved to Seattle, Verizon had recently abandoned its fiber network there, and my neighbors were so mad about it that they didn't trust any ISP.
I tried to Google it, but Big G disappoints. Perhaps someone who lived in Seattle (especially the East Side) and remembers this can shed some light on it.
If anything, Verzion plowed a ton of money to run fiber to most homes, then saw poor uptake and a small telco they could dupe into taking on massive debt (Frontier). This has forced Frontier to offer good deals like 200Mbps/200Mbps for $40 a month. Centurylink in the city proper starts at $45 for 40Mbps/5Mbps via fiber, and Comcast is $30 plus fees for 60Mbps with a 1TB cap.
Same thing happened when EPB tried to expand their fiber network in the Chattanooga area.
Which is why the cable cartel is lobbying at state-level to make deny cities that option - conveniently, those cities are in red states.
They've got a significant advantage over Google here. Where they run DSL, they've got a customer base, and an installer base. They already have rights to the poles. They've already got contacts in the municipalities to add their new equipment boxes. They worked out how to order the fiber spools with connectors placed for the splitters at each pole. It's just a matter of buying and placing the equipment, upgrading their existing customers, and then winning customers from cable.
For customers with underground copper wiring, it's a tougher sell. And there are clearly parts of the AT&T network that AT&T doesn't care to service; places that never got ADSL2 or VDSL (aka U-Verse) likely have pretty low customer numbers these days, and it seems like they'd probably not get upgraded either.
I’m surprised that AT&T doesn’t just try to transistion their few remaining DSL customers to the cable company. I’m sure that the cable company would offer a bounty for them and keeping the internet backend must cost AT&T something.
But what about all the other cities where there was no such law suits? What about cities like Baltimore, that offered to roll out the red carpet for Google, but which Google turned down? Did law suits in two cities actually halt Fiber expansion, or is that just a fig leaf for Google abandoning a business line that will never make the kind of money it demands of its business units?
I hate to admit it, but we haven’t had any issues with AT&T gigabit ethernet service. It’s $70 a month with no additional fees, rock solid and unlike Comcast, we actually get great (900Mbps) upload speeds.
So did you sneak into almost any room in my gigabit wired house and do a speed test?
I have a wired computer in my office and two 4K AppleTVs and they all report 900Mbps+ up and down.
My son’s wired PS4 reports around 600Mbps.
I also bet that AT&T won’t abandon my neighborhood and become uninterested like Google does for basically everything in three years.
Are you saying that they’re are ISPs that never oversubscribe their lines anywhere along the route?
And of course, Google Fiber like AT&T is an oversubscribed consumer level connection. Each use shares a 2.4 gigabit up/1.2 gigabit down GPON with some number (typically 16) other users.
When companies can sue other companies for providing them with actual competition, you know customers are getting so utterly screwed. I don’t even understand how this can be legal.
So glad I don’t live in the US.
It doesn't change much, but you left out Verizon and Spectrum.
It's a dirty racket for passing a lot of money between few hands.
They lay their own fiber terminating at your home and have been steadily covering the metro area.
I am speculating it is capital intensive and not enormously profitable, and it is slow.
That is also how Stockholm built out its fiber network. And, by the way, it is illegal in almost every city, because it would result in wealthier neighborhoods getting fiber first. That is the reason, for example, why Baltimore doesn’t have FiOS whole almost the whole rest of Maryland does.
Stokab also had various mandates to connect or cooperate with schools, government, student housing, public(ly owned) housing and other entities. You would often get fiber in the equivalent "the projects" before someone in an affluent co-op. Without this I am not sure how well the approach would have worked.
I led the team that worked on a bunch of software that supported the effort (behind the scenes and consumer-facing), and am glad to see other cities following—hopefully others will also copy EPB’s commitment to net neutrality and no caps. We’ve long had the same price (a couple bucks under $70) for the gig. Lowest offering here is symmetric 300meg currently. We even do 10gig to the home now.
Unfortunately, not everyone is having the same experience. The perception here is that they are operating a shell of what the previous ambitions were. Service has been active in my area for close to 3 years now, but I have neighbors who have been on waitlists for months or years to get their house on the network, still being told to call back every few months.
It seems like any houses that missed the initial sign-up (now many years ago) are a crapshoot to ever receive service in a timely manner. Some streets that didn't receive enough sign-ups were never trenched, while adjacent streets were. Sometimes fiber is installed all the way to the house with months-long delays connecting to the network (splicing issues, conduit clog, etc). I've loved my service and it's been rock-solid, but there are certainly a lot of obstacles compared with traditional telecom providers.
https://www.traffic-signal-design.com/vehicles_detected.htm
The trench is narrow, cut with a diamond saw, and the filler (tar?) doesn't come out.
The example photo of a failure in the article, showed a relatively wide trench, presumably width required for the microduct for blown/jetted fiber.
So maybe they used a trench that was too wide, and a filler that was unsuitable.
I have also seen microtrenchs used in New Zealand across private driveways (low traffic use).
I wonder how good the trials were of the filler? I sometimes see paint tests on the road (multiple lines of different product and different thicknesses, being tested by heavy, real traffic use).
Edit: Looks like it didn’t actually work in those cities either. One wonders why the Louisville representatives missed this.
https://www.wsmv.com/news/repaving-streets-reveal-google-fib...
Along with the rest, this outlines a basic problem that modern Google and other Silicon Valley companies have with technologies. The Google people have massive talent, capabilities, and funding. Yet the use this almost as a handicap nowadays. Because they have talent doesn’t mean they have a monopoly of it.
We have extensive testing of every conceivable paving and repair mathod done over 100+ years. Engineering colleges around the U.S. (and the world of course) have a massive legacy of tests. To go in and just guess your new idea is going to work for a new project just isn’t necessary in such basic technologies.
Epoxy even sounds like a stupid idea. I mean how are you going to repair that next time. It’s impossible to dig that stuff up.
On the fiber side, I think cities need to build the infrastructure (and are often in a position to finance via low interest rate bonds etc), and then enable any party to lease/operate/compete on that infrastructure.
I almost always get in the ~700-800Mbps up and down range on ethernet connections, regardless of time of day. Never had any issues during snow, rain, fog, etc.
My guess is that Google realized that fiber-to-the-home is the wrong strategy, and too costly, just as Verizon FIOS halted their rollout years before. And I believe even ATT only does fiber to a neighborhood and then coax to the home, right? It's sad to see, but the cost is astronomical and it's a lot harder to get cities/municipalities to let you dig things up.
Especially for America with it having so much sprawl, a wireless solution will probably work better and the rollout will be quicker.
No, I have a fiber going right to my dmark. I watched them string it and terminate myself.
This seems like Google has not yet learned how civil engineering works, which is a crucial component in internet infrastructure.
It's unfortunate that Louisville is getting bit hard here.
https://techcrunch.com/2018/07/17/google-builds-its-own-subs...
Probably not, but common sense should be enough. It is 2 inches, on a road...
General overview of micro-trenching: https://www.ppc-online.com/blog/best-practice-for-installing...
It is a lot deeper than 2 inches.
Google tried to do the backfilling/slot filling with some sort of rubber compound or something that was not proper pavement grouting.
Google and their contractor fucked up the OSI layer 1.
The way it's done with the "teraspan" (company name) method in Vancouver BC is not economical to service individual residences, or small buildings, it's a set of networks built for several different companies' last mile fiber access networks to service major class A office towers in the downtown core. The diamond blade concrete saw + microduct + cable + slot filling setup is way too costly in dollars per meter of installed fiber to be used for something like Louisville.
Wireless is slower, has higher latency, is less reliable (interference/obstructions), etc. But you can put in one tower and service every customer for miles under those conditions, which is much cheaper than wiring every building.
It's possible to address those problems with wireless, but the way to do it makes the wireless network look like the wired one. Instead of one tower serving many people, you have very many towers each serving a small number of people. But then the cost benefit evaporates. You need a tower on every street corner which has to be wired with fiber.
At that point you might as well just hook the existing phone or cable wires already going into everyone's houses into that box on the end of the street in order to get the last few meters, and save yourself a few billion dollars in wireless spectrum.
This all varies based on the wireless tech used. My apartment in Seattle is serviced via point to point wireless with another building across the neighborhood. Currently I see under 2ms RTT to google.com, and can max out the 100mbit port on my router both up and down. (Gigabit is available, but requires an account with WaveG. 100mbit is free with my rent, and on the landlord's account).
Aside from a misconfiguration on my building's switch that allowed a neighbor to act as a rogue dhcp server, I've yet to see an outage in the two years of living here.
Wireless doesn't have to suck.
Moreover, the reliability you get has a lot to do with what happens in your neighborhood. Point to point wireless is great until somebody builds a new building in the path between you and the other wireless device.
That’s simply not true, and it becomes obvious why when you account for fan-out. There is a reason why independent ISPs often do MDU installs. Running fiber to one point where you can service 100 customers is vastly cheaper than running fiber to 100 houses.
You would only need to bring fiber into the home for the users that need the full performance of fiber. But then fiber is the only thing with that level of performance. You're not reasonably going to get better performance out of wireless than you can get out of coax.
There's a reason rural Internet sucks: It's expensive to build and unprofitable to operate. That's why ISPs don't want to build them out.
Nobody actually cares about the rural poor. No transportation options, limited services in general. It's just laughable how people consider themselves so giving, unless those people live outside the city, then screw those people.
The Trump campaign advocated for it in their 5G-nationalization scheme that the Trump White House for some reason wasn't aware of ([0] ?) and likely wouldn't work because of the nature of 5G penetration.
The problem is "broadband for all" is it's an issue that doesn't register as much as the culture war.
[0] https://www.politico.com/story/2019/03/03/trump-administrati...
https://en.wikipedia.org/wiki/Ultra-Fast_Broadband
Covering 87% of the population by the end of 2022. It is a public–private partnership of the government with four companies with total government investment of [USD 1 billion].
It was rolled out below budget. Then again, look at Australia trying to do the same thing, and failing (incumbent telco and govt), which might be more similar to what would happen in the US.
Size, geography, and population of NZ is somewhat similar to Oregon, if that helps. As written by someone from Oregon: "When it comes to Oregon and New Zealand, there are similarities galore. Both have a largish city in the north corner. Both have around 4 million in population. And the square mileage of their land masses differ by only 5000 (98000 and 103,000, respectively). The scenery is spectacular. Micro-breweries are aplenty. And Kiwis “get” coffee. They really do. In other words, I feel at home here."
$ Due to a wacky series of misunderstandings, both of these ISPs service my building now that Webpass is gone. They periodically send out sales reps who bring snacks and wine.
This is indicative of nothing more than Google behaving consistently.
I don’t necessarily disagree with your point but your experience may be on the rare side.
All the railroad right of ways are loaded with fiber, the freight tunnels are loaded with fiber, etc.
If you are downtown, you can add the half dozen or so fully competent local ISPs. These companies sell internet to condo associations; every unit gets a dedicated connection and the common areas get shared wifi. The cost gets added to the monthly assessment. Where I live it's $25 per month for 500 meg symmetrical.
I work in a 1920s skyscraper in the Loop. Cogent stops by at least once a year and puts coffee and (expensive) donuts in the lobby. They'll run fiber directly from their backbone to a server rack on whatever floor your office is on.
Railroad right of ways are largely immaterial to last mile home seevice.
I now work at a different fiber ISP. The industry is fine, it's just that Google sort of gave up. Even if they went full-on "we changed our mind" after I left, it's not the original team anymore. A lot of knowledge has disappeared into the void. You hear "invest in teams, not in products," around here a lot. That's not what Google did.
You simply can't promise gigabit class last mile service to every residence in a city, and actually build it with a reasonable dollar-per-house cost, and make a return on investment on it. Not unless you have an incredibly low cost to deploy it, basically a city and local electrical utility that has handed you ROW (right-of-way) on a silver platter. Or unless you are already the incumbent telecom operator, and have ROW literally everywhere, and can overbuild your POTS network with modern GPON stuff, as Centurylink is doing in parts of Seattle.
Or unless you bring the dollars-per-unit cost way down, by focusing on MDU (multiple dwelling unit) such as WaveG and Webpass have done.
It costs a lot less to light up an 85 suite condo building with 1000BaseT to each suite, than it does to build last mile GPON or active ethernet fiber to 85 individual houses.
I never saw the Google Fiber cancellation as a cost issue. It felt to me like more of a scale issue. You have to live in Kansas City to pay Google for fiber, but you can buy an Android device from anywhere in the world. Thus, one of those things is going to make more money than the other, and it makes sense to focus on that.
The economics of ISPs, the way I see it, depend on:
1) Cost of money. (Free if you're Google, 18% a year if you're some guy using your credit card.)
2) Expected lifetime of wireline Internet. Wireless internet isn't getting _worse_ every year. While a lot of us here won't be giving up our wired connections any time soon, many people already have. Their phone is their computer. They can't even plug an Ethernet cable into it if they want to.
If your fiber is going to be good for 50 years... after you've paid off the loan, it costs basically nothing to provide Internet service over. And yet, people will pay more than $0/month to have Internet service. So economically it works out... if fiber continues to be relevant. That is the if that scares people. (The other issue is that it's hard to find free money unless you are, say, Google.)
In developing nation environments where there is no pre-existing wireline infrastructure, and incomes are much more limited, it's now incredibly common for people to only have a phone with LTE radio in it, and no fixed internet at home. Examples would be in Rawalpindi, Pakistan or in small cities in Turkey.
But I don't see people downloading 90GB xbox one games on LTE last mile connections any time soon, even with carriers' advancements in fixed LTE and coming "5G" stuff. The capacity of last mile wired connections will always be much greater. And the typical residential consumer in the US or Canada has much more ability to pay a $75 monthly recurring bill for proper home broadband, on top of their cellphone costs.
And discussed here at the time: https://news.ycombinator.com/item?id=19106998 (207 comments)
If anyone suggests a better one (i.e. more accurate and neutral), preferably using language from the article itself, we can change it again.
What does that mean exactly?
I would think getting more competition would be welcome.
Louisville should have looked at the model set by Chattanooga's EPB. Fastest internet in the country and a highly rated customer service team. Their roads suck too but it's not the ISP's fault.
A couple of months later Time Warner upgraded my cable modem speed from 15Mb to 50Mb download for the same monthly price.
Then 2 years ago AT&T came through my neighborhood installing fiber and turned it on 3 months later.
Google Fiber is concentrated in a few small areas. AT&T Fiber is expanding all over the area. I know 1 person with Google Fiber and about 15 with AT&T Fiber.
I would prefer Google over AT&T but the AT&T Fiber works and is reliable and Google is so slow at expanding that nobody cares anymore.
The moral of the story: beware geeks bearing gifts. Google should have been put into a contract to only field-test shallow trenches, if they committed to remediation into deep trenches at their own buck.
"they would have gone somewhere else" is the voice of Amazon city bidding nonsense: this is public utility infrastructure. They ballsed it up, they should have been contractually obligated to get it right.
If it was a testbed freebie, different story. You let somebody in to test something for free? Sure. Public Liability? sure. Disappointed customers? Sure.
Hang on.. who are they disappointed in? the elected Lousville Officials?
It's so weird to just use cities as A/B tests and just disregard all the people and plans built around a failed case at the drop of a hat. Are they going to start A/B testing countries against eachother next?
It would have risked much less bad publicity if they had done the 2" experiments in a city they were willing to use 6" trenching in. Do part of the deployment with 2" and part with 6", and if the 2" part doesn't work redo that part with 6".
I believe YouTube also experimentally surfaces new recommendations slowly to see how many clicks they get and if it should be surfaced more or not
https://i.stuff.co.nz/technology/social-networking/103511194...
It's much messier when it's accidental, but ... wouldn't it be smart for say, new state propositions in California to be tested in a trial to verify they produce the consequence voters asked for?
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
1 192.168.1.1 (192.168.1.1) 0.394 ms 0.383 ms 0.470 ms
2 * * * (*) 2.460 ms 1.754 ms 2.599 ms
3 10.5.1.9 (10.5.1.9) 1.518 ms 1.424 ms 1.614 ms
4 206.51.46.41 (206.51.46.41) 1.926 ms 2.240 ms 2.138 ms
5 one.one.one.one (1.1.1.1) 1.756 ms 1.945 ms 2.015 ms
I know lots of people now who use wireless plans at home - they whine at the end of the month when their cap runs out but won't switch because it's cheaper (monthly cost similar, but free signup vs $200 install fee + router purchase).
So 100% yes, as a business it makes little economic sense to build a consumer FTTH network. I'd still like to see governments and incumbent telecoms do it though as an investment in quality, stability and diversity. I feel a lot more comfortable replacing the POTS network with fiber than wireless.