HNHacker News
TopNewBestAskShowJobs

jlivingood

209 karma · joined September 15, 2014

submissionscomments
jlivingood··on A monopoly ISP refuses to fix upstream infrastructure
For folks following this here - node and neighborhood look clean. The issue appears to be a failing cable modem (MB8611 - they seem to suffer frequent disconnects when they fail). We have suggested the customer replace the modem ASAP and we will re-run all the DOCSIS diagnostics. Suitable alternative COAM modems are listed at https://old.reddit.com/r/Comcast/comments/1kg99v3/buying_you...

- Jason (I work at Comcast)

jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
Several D3.1 modems support it now but most will need to be updated. Many of the vendors have been testing at quarterly L4S interop events, so I would expect them all to have production grade s/w next year.
jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
UTP is intended to be less than best effort priority. This is about all the apps trying to share queues at best effort (1 level up).
jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
I wrote a simplified summary of the IETF specs FWIW: https://github.com/jlivingood/IETF-L4S-Deployment/blob/main/...
jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
It makes new cloud-based apps realistically & reliably workable - think cloud gaming and cloud AR. It also makes interactive stuff like gaming and video conferencing perform a lot better w/o lag. But really anything interactive (user & device) should be better given how many round trips it currently takes to paint a web page or stream video to handle an AI assistant (Alexa) interaction.
jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
See my comment above - doubt this will happen - rather it will become a differentiator like thoughput.
jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
100% agree - read my IETF Internet Draft for more on that. :-) https://www.ietf.org/archive/id/draft-livingood-low-latency-...
jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
> "Not sure where this leads but I guess ISPs will start charging toll for express lanes"

Doubtful IMO. I think latency becomes another competitive differentiator, much like throughput/speed is today. (this is a personal comment but I work at Comcast)

jlivingood··on Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: RFC 9330
It has definitely changed. Looking at measurements on this it seems there is only 1 major transit network doing that, and they are working to fix it.
jlivingood··on Comcast, Mozilla strike privacy deal to encrypt DNS lookups in Firefox
> How would this work?

A 1st draft of the steering mechanism just posted today for comment at https://tools.ietf.org/id/draft-rescorla-doh-cdisco-00.txt

jlivingood··on Comcast, Mozilla strike privacy deal to encrypt DNS lookups in Firefox
> Comcast sniffs / records / tracks their user's DNS traffic

Actually not only does Comcast say they don't do that (https://www.xfinity.com/privacy/policy/dns) but now has signed a contract to this effect as well, thereby meeting the same level of commitment as the other TRR operators. This means IMO that Mozilla is doing a good job leading the industry on DNS privacy and convincing many of the merits of a strong pro-privacy philosophy.

(disclosure: I work for Comcast and have been working on encrypted DNS)

jlivingood··on Comcast, Mozilla strike privacy deal to encrypt DNS lookups in Firefox
> Why would DNS have "optimized, localized results"?

Any content that is CDN-based (which is most content) dynamically responds to DNS queries based on network and geographic location - to support CDN localization. In this way, Akamai for example knows the end user is in Boston on a Comcast network and will send the recursive DNS server a dynamic response that points to a directly-connected local-to-Boston content server.

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> Why traffic injection instead of mail pieces? I mean, I open all of mine, even the 75%+ that are upsells I don't want, on the off chance one of them will tell me something I need to know.

Lots of reasons, including years of experience with response rates for particular types of messages / calls to action. Clearly one particular communications channel won't work for everyone - each person has their own preferences. One of the things we're working on is to better enable you to control just that - basically one person may ask for SMS messages, another alerts via their mobile app, another via email, another via phone call, etc. You can see the beginnings of that in MyAccount / Settings / Communication & Ad Preferences.

> But with Let's Encrypt, browser manufacturers, and friends leading the charge toward TLS everywhere or as nearly so as is practical, and with most sites that most people use already employing TLS, the attack surface is closing for even an other-than-innocuous variant of your notification methodology.

Agree. And more TLS is better IMHO. I also like the work that Let's Encrypt has been doing - they've had a really big impact on the adoption of TLS. (See also http://labs.comcast.com/innovation-fund-spotlight-lets-encry...)

> Of course, that also means that that methodology itself is reaching a natural end-of-life, as it cannot work anywhere that TLS exists, and the majority of the web where it does exist continues to grow. If this low-latency notification scheme is of unique value to your business, then now is the time to consider replacing the outdated technology that underpins it with something which will continue to work reliably over the next decade or two.

You bet - totally agree! One of the places we're engaging to try to do that is in the IETF's CAPPORT working group and I think the charter describes reiterates all the points you made: https://datatracker.ietf.org/wg/capport/about/

> All that said, I appreciate your decision to engage in this forum. That's unprecedented in my experience from someone in a position like yours, and I wouldn't mind seeing more of it.

My pleasure & thanks for being a customer that's willing to offer constructive criticism. :-)

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
It usually means you are about to get a speed upgrade that will go beyond what your modem is capable of delivering. In that case it is possible you could have a 1st generation 4x4 modem (so it can bond 4 downstream and 4 upstream channels).
jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> They emailed my Comcast.net address, which I didn't even know I had.

I recommend you add your primary email address. You can do this via the self-service portal.

Go to https://customer.xfinity.com/#/settings/account under Account / Settings / Contact Information. IIRC you are sent a confirmation email you have to act on before it takes effect.

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> edit: Proof https://imgur.com/lzKBkMs

If you look at the far right device you see a non-EOL SB6121. The one on the left that is EOL is the leased one, and the retail one is still allowed. I'm not sure if you have a leased device or retail device.

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
An RFC is not always a standard - often they are simply 'informational'. For us, when we wrote the document, it was a way to document as transparently as possible how the system worked so that folks would not need to speculate about it and for us to explain the rationale and alternatives considered. This seemed to me at the time far better than being evasive about it. And a request for comment is often a way to solicit exactly that - good comments (e.g. suggestions on alternatives). In this case, it has led in part to things like the IETF's new(ish) CAPPORT working group being created to develop a better Internet-wide standard for how to interact with so-called captive portals. See https://datatracker.ietf.org/wg/capport/about/ for more details and feel free to join the mailing list and contribute!
jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> I sincerely disagree, especially as per the report Comcast's own second level confirmed there was no need to replace the modem.

I am skeptical of this - maybe we made a mistake in telling the customer that. The people that are sent notifications are carefully checked to match the EOL/EOS modem criteria or speed mismatch criteria and would not be sent otherwise. It is sometimes the case that a customer has recently upgraded their device but their old device remains provisioned and on their account (and needs to be removed), which sometimes explains this.

> It was an automated advertisement done in a very not good way;

It was not an ad - it was a request that the customer replace/upgrade their device. They can buy that anywhere, whether used on eBay or new on Amazon, etc.

> Comcast's own billing system notifies you of just about everything else; you can forward your billing statements and other such information to other emails, why not this?

We've been working to greatly simplify billing, as customers have told us for some time that we were packing too much info into those statements and it was sort of information overload.

> The reason everyone is freaking out is because they feel pretty darn strongly that the ISP should not be injecting code into webpages delivered,

Available alternatives are not great, such as using DPI everywhere, DNS modification (we use DNSSEC), or a walled garden (all service disrupted while in walled garden). These methods tend to be more costly and cause more disruption for customers. As noted elsewhere, we're working on better methods and part of that might depend on Internet-wide standards rather than something Comcast-specific (which is always my personal preference).

> If this is to be a service, the bar for what is necessary for such information must be far higher than "an automated system decides it's time." We get into really scary territory just by doing this in the first place, but to use it for advertisements or basic maintenance? That is a misuse of such technology.

It's not basic maintenance - that should always be transparent to customers. This is about moving to new technology from outmoded technology. A good example of a key concern for modem upgrades is that the vendor does not support it any longer and the software/hardware is 8 - 10 years old.

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
What I meant in the context of that exchange was that the notifications come only after for example multiple emails have not resulted in the device being replaced.
jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> HTTPS everywhere is excellent advice

I couldn't agree more! That's one of the reasons for example we have supported groups like Let's Encrypt (http://labs.comcast.com/innovation-fund-spotlight-lets-encry...) and CrypTech (https://cryptech.is/).

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> Exactly. And the response, "we're not trying to sell you a modem, we're just encouraging you to strongly consider buying a new one" is such a hair-splittingly asinine response considering the rather serious breach of trust posed by the notification system.

Well, what I meant (within the response length constraints of Twitter) was that we're not saying you can only buy it from us. Just that the customer needs to buy it someplace. That way a customer can do as the wish - ranging from buying a used one on eBay to getting a new one from Amazon or Best Buy.

Ultimately the objective is to ensure a customer is on a device that can (1) deliver the performance for which they pay and (2) is up to date technically (i.e. supports IPv6 and channel bonding) and is supported by the vendor (i.e. software updates & bug fixes).

One of the big risks we have to help mitigate is when a device goes EOL, which means no more software updates, and a security or significant performance issue arises in the future. By proactively beginning the replacement process this helps minimize any future impact when it is a major issue like that. So taking action gradually on a proactive basis prevents a more severe impact later on. In many cases, these are DOCSIS 2.0 devices and that technology and often the software is from 2001, the same year as the 1st gen iPod and when Windows XP was released.

Eventually a modem will go into End-of-Service (EOS) status. At that point there is a definite date/time limit for the device, after which it is de-provisioned from the network and the customer must replace it to continue service. This has been the case in the past with DOCSIS 1.0 and 1.1 devices for example, after years of work to encourage customers to replace them.

See also https://www.xfinity.com/support/articles/end-of-life-devices and the start of the EOL/EOS process for DOCSIS 1.1 devices https://www.dslreports.com/forum/r27473499-Speed-Heads-Up-Ti... and https://www.dslreports.com/forum/r28497383-Speed-Upgrade-You... and https://www.dslreports.com/forum/r30524429-Equip-Reminder-Pl... and https://www.dslreports.com/forum/r30450278-Speed-Heads-Up-Ti...

jlivingood··on Comcast is injecting 400+ lines of JavaScript into web pages
> Although I disagree with Comcast's method and categorization, it would be interesting to learn what modem the OP was using.

We start telling customers that a modem needs to be upgraded when one of two things happen: either they are about to or just had a speed upgrade that their modem cannot support or the modem has gone end-of-life (EOL) from the vendor.

In the former case, if the device is leased, you are send a new one to replace the device and just have to basically say ok. In the latter case, it is a customer-owned device so the customer is asked to go buy a new one someplace (e.g. Amazon, BestBuy).

And in the EOL case, the vendor may have gone out of business or shut their cable modem business down, or otherwise decided to no longer support the device due to its age. That of course means that if a security issue came up, as they do, that the vendor would not be able or willing to provide a software fix for the device. So it's best to get the ball rolling to get those devices replaced when that occurs. Most of our EOL devices today are DOCSIS 2.0 devices (10+ years old), which can only do a single upstream and downstream channel (no channel bonding) and 1st generation DOCSIS 3.0 devices (5 - 8 years old).

jlivingood··on Setting the Record Straight on Tor
Great quote!