Hacker News Reader

Comment by dessant | Hacker News Reader

While the controversy centers around ad blockers, the most important aspect of the Chrome extension API changes seems to have been missing from the discussions of the past few months. You will not be able to fully control every aspect of a request in your own browser, despite installing extensions that run priviledged code that you trust.

HTTP headers can no longer be freely edited, unless they are part of a limited whitelist blessed by the Chrome team. This kills innovation in the browser extension space in its tracks, and ad blockers are just a subset of use cases that will be impacted.

The changes will inevitably find their way into Chromium forks which do not have the resources to maintain the deprecated API while merging upstream changes, further limiting choice and hindering innovation.

View on HN

I understand the challenges at scale, but there's definitely a decade-long trend towards locking down devices and software from user changes, because any API or configuration that's accessible to knowledgeable users is also accessible to malware and social engineering.

This coincidentally aligns with business goals (watch our ads, replace your phone every 2 years, sync everything to our cloud because anything else is risky and cumbersome, do not integrate our devices with a non-blessed solution etc.) but for some reason discussing these goals is frowned upon.

Correlation does not imply causation, but an event can have multiple causes at once, and only some are allowed in polite discussion, and not everything should be attributed to only malice or only stupidity.

> but for some reason discussing these goals is frowned upon

Well, the reasons are obvious - execs don't like having their motives questioned by the little people, especially when those motives are self-serving and bad. How dare you question your betters?

One reason of many I banned Google.

Mind you, another way to say that is “malicious extension authors can no longer snoop on all network traffic.”

As someone who was, at one point, pitched a project proposal for a browser extension our company could make which would “make users think they’re getting one useful effect, while secretly using their browser as a node in a distributed web-scraping farm to social-network sites they’re logged into” — I’m sure this is more common than people think, and very wary of extensions saying they do one thing while doing quite another (which happens to fit into the same permissions set as the explicit purpose of the extension, making users none the wiser.)

I'm curious if you wouldn't mind sharing what was the context around that pitch, and your subsequent response. It sounds like you didn't go forward with implementing that ... was it your decision to not move forward, or were you in a team and that's just what ended up happening?

The startup that I was working at, that came up with the pitch, fell apart before we ever had to make the call to implement it. Which I’m somewhat glad of; I don’t really like the idea of being forced to choose between ethics and helping a few friends survive. (And don’t get me wrong, building this extension would have been a matter of survival for the startup—we were depending heavily on piecing together a unified social-network graph to do Dynamic Network Analysis to, and social-network services actively resist having this data pulled from them.)

If you’re curious, the point of the startup was to path-find between you and people you want to be connected to but aren’t (investors, say), telling you which friends-of-friends to prioritize getting to know in order to later leverage them as intermediate links in the chain to meet the people you really want to meet, that are three or four or five steps removed from you, and so entirely invisible to your local part of the network.

(And yes, that’s a really darn cynical play. But the data that we did have showed that the intermediate “links” in your chain would almost always be business-people, “growth hackers”, etc. for whom getting to know you was valuable in its own right—they would value any chance to grow their Rolodex. So it at least seemed like a positive-sum interaction on the surface.)

So basically Tinder for Recruiters?

On the surface, it sounds useful, but also like Tinder, would've likely been abused.

More like a creepy(er) LinkedIn.

The discussed API changes do not limit malware to snoop on browser traffic, requests can still be observed, and scraping is also not affected.

Ad blocking is arguably one of the best ways to protect yourself from malware, and that is the capability Google attempts to limit.

Ah, alright. Too bad, actually; I was hoping there was a silver lining.

Well, maybe—did the API previously give extensions the ability to rewrite the request in-flight? Such as e.g. to rewrite Amazon requests to embed the extension-author’s Amazon affiliate ID? I know extensions can do that, but I’m not sure how they accomplish it; it’d be nice if that could be quashed.

But really, you shouldn’t be equivocating the particular capabilities this API provides with the ability to block ads altogether. There are many ways to implement ad blocking, and not all of them require the capability to have your extension active “on the wire”, intercepting network traffic and doing arbitrary Turing-complete things in response with full extension-level capabilities.

Personally, from my perspective (and Google might share this perspective), this API was never “the ad-blocking API.” A true “ad-blocking API”—a single-purpose one that can’t be used for any other malicious purpose—is the thing Safari has, where your extension can feed a block list into the browser, and the browser will prevent network requests matching the block list itself, without ever informing your extension that it has done so.

Of course, Google would never implement that API in Chrome. And that’s the real problem.

I don’t blame Google for limiting some other API that isn’t really “about” ad-blocking, but which can be used to block ads as a side-effect; just like I don’t blame Intel for making their processors slower in response to Spectre/Meltdown. Sometimes security for users trumps useful features or performance.

But I certainly do blame Google for not implementing an API that has absolutely no security impact—is perfectly safe, perfectly performant—and blocks ads.

If anything has ever been decreed on-high from Alphabet to the Chrome team, it’s likely “don’t implement the Safari content-blocking API, or any standard based on it.” Not anything to do with this API. This API is just problematic, and the Chrome team themselves want its capabilities out of the hands of extension authors.

There are already ways to limit access to the webRequest API using host permissions, for which the user gives explicit consent.

Once the host can be accessed the webRequest API isn't even needed to rewrite requests and read or modify page content, all of it can be acomplished using content scripts.

There are really no meaningful security or privacy benefits in limiting the webRequest API in ways that are being proposed by the Chrome team.

I don't think the host permissions do what you think they do.

1. They don't address the performance issue.

2. They still allow redirection of requests, which is a bigger security issue.

V3 fixes 2, which is a security issue, and a unique one if your threat model is a malicious extension.

Host permissions are required to access the requests through the webRequest API. Once you have set the host permission, you can redirect requests using a content script, without needing to touch the webRequest API.

The performance impact is negligible in sensibly implemented extensions, and there are several other ways to make the webRequest API more performant if that's really a valid reason for the change, some of which are already implemented in Firefox.

I don't believe most honest software engineers would rip out a core functionality of the browser that so many people depend on, just to gain a couple of microseconds, at least not before trying other optimization techniques.

> Once you have set the host permission, you can redirect requests using a content script, without needing to touch the webRequest API.

Then I'm a bit confused, why does this change cause a problem for ad-blockers? Can't they block and redirect requests via a content script after these changes?

No, you can't block requests from a content script as you would with the webRequest API. You can however redirect the page from a content script, read and replace page content, and steal passwords.

The webRequest API gives you fine control over all requests on the page in a performant way. A content script based ad blocker would be able to block ads to some degree by removing the DOM elements that initiate the request, though it would be limited, prone to breaking, and not performant.

Ah. I'm caught up now. That jives with my understanding.

(this was meant as an edit to the prior comment, if it seems out of place):

Looking into this more, I think what you're saying is that you can request both a content script and a host permission for the webrequest, and the user will approve both at the same time. But you have to specify them differently in the manifest, ie. it looks like

    permissions: [
        "declarativeNetRequest",
        "<all sites>",  // old way
    ],
    host_permissions: [ // new way
        "<all sites>"
    ],
    "content_scripts": [
   {
     "matches": ["<all sites>"],
     "js": ["contentScript.js"]
   }
 ],

My (limited) understanding is one of the points of the new manifest is to make things more declarative so that they can be statically analyzed. So that, for example, host_permissions "<all sites>" might be allowed, but would require manual review, and content_scripts: ["<all sites>"] could be flagged even more strongly, etc.

If you don't specify the content_scripts portion of your manifest, you don't have the ability to write content scripts, so there isn't a security concern, you're limited by the declarativeNetRequest sandbox, and can't do the snooping.

For reference, uBlock, which I use, requests a ton in the manifest[0], and there are valid reasons for that (the userscript is, iiuc, for using the content picker). But a minimal ad-blocker could declarativeNetRequest, host:<all sites>, and nothing else, if I'm reading correctly. And that limited footprint can be statically validated. You know that extension can do nothing but modify requests, the way the requests are modified is statically verifiable. That sure sounds more secure to me.

[0]: https://github.com/gorhill/uBlock/blob/98271f6140bb188222e21...

Declaring content scripts in the manifest helps with performance and perhaps static checks, but it's not required to run code in pages, only the host permission has to be declared in the manifest, and you can just inject the code when the page loads: https://developer.chrome.com/extensions/tabs#method-executeS...

Malware would prefer executeScript to avoid scrutiny, and also wouldn't care about performance.

The declarativeNetRequest API is actually a welcome addition, but it's not a valid reason to deprecate parts of the webRequest API. We should be able to modify request headers for various use cases, some of which can't be forseen, because that's just how innovation works. This is why neutering that API is considered so harmful by pretty much everyone outside of Google.

If security and performance are an issue, webRequest API features can be decoupled into granular APIs, optimized, and accessed with granular permissions. The new proposed APIs do not even come close to the feature set of the webRequest API.

When feature parity is achieved through new APIs, deprecating the webRequest API makes perfect sense.

FWIW, it looks like changes to lock down tabs.executeScript and content scripts were originally considered[0], but I agree that tabs.executeScript not requiring a declared permission is a giant issue.

Thanks for this discussion btw, you introduced me to a bunch of quirks in the chrome apis I wasn't aware of.

[0]: https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...

I'm happy to chat, and I hope the discussed concerns about deprecating the webRequest API will motivate you to bring up the topic internally at Google.

This also isn't even a first step where eventually privacy restrictions around observation will be added later. From the discussion thread[0]:

> Chrome is deprecating the blocking capabilities of the webRequest API in Manifest V3, not the entire webRequest API (though blocking will still be available to enterprise deployments). Extensions with appropriate permissions can still observe network requests using the webRequest API. The webRequest API's ability to observe requests is foundational for extensions that modify their behavior based on the patterns they observe at runtime.

It's not just that the people claiming that this will improve privacy are wrong about this specific change. Google's stance is that blocking request observation is fundamentally not acceptable, specifically because it would stop extensions from tracking and responding to user behavior.

To all the people saying, "well, it's about time extensions got locked down" -- Google is not your friend and they don't want the same things as you.

[0]: https://groups.google.com/a/chromium.org/forum/#!topic/chrom...

> though blocking will still be available to enterprise deployments

So I guess the answer is to be an "enterprise", then.

Not necessarily, since having the API available for you personally is only one part of the problem.

The other question to ask is, will you even have an ad blocker to install, given that Gorhill has hinted these changes might mean abandoning Chrome as a target platform?[0]

If enough ordinary people move to enterprise, Google will eventually restrict that as well. If not enough people move to enterprise, extension authors won't support Chrome and you'll be stuck writing or forking your own ad blocker. So you're gambling on being able to walk very thin line alongside a company that's already shown you that it is actively hostile to your interests.

The more permanent answer is to just switch to Firefox and stop playing their game. Switching to enterprise will probably at best buy you some time before you're forced to make that decision.

[0] https://github.com/uBlockOrigin/uBlock-issues/issues/338#iss...

Oh, I switched to Firefox years ago (Chrome still doesn't offer any reasonable equivalent to Tree Style Tabs). This would be more for the folks that I haven't quite convinced yet to do the same.

Can confirm that using users for social media scraping is huge. I worked in that field for some time, though the product was never brought to production.

Note that in our case, though, the user was completely informed that they were acting as a node in a network. They provide the ability for us to collect non-public data on other users, we provide them with non-public data on other users that they do not have access to.

"They provide the ability for us to collect non-public data on other users, we provide them with non-public data on other users that they do not have access to."

Really doing God's own work there ...

Oh, we knew it to be the devil's work. The idea was to gauge how much people are willing to share, in order to get something that they want. It was disgusting just how many people were willing.

Far from a technology demonstrator, it was a social contracts demonstrator. Most people have no morals and will happily turn over their "friends'" personal information. Just look at how many Android apps require the Contacts permissions.

So, do the new API changes block people like what you were working on? Because I sure as hell hope they do, and if the changes mean ad blockers have to adapt, so be it.

I don't think they do.

> In Manifest V3, we will strive to limit the blocking version of webRequest, potentially removing blocking options from most events (making them observational only). Content blockers should instead use declarativeNetRequest (see below).

[1] https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...

Of course the new API changes cannot block people what we were doing. The API has no way of knowing that Joe User will now share the non-public information.

In fact, an argument could be made that we were not enabling anything that was not enabled before. Joe User could already log in and see his friends' information and then go off and gossip. We just demonstrated that Joe would be _willing_ to gossip in order to hear more gossip. It was absolutely disgusting.

That argument generalizes to not allowing users to have any say and to trust big brother knowing what's best for you instead.

"HTTP headers can no longer be freely edited, unless they are part of a limited whitelist blessed by the Chrome team."

Are you suggesting that Chrome/Chromium users will not be able to control the headers that the client sends? (short of editing the Chromium source and recompiling)

> This kills innovation in the browser extension space in its tracks,

I was under the impression that innovation in the browser extension space had long been killed since the transition of extension APIs to WebExtensions in the major browsers.

Not quite — if you run Firefox nightly you can develop and use "webextension experiments", which have the same power as the old pre-webextension Firefox addons. If you wish, you can also apply to have your experiment be turned into an official webextension API for Firefox.

This is considerably more cumbersome than previously, but still far better than having to fork the whole browser. (On the topic of forks, if running a bleeding edge browser — nightly — is not for you, you can use Waterfox, which is approximately Firefox ESR + the ability to run full-powered extensions.)

> you can use Waterfox, which is approximately Firefox ESR + the ability to run full-powered extensions.

But if you're going to use Waterfox (as I do), and you want to use the most recent version, be sure to check if the extensions you want to use will work. 68 introduced changes that do prevent some older extensions from working.

There's plenty of room for innovation in extensions just using the existing WebExtension APIs.

When the existing APIs run out of steam, you can work with Mozilla developers to design and implement new APIs and submit them to Mozilla as a contribution to Firefox. No guarantee they'll be accepted --- performance, security, maintainability must be maintained --- but people have done this successfully to enable their extensions (e.g. NoScript).

Mozilla needed to clear out the unmaintainable mess of letting any extension poke the browser's internals in any way. They've done that, without "killing innovation".

> They've done that, without "killing innovation".

Yes, "killing" is too strong, but they've certainly reduced it. The loss of functionality in the new extension system is a big problem for me, and prevents me from using the new Firefox in any serious way.

And yes, I know that I'm in the minority here and my opinion means nothing. Mozilla has shifted what market it's trying to address, and I am not in the target demographic. I recognize that's a personal issue and it's not really a criticism of the new Firefox.

Mozilla hasn't "shifted what market it's trying to address". Mozilla has always understood perfectly well that if Firefox isn't fast, stable and secure enough to be used by the masses of "regular users", it will become irrelevant. It just became increasingly clear over the years that the old extension system was hobbling that.

Forks can be patched. Otherwise, what's the point of maintaining a fork?

Although a small team of occasional volunteers might not have the resources to maintain this, you can be sure that Microsoft does. It's also considerably easier than doing it all yourself like Firefox.

Reply on news.ycombinator.com