HNHacker News
TopNewBestAskShowJobs

zbentley

6,089 karma · joined June 30, 2017

Zac Bentley

he/him

github.com/zbentley

blog.zacbentley.com

submissionscomments
zbentley··on AlpineBSD
> which was selling people only for people that either don't understand docker layers or can't make Dockerfile correctly.

...or people who had a test/deploy/debug cycle on environments that necessarily had no layer cache (ephemeral VMs, one-shot kubernetes clusters, OpenWRT routers that lost the layer cache on reboot). Or people with network-constrained traffic from registries or mirrors thereof.

Or people in environments that took container security findings (perhaps too) seriously and were tired of triaging endless reports for random dependencies in "FROM ubuntu" containers, and wanted a default required-by-policy base image that didn't come with the world.

Or people who needed a relatively full-featured build environment against musl without having to do bizarre incantations in e.g. a Fedora image to switch away from libc while still being able to link against other system libraries.

zbentley··on Microsoft offers mass buyouts for roughly a quarter of its workforce
Sometimes--very rarely--one's ethics may dictate not taking it, as with the "fork" email to U.S. Federal Government employees. Or when an early-stage company layoff is being done in tandem with a buyback/tender to reduce employee ownership before the board does something crazy.

Most of the time, though, you're right. Take it.

zbentley··on Anthropic S-1
Sort of. If they can rely on differentiation and loyalty now, pre-IPO, from their large-but-cheap customer base, they can focus their growth on serving enterprise/other markets, and lean on their existing base as a low-churn-risk (compared to the rest of the AI ecosystem) customer and marketing surplus. Focus is good.
zbentley··on All you need is Wide Events, not "Metrics, Logs and Traces"
All true and fair. But I do like having the instrumentation API in your code speak wide events. Then have your telemetry config decide what you want to transmit as a verbatim big event, what you want to sample/transmit as a metric, or a trace (or whether you, at the moment of transmission, need to go grab a few things out of the local tracing context stack and use them for preprocessing), and so on.

If you do all that on the client side (not in a sidecar, ideally: actually in your client code when deciding what to transmit) you have a fairly nice way to temporarily turn on e.g. rich trace recording to hunt a bug, or start spraying event bodies to a local file when debugging a troublesome service instance, and so on. The rest of the time, things can transmit a few big payloads and a lot of small metrics, and things stay cheap.

In other words, it's easy to go from "recorder.record_rich_typed_event(...)" to capturing any or all of events, traces, metrics, or logs. It's not easy to go from e.g. "metrics.record_integer_metric("somemetric", 15)" to recording a richer, more complex event in a pinch.

zbentley··on JetBrains Thinkrail
Have you tried T3? It seems more session-oriented than IDE-oriented, but I keep seeing folks who use/like it. Maybe one of these days I'll give it a try.
zbentley··on Roku developer repositories moved from GitHub to GitLab
I think this might just be a you problem.

> create new account

I don't think "need to create an account to figure out that you can't contribute with a given forge" is a real problem. They're just as publicly browsable sans-login as GitHub, for the same reasons: people want to share their stuff. And when you do need to make an account, most software forges have SSO with the big/common providers.

> find it archived, google for new repo, find that one archived, go to new repo

You needed to RTFM and do a bit of research to figure out where/how to contribute for GitHub projects too. They got archived/moved around/invisibly-forked just as often as people move forges today. How many times have you seen. "FizzBuzz is archived and no longer accepting contributions; an actively maintained rebuild of this project is at https://github.com/somebody/FizzBuzz-ng"?

> find they now take patches via email instead of PRs

Plenty of GitHub repos either don't accept contributions via GitHub PR (Linux) or require lots of nonstandard hoop-jumping to get a contribution approved (Python).

> 2-3 different protocols

Git is the protocol. I don't think the move away from GitHub is making anyone move away from Git. hg/jj/svn are still minority tools for the same reasons they always were.

> one index

It's Git. The index is folders on your hard drive. List their origins if you forget the URLs of their forges. Make a browser bookmark if you don't want to keep the checkout around.

Under the most charitable interpretation of your complaints that I can come up with, contributing in a not-universally-GitHub world is like ... 1% harder per repo contributed to. At most. To the tune of, at the very absolute worst, like 2 minutes of extra work when making your very first contribution to a project. I don't think it's that hard to stop googling "ProjectName github" and start googling "ProjectName git source" or "ProjectName contribute code".

zbentley··on Security Disclosure
Genuine question: what do you recommend as an alternative?

The only alternative I can think of is "own and operate all of your site's hardware on physical premises to which you have sole access", and even that's not perfect.

Are you arguing against hosting/cloud/datacenter services in general? Or is there a specific technique you would prefer people use to secure their keys in a non-owned environment?

zbentley··on Broccoli2Shell: Brotli decoder RCE in Node.js, Python and more
Please feel free to publish and/or link to your alternative implementation. Many eyes can help with identifying security/functionality issues, and if your implementation proves superior to libbrotli in general, this area of the ecosystem is in dire need of diversity.
zbentley··on Broccoli2Shell: Brotli decoder RCE in Node.js, Python and more
...which points to an area where Python has a significant drawback: optional/accelerator dependencies are loaded via try/except-ImportError. That's suboptimal for a bunch of reasons:

- If a vulnerable dependency is in your environment as a transitive of something that's not at risk (e.g. code you don't use, code that doesn't talk to untrusted sites), code that shouldn't use it will pick up that transitive.

- Autodetection of optional stuff is often silent. E.g. libxml vs. pure-python XML parsing doesn't really act any different at load time or most of runtime, besides being faster, so it's non-obvious what's being used when.

- Probing for packages that aren't listed as a concrete dependency by importable name risks loading unintended code due to name conflicts (malicious, or accidental/caused by local files with the same name or weird PYTHONPATH).

- There are plenty of ways to get an ImportError after running a lot of Python code in a dependency before giving up! That code might mutate global state/change behavior, or do insecure things, etc. Malicious actors could take advantage of ImportError being raiseable by user code to conceal naughty behavior at import time.

I really want Python to come up with a saner way to concretely identify (avoiding name conflicts) and probe for optional dependencies at import time. try/except-ImportError ain't it.

zbentley··on Broccoli2Shell: Brotli decoder RCE in Node.js, Python and more
Eh, it's a personal choice and a not unreasonable position to take, but there are a lot of things on PyPi that aren't packaged by e.g. Debian. Database clients, immensely popular frameworks, scientific computing libraries ...

Basically once you leave the realm of the 5-10 most popular building blocks in an area (which OS packagers do distribute), you're still left with plenty of packages that have 10ks+ of users, sane maintainer/security/ownership models, and code that would be very difficult to just copy-paste a la stackoverflow (e.g. native/compiled deps with complex wheel build tooling).

zbentley··on Roku developer repositories moved from GitHub to GitLab
> figuring out where and how to contribute has regressed 20 years

What?

Step 1: go to project's manpage/readme or website.

Step 2: find the URL of their software forge (git/svn/hg site).

Step 3: clone and contribute using that website.

That process has been largely the same for a lot more than 20 years. Most people were not using GitHub as a discovery/random-repo-browsing system.

zbentley··on Today, we hacked what we'd set out to hack: the iPhone
... is an example in support of GP's point.

Great for professionally administered servers or expert power users. Not for much else. Until recently, esoteric use-cases like playing audio were catastrophically unsolved. And then those got better around when screen sharing, accessibility devices, and remote-assistance tools all broke because of another OSS schism. Installing and running an ordinary program requires choosing among a handful of ways of doing that (container systems, package managers) even on the same distro. And then if a graphical program wants to do crazy stuff like "open a file", well, the permissioning situation is "fragmented" to put it generously.

Gaming? Gaming works great on Linux now! ...because a corporation with centralized planning, KPIs, and incentives spent hundreds of millions of dollars on it.

I like Linux. It's been my daily driver for years, on and off. I use it for a living, I use it for a hobby. But that doesn't mean it's not a perfect example of every single one of the OSS flaws that GP listed.

zbentley··on Docker is weird
Reasonable clarification. My response to your original "never do this" is a is a bit more specific then. There are two broad userspace concurrency models: cooperative and preemptive. Cooperative is a "sometimes the right call, sometimes more trouble than it's worth" situation; I can see what you're getting at there. But you're flat wrong about preemptive.

Cooperative concurrency: it depends. It's sometimes the right move, but has drawbacks. It's usually implemented to enable efficiency or partial parallelism of kernel-space code (IO). There are plenty of platforms that absolutely need it for that purpose: c10k, nodejs's rise to fame, all that. It behaves poorly when mixed with non-cooperative (blocking) code, and is poorly understood by many folks who expect userspace code to be parallelized as well. A common claim is that cooperative concurrency is harder to use than pre-emptive, but that's not true: some common things are harder, but some less common, but more dangerous things like race conditions tend to be much easier. Tokio and other cooperative systems that schedule onto fixed kernel threads can help with the hard parts a bit, but it's a partial mitigation at best.

Preemptive userspace concurrency: firmly disagree with your original point. Green threads are usually combined with userspace-code parallelism (e.g. goroutines scheduled onto multiple threads) and efficient kernelspace interactions (IO multiplexing). In the popular implementations of green threads I know well (Erlang and Go), this is an unalloyed blessing, in that it both removes many of the dangerous edges of threaded programming, facilitates increased efficiency, and lets users semantically model concurrency/parallelism with much less fear of e.g. creating too many threads, torn reads, etc. That's a big benefit: now we can model things like actors or CSP as subprograms written in procedural, synchronous, ordinary-looking code without getting Ph.Ds in operating system internals or having to carefully reason about complex interlocking layers of callbacks and futures to minimize risk of blocking or deadlock. In many ways, this brings the good parts of single-responsibility/what folks call "the UNIX philosophy" into a single process with minimal pain.

Rare cases where the userspace-concurrent runtime fights with your ability to interact directly with the OS, as with Docker's fork/thread/process-global-state challenges, do emerge. But I don't think those are primarily because of userspace concurrency; rather, they're because it's hard to do low-level things in a high-level, managed runtime. You can fork in the JVM, too, and you'll have problems. Different ones, because the runtime's different (e.g. endless RSS burn at idle due to the child GC moving parent objects), but problems all the same. Same for Python: forking in the presence of regular OS threads and/or a running event loop is equivalent to arming a minefield; doable but troublesome, just like it was for the Docker folks with Go.

To be fair, some of the same pains as the cooperative model appear with pre-emptive, too, when userspace concurrency systems are retrofitted onto runtimes that do a lot of other kinds of concurrency as well--but that's because of the hybrid/retrofit nature of the product, and not because of green threading as a concept. Java's rocky adoption of virtual threads is an example of that.

zbentley··on Docker is weird
Tokio's not a lightweight thread/green thread system. It's an event loop, just like Javascript's, optionally with a fixed number of threads running work underneath.

For Go: you're likely right that OS thread overhead was the main reason for Goroutines, but I don't think it was the only one. There is also a ton of ergonomic efficiency to be had via green threads; like async/await, they're kind of a syntactic unlock that allows you to very liberally use goroutines/channels as a means of semantically modelling actors/state/data flow in your program, where doing that with regular OS threads is both dangerous and (given the start/stop overhead of threads on lots of other platforms) feels like kind of using a howitzer for a flyswatter. Golang saying "yeah, create a million of these, we don't care, it's 4 machine instructions per to start one" was pretty liberating for a lot of people.

The goroutine architecture also unlocked some nice performance at the OS level: GC latency improvements became easier, and some IO multiplexing efficiency a la epoll became almost free to implement due to the ability to cheaply move and suspend goroutines.

zbentley··on Docker is weird
It worked out pretty well for Go. Nothing else besides Erlang cracked the parallelism UX/runtime-performance code as well as Go did, and it didn't have the drawbacks of Erlang.

> rule broken by almost every language in common use

I don't think so. Go/Erlang/Java do break it (Java very recently). C/C++/Rust/C#/JavaScript/Ruby/Python/Perl/TCL don't break it. PHP ... sort of breaks it, (IMO they just hide cooperative nonparallel concurrency in a nice API, but I'm not well-informed on php)?

zbentley··on Docker is weird
You absolutely can do that. Very easily in fact: https://blog.ttulka.com/building-container-images-without-do...

And for most people, that's a bad idea. Most people don't have to know what RPATH, ldd, multiversion/simlink library discovery, etc are. And even if they did, Dockerfiles give you an easy way to do something you couldn't easily do before: multi-stage builds and a highly robust way to say "drop the parts of my transitive hull that were only relevant at build time". Dockerfile semantics kind of descend naturally from needing the behavior of stages, parallelization, cache keying for image layers, semantic layering (FROM), and making simple use-cases relatively at-a-glance/contained.

zbentley··on Docker is weird
I think there's two questions there:

Q: Why not have container builds and runs be invoking-user-specific by default? A: As the adjacent commenter pointed out, this would be an overlay on fundamentally rootful behavior. It would also remove a lot of "works on my machine, works anywhere", since different user accounts running/building containers could be restricted in different ways on different computers, so my "docker build" and your "docker build" might both work, but do different things or fail in different ways. Note that this isn't the same as saying "rootful dockerd is good"; it's not (usually)! That's why rootless docker (https://docs.docker.com/engine/security/rootless/) is recommended in many environments. But even in rootless, the run-user of the container and build are still uniform with the rootful behavior (albeit they require more participation from the daemon-owning user account WRT UID mapping/user namespaces/privileged actions), which I think is a good thing. Rootless docker, last I used it, behaved sensibly re: vs. user container images (kept per-user) and globally-cached layers (removable by anyone), but that may have been a specific configuration where I was using it.

Q: Why aren't docker images user-browsable files? A: I can't say for sure; I didn't build Docker, but I hear the guy who did is in the neighborhood :). I suspect it's a combination of caching (since build stanzas/Dockerfile lines are intended to be proxies for the content address of a layer, there's nothing to gain by making layer caches per-user), implementation convenience, and an influence from the Golang platform wherein (after the age of vendoring wound down) the developer tooling has many convenience layers that de-emphasize user visibility of build/download artifacts a la "go run". Docker goes further, though. And yes, I know that you can get at OCI images/layers as files. But most people don't.

zbentley··on Docker is weird
The main primitives that underlie Docker were originally intended to facilitate security and isolation. Docker's innovation was combining filesystem-in-a-box with those security primitives, and delivering all those capabilities with highly intelligible UX. That's not meant to minimize; I think those are monumental (and monumentally impactful) achievements. But there was no "re-pivot" to secure sandbox isolation; that's what things like cgroups and namespaces were created for!
zbentley··on Docker is weird
I think there are some interesting points there, but your rude, arrogant, and aggrieved tone overwhelms them.

If you decide to edit your tone, I'm happy to discuss more here re: filesystem transactionality tradeoffs, since that's something I've struggled with in both directions WRT containers.

zbentley··on eMudhra to exit global SSL/TLS certificate business
In addition to issues pointed out in the other comments:

- There are a lot of countries small/technically incompetent enough that this would be contracted out. The contracted-out CA companies would create a larger single point of failure than just one country.

- CAs are used for keys that sign a lot of things other than domains/websites. What about application bundles? Some of the eMudhra intermediates were for that purpose. Who operates those CAs? IMO "OS/app store vendors" is a good answer to this one, but that's not the status quo.

- What counts as a country? There are famously lots of disagreements there. So suddenly Microsoft (or whoever's preloading CA certs onto a lot of computers) has to be in the business of litigating what countries do/don't exist depending on where their devices will be sold. They don't want to do this and so will resist this proposal.

- Business/organization redundancy would be needed. Companies go out of business. Governments get deposed. Datacenters get bombed or hacked. There's no guarantee that CA root or intermediate private keys will be handed over to successor/backup operators when those things happen. Now you need <number of redundant CA operators> X <number of countries> CAs.

- What about non-cc TLDs? If you say "those are less secure and don't need to be country-locked", that's fine, but now you need some additional number of CAs.

- Governments, famously, don't go out of business when they screw up. So it becomes much harder to censure a misbehaving CA.

- When someone (e.g. browser/OS vendors) removes a country-locked CA for misbehavior, this now has to be reviewed as a political act, at which point many more cooks enter the should-we-revoke-this-cert kitchen. We know this from experience with ccTLD domain registrar politics. "We think you revoked Canada's CA because you're in a trade war with them, not because of your stated reason that they did something insecure" or "Don't revoke Canada's CA for using a private key of 124; we need to be nice to them because we're negotiating a trade agreement".

- The net number of CAs needed to operationalize such a system is large: current/legacy ones, plus country-specific ones times redundancy, maybe plus non-cc operators. That's a much bigger list of roots, with a proportionally much bigger risk of revocation/update churn. More churn means more devices running a larger variety of outdated/mismatched configurations, which means much more frequent certificate errors for users, which means an increased tendency to bypass verification or not perform it in the first place.

I agree that the status quo of "dozens of random companies with highly variable competence and affiliation" is bad and should change. I'm not sure that this proposal is better.

zbentley··on Investor-owned utilities charge 23% more for electricity than public providers
An adjacent commenter explained further: https://news.ycombinator.com/item?id=50209678
zbentley··on OpenSSL 4.1.0
> what if Microsoft would give their engineers a few decades of time just to refactor their existing systems

They wouldn't, even if they didn't care about shipping new features. The technical complexity inside Windows/Office/old core MS offerings arises only in small part from ordinary tech debt--that is, things fixable without changing observable behavior. The lion's share of that complexity is because of Microsoft's priority to avoid breaking compatibility (backwards, or with hardware/software combos that work) at all costs.

That means that a refactor-only decade would achieve little unless that compatibility priority were changed.

zbentley··on Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds
Oh, I thought you meant "private key up to N bits", not "private key up to the exact value N". In that case, sure, but for non-eMudhra-issued keys BSGS's more relevant complexity figure is O(2^keybits). Unless you've a quantum computer on you, in which case I'm told it's easy-peasy :)
zbentley··on PHP 8.2.35
Java has had plenty. Remember when everyone in security flinched when they heard the words "java.io.Serializable"?
zbentley··on PHP 8.2.35
> I can imagine a truce

This is very entertaining writing on that subject:

https://nesbitt.io/2026/06/26/incident-report-cve-2026-lgtm....

zbentley··on PHP 8.2.35
> SMS may take some tricks (again, iMessage can be useful here).

How would iMessage help? Every SMS MFA implementation I've seen only works with phone numbers. If you mean "have a phone in the office hooked up to MFA and sign into iMessage with that phone's Apple account on multiple colleagues' devices", I've tried that. It technically works, but is flaky (Apple's reasons for forwarding-or-not greenbubble SMSes are opaque, and much less reliable than iMessages, though I'm told that improved recently due making things give up on Continuity and fall back to WAN more often). It also slams into the reality that most people want to use the Apple device that's getting the SMS codes with their personal Apple account, which causes friction.

If you need that capability, it's probably both more reliable and cheaper to use a web-controllable phone number (Google Voice & competitors).

zbentley··on PHP 8.2.35
I think the very success of PHP, JS, and all the other slower server-side languages is evidence to the contrary. Making the world use more compute for convenience was a real answer, and we know it was real because it happened. So why not do it again but for security?
zbentley··on What Is Gridifcation
They do. Suno markets itself to artists and producers heavily. Why hire session players when you can use AI? Why have a studio/SE prepare your masters when you can use AI? Feels gross to me, but that’s the pitch.
zbentley··on Quad9 to exit France
I’m not defending it and hate it too, but it’s worth remembering that from the perspective of many governments and regulators, the internet is a legally broken Wild West that the law needs to catch up with. Viewed through the lens of legal precedent that balanced privacy/safety/liberty before telecommunications networks, that is true: surveillance was permitted within a very broad set of parameters (by default, even: that’s why we talk about e.g. private residences being protected exceptions to no-privacy-by-default).

Again, I don’t endorse that framing. But that’s where a lot of lawmakers are at. They’re not corrupt or fascists or hysterical about porn or whatever (well, plenty are, but not most). They’re trying to correct what they identify as a dangerously uncontrolled area.

zbentley··on Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds
I don't think that's true. Got a source?
Page 1 of 34Next →