HNHacker News
TopNewBestAskShowJobs

jmillikin

11,783 karma · joined March 23, 2009

https://john-millikin.com/

john@john-millikin.com

Currently semi-retired; formerly at Google (2011-2017) and Stripe (2017-2022)

submissionscomments
jmillikin··on Sorry Purism, I’m not investing. It’s (possibly) not even legal
There's nothing wrong with someone pointing out that something is illegal, for example I don't have to be a lawyer to note that stealing a car would violate criminal law. It's reasonable for the OP to note that if they were to invest in that deal, then someone must be breaking the law.

Note also that what you call "self-certification" is just ... it's how things are for everyone? It's not like you can ask the SEC for a piece of paper documenting that you qualify. There are some professional licenses that automatically count, but if you made $210,000 in W-2 income last year there's no report submitted to a central registry of People Allowed To Make Risky Investments.

jmillikin··on The seven specification ur-languages
The thing with taxonomies is that trying to make a category more precise tends to exclude things you want to include in it, and vice-versa. It is especially easy to find examples of this in nature, because nature has existed since long before humans had opinions about how things should be organized.

What is a cat? It's a small furry quadruped in family Felidae.

- Exception: cats may be quite large (lions, tigers).

- Exception: cats may have no fur (sphinx cats).

- Exception: cats may have fewer than four legs (e.g. due to injury).

Even questions that seem like they should be quite easy to settle, like "are these two gulls of the same or different species?", might be impossible to define formally due to things like <https://en.wikipedia.org/wiki/Ring_species>.

With the specific example of "no such thing as a tree", the category of plants that humans call "trees" isn't a genetically coherent group. Lots of different types of plants have woody stems and bark and leaves, and you can't group together all the things humans call trees without also including things that we definitely don't.

jmillikin··on Scaling up the Prime Video audio/video monitoring service and reducing costs
Yeah, all of those mitigations don't work as well when you've got thousands of engineers whose work would be blocked if some intern's badly written test blows up.

A monolithic build means that your ability to develop and deploy your team's code is dependent on every other team. As the number of teams gets larger, that multiplier really hurts.

jmillikin··on Stabilizing async fn in traits in 2023
Funny, I saw that same line and thought "oh yes, that is much cleaner". Well, it is a little goofy to have what looks like a function call in the middle of a type signature, but the `Send + 'static` are simply what's needed if you want to use some sort of magical N:M thread pool in a language with TLS.

Honestly, a lot of the challenges people face with Rust async is because they're trying to use a really low-level primitive instead of just using normal threads. They come from a language like JavaScript or Python where async is the only option for concurrent I/O, and they don't realize that JS async is a completely different construct from Rust async. They'd be better off using threads.

jmillikin··on Scaling up the Prime Video audio/video monitoring service and reducing costs
1. Add a test with a time-bomb (such as a test certificate with 365-day duration), wait a year, and now your test fails without having changed.

2. Add a test with a network dependency, and when that dependency is slow / down / turned off, the test starts failing.

3. Add a dependency on a third-party Github repo that clones from `main`, and the next time some dev touches a file in that repo your test starts failing.

4. Add a test that allocates memory in proportion to size of the codebase (e.g. because it tries to build a giant in-memory tarball of all the .mp4 assets). Eventually it will get flaky when it starts scraping up against the build machine's limit. Extra fun if your builds run without defined memory limits on machines of different sizes.

In a monolithic build, there's all sorts of ways for a single person to cause other teams' tests to fail, even months or years after they've left the company. Some of them can be prevented mechanically (such as by running tests without network access), but a lot come down to "tell them to stop doing that".

That's why big companies never run one build per repo.

jmillikin··on ECharts for Python

  > Few of the major contributors maintain flashy public profiles but many of
  > them state they live and work in China. Yi Shen has contributed the largest
  > number of commits and works for Baidu. [...]
ECharts was originally a Baidu project, released under https://github.com/ecomfe/echarts (ecomfe = "E-commerce frontend"?). They still maintain some of the auxiliary libraries.

I've been using it since around ... 2018-ish[0]? ... as a replacement for Google Charts. It was my first time using a big library from one of the Chinese tech giants -- basic docs and tutorials in English, then all the advanced stuff (and comments) written in Chinese. I was impressed by how comprehensive the charts library was, and how they'd obviously invested a lot of brainwork into the configuration system. IMO it's one of the highest-quality data visualization libraries in JavaScript unless you're willing to dive deep into something like d3.js.

The blog post's author describes running echarts in a headless Chrome, though, which seems insane to me. It's JavaScript rendering to a <canvas> -- can't it run in Node with https://github.com/Automattic/node-canvas ?

[0] A small publicly-accessible example: https://john-millikin.com/reddit-front-page-2018#by-domain

jmillikin··on Do Papercuts Matter?
Package namespacing in Rust is not foundational -- the compiler and toolchain are decoupled from the identifier used by crates.io and Cargo. Non-Cargo build systems (such as Bazel) are able to work with arbitrary package names without problem.

There are three general directions crates.io package namespaces might take, and aesthetics plays a major role in not currently having any of them:

1. Use human-readable strings as on Github, yielding package names like `jdoe/some-package`. The crates.io maintainers prefer to avoid this due to potential operational load from people requesting changes to their username.

2. Use opaque (but stable) namespaces such as an integer user id, yielding package names like `user-12345/some-package`. Some people have a strong aesthetic aversion to this.

3. Assign each package a UUID. This is technically possible in current crates.io if the leading character is [a-f], but I've never felt mean-spirited enough to actually do it.

jmillikin··on Do Papercuts Matter?
People learn to tolerate minor bugs in software because most software is buggy. If I became upset at every minor (or even major) bug I encountered in day-to-day life, there would be no time to do anything else. On rare occasions I get blocked enough to track down and report/fix the bug[0], but mostly I just try to find a workaround and carry on toward whatever my original goal was.

Even if every bug were to be fixed, one would be frustrated by behaviors that are clearly intentional but stupid, usually because someone tried to over-simplify something[1], hack in a "helpful" shortcut[2], or has aesthetic preferences against supporting complex use cases[3].

Finally, beyond those two great lakes of bugs and bad ideas, we come to the endless ocean of missing minor functionality. The comic reader that lets page order be swapped for manga but doesn't change which arrow "next page" is bound to; the web browser that doesn't allow translating a page unless its language heuristics get tickled in the right way; the mobile keyboard that couples auto-punctuation with spelling correction.

It's like the joke about wine snobs -- if you don't know much about wine then most wines taste fine, but if you take a wine appreciation course then you'll never be able to enjoy it again.

[0] three weeks ago: https://bugs.ghostscript.com/show_bug.cgi?id=706551

[1] A lot of GNOME's applications fall into this category, for example its "Document Scanner" vs xsane, or "Music Player" vs Rhythmbox/Banshee.

[2] two weeks ago: https://mstdn.io/@jmillikin/110211856191257399

[3] I am annoyed daily by crates.io lacking per-user namespaces, and the primary objection to adding them seems to be driven by a preference for single-token package names.

jmillikin··on WebAssembly+WebUSB port of the SANE scanner library
If the standard for Linux scanner drivers was WASM, then companies who publish Linux drivers (such as Epson) would receive perfect forward compatibility for free.

If WASM driver support were to be included in Windows and/or macOS, similar to how macOS and Linux share CUPS, then Linux support would be more widespread because the vendor's driver could be reused as-is.

I think the thought process behind "supporting one unified platform-independent bytecode would be easier than compiling a .so that works with all GNU libc versions" is straightforward and self-evident. Do you disagree?

jmillikin··on WebAssembly+WebUSB port of the SANE scanner library
Application software would still go through the OS native printing APIs. Office or Chrome don't directly interact with the vendor's driver.

As for browser API compatibility, I don't see much point to it since they're mostly designed with the assumption of a JS host. Ideally a WASM driver would export low-level functions to translate from (for example) PostScript to USB packets. Then the underlying native driver (Linux kernel module, Windows dll, etc) runs the WASM and handles all the IO, calling exported WASM functions as needed.

jmillikin··on sudo and su Being Rewritten in Rust for Memory Safety
You may want to re-read their comment. They said "1000 line long dependency file", not 1000 dependencies.
jmillikin··on sudo and su Being Rewritten in Rust for Memory Safety
Those are direct dependencies. The full dependency list is in Cargo.lock, which (as binjooou stated) is currently at 1013 lines.

I love Rust as a language, but one of the challenges with many projects currently written in it is that they follow the NPM model of using dozens of tiny dependencies for even trivial functionality. Luckily this seems to be limited to users of crates.io for now, and maybe things will change as the library situation matures.

jmillikin··on WebAssembly+WebUSB port of the SANE scanner library
IMO one of the biggest opportunities for WASM is printer and scanner drivers. These things have historically been distributed as native code, so the usability of the hardware on various OSes and architectures is spotty[0]. WASM is close enough to a native binary (fast, not source[1]) to be accepted by hardware vendors, but is inherently architecture-independent and in principle could be built to an OS-indendent API. Imagine sharing the exact same drivers for tens of thousands of devices across Windows, Mac, Linux, and every other OS that might exist in the future!

The same applies to webcams, screensavers (with WebGL/WebGPU), audio processing plugins, and any other software that doesn't come bundled into the OS and doesn't have a UI as such.

The utility of WebASM for non-Web use cases is incredible.

[0] My scanner is by Epson, who have good Linux support, but the driver was built against an ancient GNU libc that's no longer distributed by Debian/Ubuntu. And it's amd64, so I can't use a RaspberryPi as a scan server.

[1] I prefer open source when available, but hardware vendors often have cultural or contractual blocks against open-sourcing their drivers.

jmillikin··on GCC 13 and the State of Gccrs
Ancient big iron (SPARC, PA-RISC, Alpha), weird embedded with vendor-provided toolchains based on GCC, and recent architectures that don't have mature LLVM support (Longsoon).

m68k is already supported by LLVM and Rust supports it as a tier-3 target platform.

jmillikin··on GCC 13 and the State of Gccrs
GCC by default runs in GNU C mode, which accepts constructs that are not valid in any version of ISO C. Its extensions are not part of the language spec, and the only way to discover their behavior is by reference to implementation-specific documentation/code.

That is also the situation Rust currently is in. There is no Rust spec, only a reference manual for rustc.

People who complain about unspecified behavior in Rust (or, equally, about Rust's lack of a language spec) are not satisfied with mere documentation. They want something like the Java language specification and conformance test suite.

jmillikin··on GCC 13 and the State of Gccrs
1. GCC supports more target platforms than LLVM, including many low-volume embedded systems that are unlikely to ever get upstream LLVM support.

2. The GCC developers are interested in supporting a broad set of languages that compile to native executables, and Rust is an increasingly popular language for systems programming.

3. Whether something is "necessary" or not has never been decisive in whether or not it happens. When presented with a programming challenge, why not try it and see what comes out?

jmillikin··on GCC 13 and the State of Gccrs
GCC is first and foremost a compiler for GNU C, which is a set of unspecified extensions to C, which is a language that achieved near-universal adoption despite decades of competing mutually-incompatible implementations.

C++ templates are Turing-complete, and GCC implements them just fine.

The idea that Rust requires a formal specification (outside very rare situations, like aeronautic firmware) is silly.

jmillikin··on Standardizing BPF
No such argument is needed (or possible). If you've written an eBPF program that is a derived work of the Linux kernel (for example by copy-pasting Linux code into it), then it's a derived work whether or not it's ever loaded.

Conversely, if your eBPF program is not a derived work of the Linux kernel (because you hand-coded its bytecode in a hex editor via reference to a clean-room eBPF spec) then loading it won't change that.

If you wrote it in C and compiled it with Clang, then whether the compiled eBPF bytecode is a derived work would require (1) looking at the source code and (2) inspecting the compiled output to see if anything unexpected got pulled in via #include.

jmillikin··on Standardizing BPF

  > And, wanting to use some eBPF produced by clang, that person comes
  > to this "license" member in the manpage. What to do? Call lawyer?
  > Call police? Call reddit?
They could try reading, which is effective in this situation given that the quoted portion of the manpage is a concise and easily-understood description of the rules.
jmillikin··on Standardizing BPF

  > eBPF was linux-only in such a way that loading eBPF program into
  > kernel made your userspace program a derived work of linux kernel
  > and forced GPL-2 on the userspace.
Copyright doesn't work like that:

1. If some particular eBPF program is a derived work of the kernel, then whether it's loaded or not doesn't matter. Distributing that program (such as embedded into an ELF binary) would still require being accompanied by its source code (/ a written offer, etc).

2. If some eBPF program is not a derived work, then loading it into the kernel doesn't cause the userspace program to become a derived work. Programs distributed as interpreted bytecode are not automatically derived works of their interpreter.

3. Even if a program depends on GPL'd code in such a way as to be a derived work when compiled, its own license only needs to be compatible with the GPL, it doesn't need to be GPL'd itself. For example, it's legal to write and distribute a Linux kernel module licensed under various permissive licenses (MIT, 3-clause BSD, etc).

jmillikin··on Substack is the biggest threat to newsletters ever
Yes it does. I follow several Substack newsletters via RSS.

https://support.substack.com/hc/en-us/articles/360038239391-...

jmillikin··on IPFS-tiny: attempting to take IPFS to Space
IPFS promises a lot of things (content-addressable, unique identifiers, fast, efficient, durable) but hasn't delivered on any of them. It's basically a slower + more complicated reimplementation of Bittorrent's magnet links.
jmillikin··on Nolibc: A minimal C-library replacement shipped with the kernel
I don't want to be paid for my open-source code because that introduces expectations of support that I'm not interested in providing. The whole point of open-source is that I can just toss that stuff out onto the internet, and if someone else wants to adjust it to their needs then the only thing they need is a text editor.

Also, at $1 per user, your projects would need 15,000 new users per year just to match USA minimum wage. There's a reason commercial software has high price points (e.g. $50 for video games) or operates on a subscription model. You'd need marketing for that kind of userbase growth, which implies (1) overhead and (2) annihilation of forums like HN under a tidal wave of spam.

Finally, as a developer, there's no way I'd pay $1 per open-source project because nearly all open-source code is crap. I'm willing to pay high prices for good software, which is why the niche of commercial software is saturated and any developer with a pulse can get $100,000 per year at a cubicle farm in San Jose.

jmillikin··on Nolibc: A minimal C-library replacement shipped with the kernel
The article goes into some detail about different reasons you might want a minimal userland -- test harnesses, recovery environments, and pseudo-embedded environments. I do something similar in the test framework for https://github.com/jmillikin/rust-fuse -- to perform an integration test, I boot Linux in QEMU with a tiny custom /init.

More generally, the Linux kernel has a lot of code in it that is of generally reasonable quality and has received a lot of benchmarking/testing from well-resourced users. Sure, I could run some unikernel with a third-party network stack and SCSI drivers and ext4 implementation, but Linux already has all that stuff and it's ubiquitous. Why would I care about the extra ~30 MiB of RAM or whatever that it takes?

And that's before we even get to the topic of sharing code between environments. I can run the same unmodified binary on my desktop and on a minimal kernel-only Linux, which is not generally true of most alternative kernels.

jmillikin··on Rust's Poor Composability

  > Why is this a worst case for Haskell?
You generally don't want a garbage collector in the core request-dispatch loop, and Haskell's lazy evaluation makes it really easy to accidentally write code that can't be compiled to efficient output. It's not impossible to write a high-performance HTTP server in Haskell[0], but the skill requirement is much higher than in C++, Rust, or Go[1].

  > is the average LISP developer anymore likely to make a non industrial
  > code base than a Rust developer?
Yes. Not because of the developer, but because of how extremely flexible and dynamic the Lisp-family languages are. The power and joy of Lisp is in how it's almost a meta-language, so every project can become its own EDSL. The most famous (infamous?) example of this is Vacietis[2], which is a Common Lisp library that allows C code to be imported directly(!!).

[0] IIRC the Yesod framework's Warp does well on benchmarks, and when you look at code like https://github.com/yesodweb/wai/blob/master/warp/Network/Wai... you can see the lengths they had to go through to work around the choice of implementation language.

[1] Go has a garbage collector, but exposes the stack/heap distinction more directly than Haskell, so it's easier to write allocation-free code in hot paths.

[2] https://github.com/vsedach/Vacietis

jmillikin··on Spotting and avoiding heap fragmentation in Rust applications
Dynamic dispatch (dyn) in embedded is fine.

Dynamic allocation (Box) in embedded is not recommended.

`&dyn SomeTrait` is dynamic dispatch without dynamic allocation.

`Box<i32>` is dynamic allocation without dynamic dispatch.

jmillikin··on Rust's Poor Composability

  > Are there any merits to languages like Haskell/Clojure when the goal
  > is still shipping software?
Depends on what kind of software you're shipping.

Haskell is easy to use for building parsers, so if part of your value add is (for example) compatibility with a competitor's proprietary dialect of SQL then you'll be better off writing the parser in Haskell than in C or Rust. It also works well for software that is highly mathematical (in the "described as equations" sense, not the "HPC numeric kernel" sense), because a developer can apply formal methods without having to go all the way to Agda or Idris.

On the other hand, Haskell is not great when you need predictable optimization or precise memory management. A lot of modern commercial software is basically business logic scaffolded over an HTTP or RPC server, which is maybe a worst-case scenario for Haskell. It also doesn't work for Rust's target audience of systems programming because idiomatic Haskell requires a heavy runtime.

  > Industrial languages are obvious and so more people can jump on a
  > project without being lost?
Definitions will vary, but IMO the defining feature of an industrial language is that the language itself is not extensible. Every C or Rust project has the same keywords, control statements, operators, and type system -- as long as you don't go absolutely mad with macros. It's possible for a C expert to read C code from pretty much any codebase and understand what's going on at a tactical level. This is not true of research-y Haskell and it's very not true of idiomatic Lisp.
jmillikin··on Spotting and avoiding heap fragmentation in Rust applications
Dynamic dispatch (the `dyn` keyword) and dynamic allocation are distinct in Rust. You'll see Box<dyn SomeTrait> a lot in application-level code because the developer UX of boxed dynamic resembles a GC'd language like Java/Go, but they're not really coupled under the hood.

Dynamic dispatch allows you to call trait methods on values without knowing the concrete type of that value, similar to C++ virtual methods or a C-style manually constructed vtable. In an embedded context this will mostly show up as function parameters, since you can't move things around on the stack without knowing their size. You might also see references to a dyn value, like `&dyn SomeTrait` -- this is similar to `Box<dyn SomeTrait>`, but is a borrowed reference and therefore doesn't imply allocation.

Your link to the Oxide Hubris repo is misleading you because the hits seem to be pretty much all in build-time helper scripts (`build.rs`) and other minor tooling. You'd want to look through the source of their main binary, which probably wouldn't use allocation just for error propagation.

jmillikin··on Rust's Poor Composability
I think the author wants something like Ruby's blocks, although you're correct that monads would also be a good fit.

In Ruby, iteration can be performed by passing a sort-of-closure called a block. Unlike passing a closure (function), returning early from a block acts like returning from a for loop.

If a language has a garbage collector and no user-visible separation between stack and heap allocation, then there's lots of interesting things that language can do in terms of building "fluent" APIs.

jmillikin··on Rust's Poor Composability
Rust is a very industrial language. It tries to identify specific useful patterns and then expose them to the programmer in misuse-resistant forms.

In contrast, academic languages try to identify broadly-applicable abstract primitives and expose those, allowing users to implement the behaviors they need in libraries. The result feels more consistent at a language level because you can tie language features very closely to an underlying theory of computation.

I've written a lot of C++ (industrial) and Haskell (academic). Although I love how Haskell allows users to (for example) define their own control-flow statements, that "bag of primitives" nature leads to Haskell projects easily forming their own dialect of the language[0] without even having to resort to macros.

[0] This is also a common criticism of Lisp and Forth.

← PreviousPage 5 of 34Next →