HNHacker News
TopNewBestAskShowJobs

jlouis

2,358 karma · joined June 21, 2008

Hacker. Prefers functional programming.
submissionscomments
jlouis··on There is no memory safety without thread safety
The definition has to do with certain classes of spatial and temporal memory errors. Ie., the ability to access memory outside the bounds of an array would be an example of a spatial memory error. Use-after-free would be an example of a temporal one.

The violation occurs if the program keeps running after having violated a memory safety property. If the program terminates, then it can still be memory safe in the definition.

Segfaults has nothing to do with the properties. There's some languages or some contexts in which segfaults is part of the discussion, but in general, the theory doesn't care about segfaults.

jlouis··on Why Elixir? Common misconceptions
The core of a properly built, resilient/robust system is that you have compartmentalized code into different small erlang processes. They work together to solve a problem. A bug in one is isolated to that particular process and can't take the whole system down. Rather, the rest of the system detects the problem, then restarts the faulty process.

The reason this is a sound strategy is that in larger systems, there will be bugs. And some of those bugs will have to do with concurrency. This means a retry is very likely to solve the bug if it only occurs relatively rarely. In a sense, it's the observation that it is easier to detect a concurrency bug than it is to fix it. Any larger system is safe because there's this onion-layered protection approach in place so a single error won't always become fatal to your system.

It's not really about types. It's about concurrency and also distribution. Type systems help eradicate bugs, but it's a different class of bugs those systems tend to be great at mitigating.

However, if you do ship a bug to a customer, it's often the case you don't have to fix said bug right away, because it doesn't let the rest of the application crash, so no other customer is affected by this. And you can wait until the weekend is over in many cases. Then triage the worst bugs top-down when you have time to do so.

jlouis··on Asynchrony is not concurrency
A good way to gauge if "asynchrony" is a term we need is to test if it is useful in other contexts than just a single language, or a single concurrency design.

If it's needed to reason correctly in a wide set of concurrency models, then I'd say it's going to be a useful addition. If not, then I'd say it's not really worth using in the grander scheme of things.

I.e., does this make any sense in Haskell, Erlang, OCaml, Scheme, Rust, Go, .... ?(assuming we pick one of the many concurrency models available in Haskell, Rust and OCaml).

More generally: if things are cooperatively scheduled, then there's a need for attention to additional details. This is because it's much easier for a bad piece of code to affect the system as a whole, by locking it up, or generating latency-problems. In a preemptively scheduled world, a large group of problems disappear instantly, since you can't lock up the system in the same way.

jlouis··on Asynchrony is not concurrency
SIMD is parallel execution with no concurrency.
jlouis··on Fundamentals of garbage collection (2023)
You can make a programming language where cycles are impossible. Erlang is a prime example.

Region inference is another strategy in this space. It can limit the need for full-blown garbage collection in many cases, but also comes with its own set of added trade-offs.

Reference counting is just a different kind of garbage collection, really. It acts like a dual construction to a tracing GC in many cases. If you start optimizing both, you tend to converge to the same ideas over time. Refcounting isn't void of e.g. latency problems either: if I have a long linked list and snip the last pointer, then we have to collect all of that list. That's going to take O(n) time in the size of the list. For that reason, you'd have to delay collecting the large list right away, which means you are converging toward a tracing GC that can work simultaneously with the mutator. See e.g., Go's garbage collector.

jlouis··on Unexpected security footguns in Go's parsers
It's not very novel. There's far better ways of solving this than allowing a random string to be embedded as aux information to a struct field. Examples: F# type providers, or OCamls PPX system for extending the language in a well defined way. Macro rewriting systems also allow for better safety in this area.

This allows you to derive a safe parser from the structural data, and you can make said parser be really strict. See e.g., Wuffs or Langsec for examples of approaches here.

jlouis··on OxCaml - a set of extensions to the OCaml programming language.
Everything I see in OxCaml are things where I go: this would be nice to have in Ocaml.
jlouis··on Lottie is an open format for animated vector graphics
Zipped JSON is just massive waste of energy in a battery. We don't want that.
jlouis··on Ask HN: Cursor or Windsurf?
Anything which can't exfiltrate your data
jlouis··on GCC 15.1
Not a niche feature. Fundamental for any decent language with a type system.
jlouis··on Concurrency in Haskell: Fast, Simple, Correct
Or you do as Erlang's BEAM VM: each thread has it's own memory area which is GC'ed individually. This means upon request termination, you just terminate the thread and the memory is reclaimed with no need for a GC.
jlouis··on Concurrency in Haskell: Fast, Simple, Correct
Yes. And immutable data structures.

When data is immutable, it can be freely shared. Changes to the data essentially uses copy-on-write. And it only writes the delta change, since you don't need a deep copy due to immutability. Add that the garbage collectors of Haskell and Erlang are designed to work with a high allocation rate and have 0 cost for dead data, and this is much faster than what people think.

The way you implement a webserver in either Haskell or Erlang is rather trivial. Whenever there's an incoming request, you make a thread to handle it. So you don't have 1 webserver serving 10k requests. You have 10k webservers serving 1 request each. And since they are started from the same core data, they'll share that due to immutability. See also old-style Apache or PHP and fork().

jlouis··on Erlang's not about lightweight processes and message passing (2023)
Bandwidth vs latency. Erlang is designed to keep low latency under load with graceful degradation.
jlouis··on Go Optimization Guide
It is fairly common your type system ends up with escape hatches allowing you to violate the type rules in practice. See e.g., OCaml and the function "magic" in the Obj module.

It serves as a way around a limitation in the type system which you don't want to deal with.

You can still have the rest of the code base be safe, as long as you create a wrapper which is.

The same can be said about having imperative implementations with functional interfaces wrapping said implementation. From the outside, you have a view of a system which is functionally sound. Internally, it might break the rules and use imperative code (usually for the case of efficiency).

jlouis··on 'Destiny 2' Reveals Its 'Content Shredder' as a Result of Recent Lawsuit
The underlying thing was that the Beyond Light expansion revamped all of the engines light model. Destiny 2 runs on pretty old console hardware, and to do so, it doesn't dynamically compute a lot of stuff. There's a lot of pre-computed data, and lots of manual placement of light until it looks good.

Hence, a change in light model means redoing light for all content. By reducing a large amount of content, you claw back to the point where it is viable to do so in a single expansion cycle.

And that's just one part of the problem. I'm sure there's bit-rot creeping into game logic and effect-rendering as soon as you don't maintain a piece of the game actively. Add a few years, and you basically have to redo a lot of work from scratch because the active part of the game has likely moved on.

jlouis··on Open source AI: Red Hat's point-of-view
Turing completeness makes it a different problem.
jlouis··on Who Can Understand the Proof? A Window on Formalized Mathematics
Define "meaningful".

A lot of mathematics is about exploration. You look at some system and try to figure out how it works. If you have a good idea (conjecture), you try proving or disproving it. The goal is to gain some understanding.

Once in a while, it turns out that exploration hits the gold ore in the mine. You get something that applies to solving some problem we've had for years. As an example, algebra was considered meaningless for years. Then cryptography came along.

There are other good examples. Reed-Solomon coding relies on a basis of finite fields.

The problem is we don't really know when we'll strike gold in many cases. So people go exploring. It also involves humans, and they can disagree on a lot of things. Stephen seems to run into disagreements more often than the average researcher, at least historically.

jlouis··on Haskell vs. Ada vs. C++ vs. an Experiment in Prototyping Productivity (1994) [pdf]
It's interesting that we've known results such as these for 30(+) years, yet 99.999% of all software is still written in outright miserable languages such as Python or Javascript...
jlouis··on Go Protobuf: The New Opaque API
The big problem with Gzipped JSON is that once unzipped, it's gigantic. And you have to parse everything, even if you just need a few values. Just the memory bottleneck of having to munch through a string in JSON is going to slow down your parser by a ton. In contrast, a string in Protobuf is length-encoded.

5-10x is not uncommon, and that's kissing an order of magnitude difference.

jlouis··on Arm Accuracy Super Resolution
A key component is that when a game generates pixels, it also generates other data. Such as a Z-buffer for pixel depth and motion vectors (for motion blur, among other things). Feeding this to the upscaling algorithm vastly improves image quality.

The larger problem to solve with these upscalers are temporal artifacts when using the above information. This lead to a number of heuristics, but if you train a neural network to do the work, it tends to perform better than the heuristics. There's still a ton of research going into non-ML solutions however, because the current console generation doesn't have AI-acceleration available, nor does a lot of hardware out there. So there's some longevity to be had by doing a good job for all that hardware out there.

jlouis··on Dada, an experimental new programming language
It is a fundamental question in your language design. Some languages make side-effects explicit one way or the other. Other languages handles side-effects in a more implicit fashion. There's a tradeoff to be made here.
jlouis··on Tacit programming
This technique is highly useful in moderation, provided you are working in a language where the syntax/semantics affords it. It can certainly be overdone to the point where the code becomes harder to decipher for a human reader.

A key step is that programming languages which use the style builds a vocabulary of commonly used operations and make these into general knowledge for programmers using the language. Thus, succinctness is obtained and you can compress hundreds of lines into a few. Flip side is a steeper learning curve, which has to be balanced against.

jlouis··on A response to "Erlang – overhyped or underestimated" (2010)
I wrote this 13-14 years ago. Time flies I guess.

Keep in mind, it's frozen temporally at that point in time, and a ton of stuff has changed in computer ecosystems since then, and so has some of my viewpoints.

jlouis··on FreeBSD 14.0 Delivering Great Performance Uplift
Once you've seen the (older) alternatives, you cringe at Docker, indeed. Jails is a better solution. So is (Solaris/Illumos) zones.

Nix is a great way to jump over docker entirely. You can then output a container from Nix if you need it, but you aren't limited by them at all.

jlouis··on Lean4 helped Terence Tao discover a small bug in his recent paper
The key point is that most software don't need correctness to the level formal verification provides. There's a subset of software however, for which there's no substitute for a formal verification process.
jlouis··on Log is the "Pro" in iPhone 15 Pro
I'm suggesting rec.709 because it's what is a currently expected default for a screen. In your typical setup, your working color space is something like ACEScct or DWG, so you can map to several possible output formats with a little extra work if needed.

The integer values are nice because existing video formats encode things as integers. So we can just stuff our stuff inside a codec we already have rather than having to reinvent the wheel on the codec side as well. Re-purposing existing toolchains for new uses tend to a thing that gets a lot of traction compared to building a new one from scratch. Even if the newly built toolchain is far better.

jlouis··on Log is the "Pro" in iPhone 15 Pro
Yes, you can combine the steps to make it more efficient. In that view, it's "just" a different gamma curve. It's far harder to "split" a curve than combining two steps though.
jlouis··on Log is the "Pro" in iPhone 15 Pro
> This is definitely very useful (and is the norm if you want to do something like, say, high quality scanning), but I failed to see how it warrants a new "format".

This warrants a separate answer. Cameras are getting to the point where they can capture far more information than we can display. Hence, we need a lot of bit depth to accurately store this added precision. But adding bits to the data signal requires a lot of extra bandwidth.

In principle, we should just store all of this as 16/32bit FP, and many modern NLEs use such a pipeline, internally. But by creating a non-linear curve on integer data, we can compress the signal and fine-tune it to our liking. Hence we can get away with using the 8-12bit range, which helps a lot in storage. With log-curves, 12bit is probably overkill given the current sensor capabilities.

There's a plethora of log-formats out there, typically one for each camera brand/sensor. They aren't meant for delivery, but for capture. If you want to deliver, you'd typically transform to a color space such as rec.709 (assuming standard SDR, HDR is a different beast). The log-formats give you a lot of post-processing headroom while doing your color grading work.

jlouis··on Log is the "Pro" in iPhone 15 Pro
The transfer functions in your (rec.709) color space is non-linear indeed. However, the pixel values you store are in a linear relationship with each other. The difference between values 20 and 21 are the same as the difference between values 120 and 121, assuming an 8bit signal. I.e., the information is the same for all pixels. Further down the chain, these values are then mapped onto a gamma curve, which is non-linear.

What the "log"-spaces are doing is to use a non-linear relationship for the pixel values, as a form of lossy compression. If the signal has to factor through 8bit values, using a compression scheme before it hits the (final) gamma curve is a smart move. If we retain less precision around the low and high pixel values and more precision in the middle, we can get more information from the camera sensor in a certain region. Furthermore, we can map a higher dynamic range. It often looks more pleasing to the eye, because we can tune the setup such that it delivers a lot of precision and detail where our perception works the best.

In short: we are storing (8bit/10bit) pixel values. The interpretation of these values are done in the context of a given color space. In classic (rec.709) color spaces, the storage is linear and then mapped onto a non-linear transfer function. In the "log" spaces, the storage is non-linear and is then mapped onto a non-linear transfer function. In essence we perform lossy compression when we store the pixel in the camera.

jlouis··on Log is the "Pro" in iPhone 15 Pro
In videography the term "log" is heavily overloaded and you'd want to ask for more detail in order to figure out exactly what is meant.

A pixel value, be it integer or floating point, means little on its own. There's a context for that value which is a color space. In the typical process, you have several color spaces in play: the camera has one for capture. There's one for color processing (the "working" space). And there's one for the display. When a pixel goes through the pipeline, it's processed via color space transformations.

In the "classic" color spaces, the pixel values have a linear relationship, and all of them carry the same amount of information. The "log" color spaces all have a non-linear (gamma) curve: they retain less information at very low and very high pixel values, but subsequently retain more information in the middle. It's a form of compression.

The human eye doesn't respond equally to all levels of brightness, so throwing away detail at the ends for more detail in the middle is usually a great choice. We retain information in the signal at the brightness level where the eye is able to perceive small details and texture, while throwing away information in the signal where it isn't.

We can now map more dynamic range into the same amount of bits, due to our non-linear compression. How large a dynamic range is given by underlying color space we are operating in.

If you go up in camera quality, you will typically see pixels use 10bits or more for their values. Combined with a log-curve, this leads to more information density, which allows capture of an even higher dynamic range. In turn, post-processing can now fix e.g. exposure to a much larger extent.

Finally, a LUT is linear approximation. "Real" color space transformation will use the underlying mathematical curves for much greater precision.

← PreviousPage 2 of 22Next →