docfort

I once visited a town in Spain over which an ancient Roman aqueduct runs, elevated above most of the buildings, even today. It carried an essential ingredient foundational to life in the area, but after the fall of the Empire and the deaths of the architects and engineers who could understand it, the aqueduct stood in the midst of people who could not have repaired it. Relying on mission-critical technology divorced from human understanding is exceptionally risky. Obviously, people have done it before and will happen again, but it makes me nervous.

I think the first goal of writing is to talk to yourself. Without an external, unchanging scratchpad, most people cannot keep their thoughts truly straight. It can feel otherwise, but trying to write something about any tricky topic even a few pages in length can take a long time to be self-consistent. Clarifying it takes even longer, especially when you intend to share it. Writing as thinking is definitely a thing. I believe it’s the trickiest part of understanding how such interesting things pop out of a language model trained to predict the next token and rewarded with pleasing sequences. Since it is not thinking, it proves that certain sentences (most?) can be generated thoughtlessly.

Related: the Busy Beaver problem https://news.ycombinator.com/item?id=40857041

I know this is such a late reply, but for clarity: my foundational point is the exact opposite hypothesis. Free labor enables incentivized labor. Economic theory has a neat concept of externalities, a necessary mechanism that only that which can be valued can be traded. If we removed all economic activity, then after most people die, the remnants would revert to much earlier models of free cooperation existing in small communities. Our world does require huge numbers of people to forcibly cooperate to create resources to then allocate.

My one other point is that incentivized labor is not the same as the value it creates. Indeed, it must be less. Otherwise, our economic system could support only a fixed number of people (subsistence), which would decay inevitably because there is no margin for error. But my point is that margin in reality isn’t fully realized, even by trillionaires, because then there would be no growth to support more people growing in their standard of living. There must be slack in this distributed system and the slack wasn’t valued: it’s free labor. It’s mixed in with incentivized labor, so I understand if you reject the premise entirely, but I do believe this is the essence of modern (specialized) capitalism. If skilled workers try to optimize or invent, more resources will be available for distribution for the same incentive (i.e. “worker productivity”). You can say “yeah that’s their job,” and I can say “that productivity wasn’t fully monetized because otherwise productivity would be lower overall.”

So, incentivized labor presupposes free labor, and economic productivity is a mix of free and monetized labor.

Free labor enables capitalism, especially if you consider labor arbitrage as a mixture of free labor and properly compensated (according to the real value) labor. From literally being born, to family culture, education, and whatever level of broad social cohesion, it’s all free labor. Without that background, money itself loses its value, since an individual cannot have reasonable confidence in trading it for something of actual tangible value. It is abstract stored value, banked into society for free. Indeed, in many cases, the free labor is in the rational self interest of a group. But stability and love and peace aren’t monetized to their true value. Otherwise, markets should be much less stable. Bubbles are only notable for the large impact of a small group of bad actors. Overall, it’s pretty amazing what free labor does. Open source is just another instance of this long and critical tradition.

Interesting post, but the last bit of logic pointing to the Neural Engine for MLX doesn’t hold up. MLX supports running on CPU, Apple GPU via Metal, and NVIDIA GPU via CUDA: https://github.com/ml-explore/mlx/tree/main/mlx/backend

This was a passion of mine decades ago, but Putterman's lab jump-started interest after the cold fusion debacles. Some fun videos and pictures on the lab website. https://acoustics-research.physics.ucla.edu/sonoluminescence...

More specific info on the reference card is available in the paper's supplemental information. https://ieeexplore.ieee.org/ielx8/83/10795784/11125864/supp1.... Basically, they used special paper with a pro-Canon inkjet, along with a special ICC color profile.

The article says that implantation fails in humans 10-40% of the time. Your point is still valid, but the scale in reality is very significant.

Verification is indeed the majority of the time spent. Unlike programming, Verilog and VHDL and higher level things like Chisel aren’t executed serially by the hardware they describe like a von Neumann machine. Hello World for a chip isn’t designing the circuit, or simulating the circuit, or synthesizing the circuit to some set of physical primitives. No, it’s proving that the circuit will behave correctly under a bunch of different conditions. The less commoditized the product, the more important it is to know the real PDK, the real standard cell performance, what to really trust from the foundry, etc. Most of the algorithms to assist in this process are proprietary and locked behind NDAs. The open source tools are decades behind the commercial ones in both speed and correctness, despite heavy investment from companies like Google.

And so my point: the place where people best know how to make chips competitively in a cutthroat industry is NOT in schools, but in private companies that have signed all the NDAs. The information is literally locked away, unable to diffuse into the open where universities efficiently operate. Professors cannot teach what they don’t know or cannot legally share.

Chip design is a journeyman industry. Building fault-tolerant, fast, power-efficient, correct, debuggable, and manufacturable designs is table stakes. Because if not, there are already a ton of chip varieties available. Don’t reinvent the wheel because the intersection of logic, supply chain logistics, circuit design, large scale multi objective optimization, chemistry, physics, materials science, and mathematical verification is unforgiving.

There is some recent work [0] that explores this idea, scaling up n-gram models substantially while using word2vec vectors to understand similarity. Used to compute something the authors call the Creativity Index [1].

[0]: https://infini-gram.io [1]: https://arxiv.org/abs/2410.04265v1

Terry Tao has referred to this classification system as foxes vs hedgehogs. https://en.m.wikipedia.org/wiki/The_Hedgehog_and_the_Fox

I take your example on multitasking operating systems as not being limited to only helping make friendly asynchronous I/O, but I do think a deeper consideration of Multics is coincidently appropriate.

The telephone and electrical power networks were vast in scope (and still are), enabling interstate communication and power utilities. Echoes of the transportation utilities enabled through railroads. Multics was architected partially with the commercial goal of scaling up with users, a computing utility. But in a time with especially expensive memory, a large always resident kernel was a lot of overhead. The hardware needed a lot of memory and would be contending with some communication network whose latency could not be specified at the OS design time. Ergo, asynchronous I/O was key.

Put differently, Multics bet that computing hardware would continue to be expensive enough to be centralized, thereby requiring a CPU to contend with time-sharing across various communication channels. The CPU would be used for compute and scheduling.

Unix relaxed the hardware requirements significantly at the cost of programmer complexity. This coincided roughly with lower hardware costs, favoring compute (in broad strokes) over scheduling duties. The OS should get out of the way as much as possible.

After a bunch of failed grand hardware experiments in the 1980s, the ascendant Intel rose with a dominant but relatively straightforward CPU design. Designs like the Connection Machine were distilled into Out of Order Execution, a runtime system that could extract parallelism while contending with variable latency induced by the memory subsystem and variable instruction ordering. Limited asynchronous execution mostly hidden away from the programmer until more recently with HeartBleed.

Modern SoCs encompass many small cores, each running a process or maybe an RTOS, along with multiple CPU cores, many GPU cores, SIMD engines, signal processing engines, NPU cores, storage engines, etc. A special compute engine for all seasons, ready to be configured and scheduled by the CPU OS, but whose asynchronous nature (a scheduling construct!) no longer hidden from the programmer.

I think the article reflects how even on a single computer, the duty of the CPU (and therefore OS) has tilted in some cases towards scheduling over compute for the CPU. And of course, this is without considering yet cloud providers, the spiritual realization of a centralized computing utility.

It's too bad that they took the physicality of the monkeys into account, but assumed the key probabilities were IID. It would have been nice to see the effect of keyboard layout on the overall probabilities. Key mashing would clearly make nearby keys much more likely to be pressed in a sequence, implying that there might be an optimal keyboard layout for each phrase. And that’s before considering soft keyboards with autocomplete.

Thanks for the reality check. I was worried about how I could be conflating my own personal view as a parent with the popular narrative of "kids these days and their Instagram/TikTok." Probably says a lot more about me, but I vastly prefer the reading experience of a thick book on a phone than as a physical copy. And I have since I was a teenager (and it was just PDAs and clever TI-89 hackery).

I don't dispute the facts in the article, but this question kept popping up in my mind: how do they define reading time? I mean, in a too-pedantic sense, smartphone screen time is roughly divided into reading, viewing (photos/videos), and gaming. Given that they are not allowed to take the phones, it seems unlikely that the school knows a student's primary usage mode. For example, a student could be reading a bunch of fiction on their phone, thereby reducing their time in the school's library.

In other words, how holistic is the metric "reading time?"

A nice complement to this post is Mike Culbert’s article about how Apple chose to use 32-bit ARM6 instead of a 16-bit processor for reasons entirely unrelated to maximum memory size. http://waltersmith.us/newton/COMPCON-HW.pdf

I take your overall point, but for this specific complaint, there’s a shortcut: long press on the “+” button to take you directly to the photo pocket in Messages.

Complexity is the outcome of misunderstanding. The misunderstanding can come from lots of areas.

It could be from a requirements perspective: “I understand what I can build easily, but not what you want.”

It could be from an engineering perspective: “I understand what you want, but I don’t understand how to build that cohesively.”

It could be from a scientific perspective: “No one knows what tools we need to investigate this.”

I saw mentioned in other comments that CAD software doesn’t allow for sketching. As someone who was originally trained in drafting the old way, and who has used modern CAD systems to produce models for fantastically large physical simulations, I largely agree that sketching is lost. But the sketching that I can do on paper is just not at the same level of complexity as I can kinda do on my computer.

But the complexity of using the new tool obscures the fact that my model is much more complicated than I could otherwise manage using old tools. And that’s because I’m still learning. In fact, I have to “deploy” while I’m still in learning mode about a problem, unlike before, where I had to understand the problem in order to use the tools to draft the thing.

Being able to do something with a half-formed idea sounds like sketching, but when non-experts rely upon it, it’s pretty fragile. Because it wasn’t done.

Building a memex (something the author disparages multiple times) is super hard because we still don’t understand how to represent ideas separately from language, our original mental sketching tool. But people built Altavista and Google and LLMs anyway. And yeah, they’re super complex.

How does TCP/IP work over wireless connections? Poorly and with a lot of complexity. Why? Because the concept of a connection is ill-defined when communication relies on ephemeral signaling.

But despite the complexity, it is useful and fun to use only half-baked ideas. Just like it’s fun to use language to describe stuff I don’t understand, but merely experience. Graduation. Being a parent to a sick child. Losing a loved one.

The iTEAR 100 is not generally reimbursed by insurance. If you have an HSA or FSA, you might be able to get it through that. This whole medical device lifetime thing is because it’s regulated for its medical claims by FDA. Given some of the side effects noted here https://itear100.com/clinical-data.html, and how it changes the kind of tears that the user produces, I think it’s reasonable on its face to require a medical professional to review whether it’s working and needs more time with you every month.

I’m still surprised that this essay didn’t explore the nature of the audience. I’m not disagreeing with your take about the dangers of straw men, but PG does seem very much in the vein of essays are for communicating interesting stuff, or fun stuff, or provocative stuff. To someone! And it seems like too long of an essay to discover by the end this serious omission. A timeless essay is one that retains meaning to a person, or maybe many people, over the course of time. PG spent a while trying to explore the message in this essay and no time on trying to understand the audience.

There is research on humans showing that we can perceive at 500 Hz. There are devices that try to simulate a color by modulating a single LED (no color filter) and they don’t work on humans until you go past around 1 kHz.

https://www.nature.com/articles/srep07861

I think that for people without enough paid sick leave (maybe especially in the US), being knocked out for 2-3 weeks with a virus that is more infectious than flu is an event causing long term damage to their life. Especially with the repeal of many COVID-safety protocols at companies because of social pressure.

So I honestly thought that this was going to be another LLM article, but using the venerable ReLU activation function instead of the usual. A ReLU is exactly an if statement when rendered in a decision tree (if less than 0, emit 0; otherwise, emit the input). Given the relative popularity of the 4B parameter models (any transformer is dominated by the number of parameters in good old fully-connected feedforward layers), you can perhaps describe such models as 4B if statements. I was disappointed that the author didn’t go there as a means of parody.

It’s exactly this kind of thinking that underlies lossless text compression (not exactly what a transformer guarantees but often what happens). For that reason, some people thought it would be fun to combine zip and transformers. https://openreview.net/forum?id=hO0c2tG2xL

I think Lockdown would help here since it doesn’t decode message attachments. So the original link in the chain (decoding a PDF) would be impossible.

As for detecting unauthorized apps, I would imagine that once you’ve taken over control of the OS kernel, it’s game over for such software-based restrictions. The Halting theorem guarantees such limitations to any software-based restriction. And as long as you can form a Turing complete mechanism from pieces of the computer, such software limitations will apply.

It doesn’t use MPS, so at least that part is more interesting than the usual approach.

Bread is a great invention and a staple for many across the world. People have grown better at the process of developing grass into ever more productive wheat crops, transforming the inedible into something people find comfort in. But yes, there’s also an element of bread and circuses. Technology is not enough. It is not orthogonal to happiness, as you point out, but having the best bread doesn’t fundamentally make the course of humanity better. That begins and ends with what we individually and collectively choose: technology merely enhances our will.

Apple famously doesn’t do business units. It’s all in on functional organization.