romesmoke

I think your fix suffers from severe internal fragmentation.

It is well past due time to revisit memory allocation from first principles. I recently finished my PhD on the topic, and realized that we have yet to agree on a consistent definition of fragmentation (see Wilson et al's seminal survey on "Dynamic Storage Allocation"), much less how to deal with it.

When I was in the trenches, I had constant cognitive dissonance against the fact that (i) "it's a solved problem" and (ii) low-to-no-malloc was often presented as a feature of low-level code.

My take is, train a SOTA deep learning system on solved static memory allocation instances (which is NP-complete already) and build a dynamic malloc on top.

To see where the static SOTA was a couple years ago, see this: https://arxiv.org/abs/2504.04874

And also my general audience-friendly presentation of it: https://cappadokes.github.io/jekyll/update/2025/07/16/welcom...

Disclaimer: I've moved on past academia and I may have missed one or two breakthroughs. But the older the problem, the higher the probability of it staying open.

Most advice given by people unaffected by whatever comes out of whatever you do with the advice given is garbage (Nassim Taleb, Skin in the Game).

Sure: I can feel the inherent coolness radiating. Confessions of a mind witnessing its gradual metamorphosis into a cyborg. And sure, the productivity take-off sounds convincing. But each and every time I can't help but wonder: do these code-torrents water anything as frontier-worthy as the intelligence raining them? Where's the perfect, self-optimizing OS? Where's trading's Holy Grail? Where's the best-of-all-worlds Python-Rust-Zig hybrid?

In minds in danger of substituting counting, imitating, slop-spitting, relinquishing control for thinking, designing, daring in practice and not in prompting.

No hard feelings against the OP; they just happened to be the trigger. I fully understand I'm the shoemaker equivalent from 100 years ago. I just don't care--or rather care too much.

Thanks! Have these actively helped you? There's too much noise around this business...

I'm comfortable with Rust. Why the HW focus? I've heard about FPGAs being used for HFT, but I'm not going for sub-ms frequencies--except if it's unavoidable.

You sound lucky enough to have dodged self-hate. Happy for you. Enjoy your day.

Everyone has their values. It's OK if there's no universal system. I don't want to feel bad if a day passes and I haven't learned or done anything; I trust my intuition and experience to discern between peaceful and warlike periods. I'm all for doing hard stuff when needed; I'm all against the ideal of always; and yes, that ideal is tyrannical.

Life is not binary. Routines are not classified into hippie XOR hardcore. Not feeling bad for not improving yourself is a good sign of having actually improved. But that's just my opinion. Take care.

I wouldn't have commented if the article kept neutral on when to apply its ideas. But the push for daily learning-reflecting-etc is there. Even the segment you're quoting uses days for accounting. You're downplaying one's potential to read something like this on the Internet and try to literally live by it.

As for machines and tyranny, they're terms capturing the Zeitgeist all too well.

> Learn something every day.

It's fine if it doesn't happen every day. Don't be obsessed about this stuff. Forgive yourself if you ever behave sub-optimally. You're not a machine, neither should you be one.

Tyranny starts with the best of intentions.

The joke's on anyone outside of China thinking they're free.

> A version of Greg's research was published in the Wall Street Journal.

This is not research; it's an opinion piece by a Bridgewater employee that may prove right in the years to come.

Plus, the rhetoric of differentiating China from the "free world" paints a Big Red Flag over the article's face.

I spent the most part of my PhD trying to convince people that allocators are not simple. But the not-simple part is design, specifically the strategy/policy part. In other words, minimizing fragmentation by adapting to linked program behavior. From what I know, no product-level allocator and very few research-level ones go beyond a "one size fits all" approach.

The reason why this is so is pragmatic: fancy strategies take extra compute time that is too expensive when one's sitting on the critical path. So what follows has a good chance of not bearing practical value but (i) there are setups where memory capacity is a bottleneck and (ii) hey, it's just a PhD anyway.

So what does a fancy strategy even look like? Let's look at the simplest non-trivial version of the problem. Say, we know the sizes and lifetimes of all objects in advance, and we want to pack them in as tight a contiguous address space as possible. Mathematicians call this the Dynamic Storage Allocation (DSA) problem and it's NP-complete, i.e., no known optimal algorithm exists for the general case. Deep learning compilers battling the AI memory wall have been returning to DSA lately (it's an old problem).

The implication of the above is that a real allocator, that is, one forced to work under uncertainty w.r.t. object sizes and lifetimes down the line, can always fail remarkably. So going for a general-purpose solution begs that you know what you're doing (one could argue that most programs out there are "well-behaved", and I would agree up until the point where this observation is turned into a universal law).

Nevertheless, it remains a fact that writing a toy allocator is both educational and soberingly not hard.

I perceive a negative colouring of the "academic" notion in this post. And I totally get some of the pathologies implied: publish-or-perish is a harsh environment, lots of snobbery.

With all respect, however, this industry/academia dualism looks to me at best as a false and at worst as a harmful dichotomy. I mean, there are similarly cliché pathologies in whatever one chooses to overload "industry" with. There are successful academic projects that end up in industry (RISC-V) and successful industrial open-source (?) projects loved by academics (Rust). At the end of the day, everyone wants to build something. Tools and methods are needed. Some of them may not exist yet. The act of building one's tools and methods (or something completely new with existing tools and methods) is research regardless from where it happens or how it is labelled.

Why is this not enough? What extra benefits go with differentiating myself as either an academic or a practitioner? Honest question!

POV: CompEng PhD looking for industrial vacancies.

Location: Greece

Remote: Yes

Willing to relocate: EU or UK

Technologies: Rust, Linux

Résumé/CV: https://shorturl.at/0ZdaL

Email: cplamprakos@proton.me

There are countless pragmatic reasons to avoid a PhD, and no doubt both the article and other commenters will bring them up. The most constructive thing I can do is share a personal perspective.

I am 30 years old. I am working through my last few months as a computer engineering PhD student. Eventually, it went good. Not great (the world gives zero f*cks about my work, nobody has offered me a job yet), but not hellish either (didn't quit, still mostly sane, learned a ton of stuff that I never had the guts or prudence to delve into as an undergraduate, and most importantly, I decided I like computers).

Now my background is anything but academic: none of my parents finished high school, people from my village consider me either batshit crazy or a genius. I mean, I was thrown into the PhD archipelago by life itself, rather unconsciously. I just knew that "corporate IT" wasn't my thing, and as for the cool computing jobs, I wasn't their thing. Again, I spent my years as an ECE undergraduate burying my insecurities instead of building my future. To understand the degree of mental fragmentation I was under, I had never made the connection between my digital design courses and my operating systems courses (all of this is the story of the computer, stupid, it's in the title of your degree for God's sake!).... Anyways.

It took me six years to get to today. I am another person now. The PhD (well, and the pandemic, and all that followed) crushed all of my assumptions about the world, myself, the meaning of life. There's no way to put it in the condensed form that an HN comment requires without sounding naive, but I'm telling you the truth. Being forced to survive an alien landscape can make you whole again. It made me.

At the end of the day, talking sh*t about hard stuff is sooo easy. You could replace any polemic against a PhD with one against starting a family, or a company, or in any way rejecting "safety" for the potential of leaving your own mark on the world. Being you. Like that poem by Robert Frost, these things make all the difference.

Nah, the alternative that popped in my mind was "phantasizing", so I guess my wording aligns with my feelings. Whether these feelings are healthy is another discussion :D

Were I to design a computer (both HW & SW), the founding principle should be compatibility with legacy code at no expense of performance or user effort.

Stuff like this gives me hope that I'm not merely wet-dreaming.

I see your point. I've had my fair share of trouble as well. Talking about this stuff on HN after the storm has passed, from a financially stable place, is a luxury on its own.

Nevertheless, I don't see any value on acknowledging the delta between me and peers that happened to be luckier. It'd be useful were I on the other side: for instance, if I hadn't seen my father sink into dementia, if he was still with me, I'd better keep reminding myself of the importance and blessing of growing alongside a functional, healthy dad.

But now... Thoughts like "I have struggled more than these guys" seem dangerous to me. Whenever I've taken them seriously I've ended up using them as justification for the next tiny act of self-destruction.

Great, useful article. Remember: the same principle holds in the world of the psyche. Those people walking around, seeming effortlessly happy? Unless if they're little children, a horrendous amount of effort has been invested to their inner peace. It took me years to realize that. Being envious is much easier than being humble and diligent with getting one's shit together.

My wish for 2025 is for massive, decentralized, slow yet steady psychological magic. HNY, HN.

They claim their mission is to ensure that AGI benefits all humanity. What a preposterous lie.

I remember asking myself when ChatGPT was launched: "why would any sane person massively deploy such a thing?"

It's the money and the power, stupid.

OpenAI doesn't have any mission. To have a mission means to serve a purpose. To serve means to have higher values than money and power. And yeah, there is a hierarchy of values. The closest statement I can accept is "we will try to get all of humanity addicted to our models, and we couldn't care less about the consequences".

Wanna know why they'll lose? Because they'll get addicted too.

Wait a minute:

1. Formulating DSA as ILP doesn't scale. TelaMalloc starts from this fact as motivation.

2. There's no 1-1 mapping between tensors and allocations. Buffer re-use is important.

3. There's room for offline DSA instances as well. IREE's Stream dialect has a related transform (LayoutSlices.cpp).

Anyways, I'm no ML expert. DSA is much more generic than deep learning compilers. I can't wait to graduate myself and never hear the involved keywords again.

TBH I don't know what you mean either :D

I know that DSA is NP-complete, and none of the existing deep learning compilers implement the theoretical SOTA (you're probably referring to the 2+ε bound by Buchsbaum et al.)

I also know that scaling laws and the memory wall are soon gonna overpower the currently used heuristics (e.g., sort by size and do best-fit).

Anyways, I'm glad I "met" someone who has struggled against the same problem. Good luck with your research!

Hopefully I'll defend next summer.

Papers-and-code-wise, I haven't published the "final" thing yet. Right now I'm looking into integrating it with IREE, to get some end-to-end figures.

Hm, you sound like you approached it from the theoretical ML side. My strategy was much less sophisticated: I implemented the Buchsbaum et al. paper you posted in another comment of yours.

An interesting variation is Dynamic Storage Allocation, in which the rectangles can slide only alongside one axis.

AKA "static memory allocation", since the heights of the rectangles can be interpreted as buffer sizes, and their widths as lifetimes.

Most of my PhD's effort has been devoted to beating the SOTA in this. Problem's importance nowadays is owed to deep learning's memory wall.

The notion about "groups" is nowhere in the article. Your comment, however, is written as if belonging to some group is necessary. Worse, your comment delineates the example groups it gives based on social media apps!

...and there's the irony. Because in my view, this unquenchable need to tribalize and to identify enemy tribes is a direct consequence of the crisis described in the article. Which is fully aligned with my day to day experience BTW.

For a more elaborate, complete take on the value of attention, I can't recommend the work of Sir Iain McGilchrist enough.

The Elements of Computing Systems, by Noam Nisan and Shimon Schocken.

At the end of the day, CS deals with computers. The book guides you through building and programming your own computer.

I'm a CompEng PhD myself and going through it right now. I keep finding gems at every page.

Not working. The other day I uninstalled Skype from my cell because some group chat notifications had gotten past tolerable.

August is a sanctuary. It's the only thing I can hold every day of the rest of the year against Tech's meaninglessness. It's when I can read and do what I want instead of what I must.