LayerFS: A filesystem for AI coding agents
layerfs.ai
layerfs.ai
https://www.usenix.org/conference/usenix-atc-10/apiary-easy-...
https://www.usenix.org/legacy/event/lisa11/tech/full_papers/...
ephemeral, instant boot strap, append only (I'm pretty sure I was the first person to use ephemeral computing this way, at the time I couldn't find anything that touched it. People used containers, ala ZFS/Zones, but they were generally meant for long term computing and not for bringing up an environment for a very small period of time and then tearing it down.)
This paper with bellovin applied it to system administration - https://www.usenix.org/legacy/event/lisa09/tech/full_papers/...
though we also approached recording and forking your computation at will - https://dl.acm.org/doi/10.1145/1323293.1294289 (I'm proud of coining the name), though it strongly relates to Microsoft's recall so it would probably be viewed less positively today (in fact, this might be closer in implementation to what they are doing than the above).
with that said, it was a bit more tongue in cheek than anything.
as an aside, I sort of wonder in the age of AI and the ability to reproduce anything easily, we might have to expand patents (related to comment below that got downvoted), but dont have strong thoughts on this yet.
PS: it’s just a fun play on memory/storage and AI. Hope you enjoy :)
Here’s the first write up with the CLI version: https://nicobrenner.github.io/lossy-git/
And repo code: https://github.com/nicobrenner/lossy-git
Celebrating and promoting this stuff does not help us get better at discovering and building actually new things; change my mind.
I am the founder of Substrate, a project that aims to provide this capability: having just one memory for all your agents, sessions and tools.
What two harnesses do you switch between most often?
This is not a technical restriction, but a commercial one. Any vendor agnostic harness can implement session sharing as a feature, but because you would be made to pay higher token pricing, there will be little interest among the consumers.
What kind of memory has degraded your performance? Memory is quite tricky to tune and simple a direct implementations of an LLM Wiki concept can indeed have a lot of problems, as context accumulates.
Even those bashing for being AI generated have to admit it’s technically impressive and reflective of where software engineering is headed.
Do not be fooled by the "vibes" of the website design which many here are. If it is too good to be true, it most likely is.
And still crashes and corrupts filesystems all the time.
(I have one server platform where just doing `insmod btrfs` immediately locks up the server, hard, without even any kernel panic or log message.)
Only in a sense that i went to a battle and lost all of its data to corruption. Should be removed from in-tree so people don't accidently use it.
https://layerfs.ai/benchmarks/dsh-dedup/
I think it’s actually a fairly useful thing but I worry about something so obviously vibe coded hitting 1.0.0 and without a track record but the ideas seem pretty useful. I’ve certainly found the work tree semantics most agents and harnesses follow at scale to be obnoxious at best and it seems to tackle issues head on.
They even have some explanations. Files writes in btrfs snapshots recreatethe contents as opposed to storing what's changed. Btrfs has no dedupe, if multiple snapshots happen to have the same data. And small files store metadata twice in btrfs for reasons.
I do think their use case is a bit pathological, checking out the same harness at every commit. Most of these btrfs gotchas don't feel like they would be super impactful to me. But really cool results none the less.
I guess the win is that often build artifacts are the same in many cases, so a pristine checkout still de-dupes. There's very good build caching systems out there that help with not needing to keep re-compiling the exact same stuff you've compiled, so I tend to discredit this usage some. But each of those workspaces would end up with it's own copy of the build artifacts, so there is a real savings to layerfs in either case, with it's dedupe.
Good for you on your "speculative merge topologically sorted task list merging into a merge train". Fun hard problems to have! Great. I don't want to be too rude, but also, personally, I think you could have simmered that down a bit, at the cost of not presenting quite entirely how super advanced elite super cool you are.
If your commitment is just to bragging keep doing what you're doing. I'm just trying to help you be effective. Apologies, I guess? You seem not only not interested, but you're showing some gross colors in the process (virtue is good, actually).
PS, it’s about builds on the same tree but about edits on the same set of files in the tree that would cause block COWs in brtfs. If you read the original links docs you will see that’s the use case they’re targeting, and it’s a lot different than build artifacts. Their proposal isn’t just assisting in deduping but in merging rapidly files that have interleaved changes.
That all made me pretty sore. Complaining about virtue signalling & this post here, well, we're done here. I don't feel like you've done anything to connect me with the problems. Btrfs seems like it would work just fine for your use cases. You haven't talked to the problem or drawn any connection. Not really interested anymore. Good luck.
I disagree. Its full of AI slop. Too much animation (cute but distracting) I literally tried for 20 minutes and couldn't focus on reading. I just couldnt comprehend anything.