Hacker News Reader

Comment by fnordpiglet | Hacker News Reader

I have this pattern but because I run pipelined agents working on a speculative merge topologically sorted task list merging into a merge train. This leads to lots and lots of work trees that largely overlap except for slight variants. Their articulated use case is mine, and I can have up to a hundred work trees at a time. The storage and related back pressures can be complicated.

View on HN

For the source code part of that, the difference between using layerfs and btrfs snapshots feels near trivial. There'd be so little difference at the source layer. Unless your changes are massive massive massive repo wide changes, most of the content is going to remain the same, and the btrfs snapshot is going to mean each workspace is a proportionally a tiny delta, have as many as you want.

I guess the win is that often build artifacts are the same in many cases, so a pristine checkout still de-dupes. There's very good build caching systems out there that help with not needing to keep re-compiling the exact same stuff you've compiled, so I tend to discredit this usage some. But each of those workspaces would end up with it's own copy of the build artifacts, so there is a real savings to layerfs in either case, with it's dedupe.

Good for you on your "speculative merge topologically sorted task list merging into a merge train". Fun hard problems to have! Great. I don't want to be too rude, but also, personally, I think you could have simmered that down a bit, at the cost of not presenting quite entirely how super advanced elite super cool you are.

Perhaps but it wouldn’t have been explicit about why I have that problem. I don’t think it’s a normal problem to have, whether it’s elite or not, it’s IMO not common. The speculative pipeline and the merge train is the key part because they overlap with changes in the same files. This is why block based doesn’t help me as much. I’d prefer to not censor myself with respect to my problem statements for some internet humility virtue signaling.

"I have lots of builds on the same tree" is not really losing much relevant signal. I don't think I'm trying to make you earn "virtue signaling" humility points, I'm trying to increase the chance people understand and ascertain what you're actually saying. Without having to spend time digging through all the inflated noise you're giving off. Communication and brevity are worthy objectives.

If your commitment is just to bragging keep doing what you're doing. I'm just trying to help you be effective. Apologies, I guess? You seem not only not interested, but you're showing some gross colors in the process (virtue is good, actually).

As much as I appreciate your intention might be to be helpful it was in fact quite rude and arrogant. I think telling someone in a technical forum to not be so technical and lecturing them on clarity of communication is pretty offensive - you don’t know me, know what I do or who I am, what my background is, and clearly my message wasn’t intended for your consumption since you found it not for your consumption. Some advice since we are dishing it out, this sort of stuff is as annoying as correcting grammar on Usenet posts or all the other forms of things that disingenuously begin with “not to be rude” - if you are aware it is likely to be read as rude, maybe just not type it next time? Just trying to help you be effective and not show such gross colors next time.

PS, it’s about builds on the same tree but about edits on the same set of files in the tree that would cause block COWs in brtfs. If you read the original links docs you will see that’s the use case they’re targeting, and it’s a lot different than build artifacts. Their proposal isn’t just assisting in deduping but in merging rapidly files that have interleaved changes.

I started on a bad foot when you messaged me, and here's why it rubbed me wrong: it seemed like btrfs snapshots would give you 98% of the wins just as easily. With far better reliability than using a nosync sqlite as a backing store. And I'd talked thought some of the differences. But it didn't seem like you were connecting with or going further into any of the material or tradeoffs or how the two would meet your workload. You didn't connect your work to the subject at all other than to show up and plant a flag. I found it rude and arrogant that you'd just show up to share your thing and be so technically disconnected from the topic.

That all made me pretty sore. Complaining about virtue signalling & this post here, well, we're done here. I don't feel like you've done anything to connect me with the problems. Btrfs seems like it would work just fine for your use cases. You haven't talked to the problem or drawn any connection. Not really interested anymore. Good luck.

Reply on news.ycombinator.com