My Thoughts on Monorepo | Hacker News Reader

My Thoughts on Monorepo

shekhargulati.com

37 pointsshekhargulati18 comments

Well informed article, thank you. In practice I'm pretty happy with a huge monorepo, but have a somewhat surprising problem with it: it makes fine-grained code reuse too easy. People end up adding tons of dependencies, and building anything requires building ten thousand targets at head. (It seems the npm ecosystem ran into the same problem from another angle, they also made fine-grained reuse too easy and ended up with huge dependency trees.) In theory, the solution is using larger libraries with less frequent releases, but then we run into lag and the diamond problem again. I don't know how to square this circle.

That's why bazel has visibility rules. You have to explicitly mark things public or express which package can import which package. If used judiciously it can help keep things in check or at least have the teams involved be aware of the maintenance burden.

Atomic commits are a red herring because deployment isn’t atomic or irreversible. You still have to think about how the system stays healthy while a new build is halfway deployed. A deprecated API still needs to work until you’re certain you will never roll back any consumer and start calling it again.

At least with a monorepo you only have to worry about deploy ordering. With a multirepo you have to worry about code review and merge order before you get to worry about the deploy order. I would like to be able to review the library API change together with the changes in the app that need the new API. At my work we either review library changes in isolation, or we skip the review on library changes (if the author deems them trivial) and only do review on the main app changes, but that means we often miss opportunities for making the library API fit the use-cases as well as they could.

Having tried both worlds. If a company offers me a job and says they develop in a monorepo I am not going to tell them to f*ck off. I would just ask for twice the salary I would agree to work for in a multirepo company.

Stopped reading when less than 10 seconds in a sign up pop up covered the article.

No popup for me in Firefox + uBlock Origin. Have you tried a reputable ad/tracking blocker?

Yes, I am also using Firefox with uBlock Origin locked down - no 3rd-party scripts, 3rd-party no frames, etc... Local network also has a pi-hole. I should clarify that it was an overlay style popup - not an actual browser window popup.

I also didn't get a popup. I then tried after disabling Ublock and Privacy Badger in a private browsing window to ensure that no cookies affected the result. After doing this, I still did not get any popup.

The disadvantages are more like "it's not affordable"

``` Monorepos could slow down developers because of slow build times, poor tooling, and merge conflicts.

Most developers still in 2020 struggle to cleanly merge code. Git is slow for projects with large numbers of files and history. There is cognitive overhead involved as developers have to get comfortable with a much larger code base than they would have with multirepo setup.

To do monorepo well require investment in tooling that most organization non-tech leadership will fail to understand ```

Monorepo is clearly more desirable than distributed repo. But it's not a cheap solution.

> Most developers still in 2020 struggle to cleanly merge code

Sounds like the hiring bar is too low.

We regularly¹ have unclean merges at my company, and I consider my coworkers competent, and well above "the bar". The biggest problem we have with clean merges is that nearly every CI system & code review tool out there doesn't test the final product — the merge commit — until after it is merged. GitHub follows this flow by default, and usually even with more CI tooling on the side. Something like bors-ng is what one has to adopt. (And we're struggling to do this for our monorepo due to other points the article touches on.)

While it is possible (if difficult) in git, some VCS systems struggle even harder with this: Perforce and derivatives, for example, simply can't, by design, perform atomic test & commit.

¹like once a month, or so, over the whole company

Could this be fixed by using a better branching strategy?

This is my experience too. Traditional ci hygiene requires that the build runs clean on the developer machine before publishing a change. But once the build time crosses 10 minutes, people resent waiting and start to run test subsets, and count on the ci server to catch breakage. Once that becomes normal, breakage is inevitable. And if commit tempo is higher than a developer can finish a test-run, then it is impossible. Solutions include patch queues, pr builds, etc. but those are all more complicated than a single stream on head.

merge into a stabilize branch and test that...

This would be a rose by a different name. So long as you're testing your branch's state, and not the end result of the merge — as most CI systems & GitHub end up pushing people towards — you're going to run into problems.

What makes testing the end result of the merge tricky, I suppose, is that it changes over time. (As other people merge into whatever "stabilize branch" you have, the result of merging your branch and the new HEAD of the stabilize branch must change.)

I haven't used github in a while, when does an unclean merge happen? When two people merge their branches at the same time?

Any two commits that conflict in a semantic sense (either change individually is okay, but both together don't work), but don't conflict in a syntactic sense (the changes are not textually near enough to each other to cause git to consider them conflicts, so an automatic merge can & will still happen; alternatively, they make the exact same change, and so don't conflict).

We see this often with lockfiles (someone updates a dep, someone else adds a new dependency; the merge is a new dependency to a version of the dependency that no longer exists), or version numbers that get incremented (each author increments once; the second merge, git will consider it the same add on both sides, and not a conflict, so the result it original_version+1, when it should be original_version+2).

Reply on news.ycombinator.com