I think my mistake was treating it like any other unix-y [multi-user] system, because I installed everything with one account, then created a dedicated 'build' account for the agent to run as -- exactly as our linux (and even, ahem, Windows) agents were setup. This caused all kinds of failures for the 'build' user as everything was owned by and had permissions setup for the first user.

I came away kind of disgusted at the horrible patterns in use. Windows has spent two decades cleaning up that mistake, and here's a comparably new system making the same, awful mistakes?

User-writable/owned stuff goes in ~. Period.

Then GNU took over and made some opinionated choices about how those subdirectories were to be used, which is fine.

However, because /usr/local is under /usr, it meant that it was more difficult to mount /usr as read-only or shared. There is also the issue of whether files under /usr/local should be owned by non-system users.

A while back, the creation of /opt and /opt/{some-app} (and /var/opt/{some-app} became a way to avoid writing into /usr and also made it possible to mount applications with their code as read only with writable locations under /var.

So I've taken writing to /usr/local as a packaging smell these days, purely to avoid writing into /usr.