I'd be quite curious to see how using the Magic Nix Cache Action compares to your setup in terms of performance: https://github.com/DeterminateSystems/magic-nix-cache-action
(full disclosure: Determinate Systems employee)
(full disclosure: Determinate Systems employee)
Just need to figure out what the cache size limitations are..