To get Rust incremental builds, did you consider using something such as crane https://github.com/ipetkov/crane ?

And regarding OCI images, i built nix2container (https://github.com/nlewo/nix2container) to speed up image build and push times.