That is because rustc really maxes out the cores.
Ask your agent to build a simple bash/zsh/whatever cargo wrapper that maintains a machine build lock.
It should exclude cargo check, tree etc. ofc.
CPU contention from parallel cargo runs absolutely kills (build) performance.
Also my setup has instructions that agents keep working if they have open items while the lock is taken; i.e. launch a subagent or a shell with a watcher to run the build and don't block on it.
This makes all the difference when you have multiple agents on multiple workstreams on the same machine and they target Rust.