If only the Linux kernel used as entropy everything that was available to it. That's the pitch of haveged(8) I think. There are significantly more timing events the kernel could use for input entropy that it's not. It would be great to see these timing events land in random.c, but they're not there.
Lastly, slightly off-topic, but the Linux CSPRNG is poorly designed. In reality, while preimage SHA-1 might still have a sufficient security margin, it suffers from poor performance due to ad-hoc design. Instead, using AES-128 CTR_DRBG with AES-NI would give the kernel CSPRNG a much needed performance boost, provide backtracking resistence, and remain secure. Current testing shows it could improve from 20 MiBps to 2 GiBps.