As HW guy you are making the hardware so you can make a mock of it relatively easily, that is not the case if driver is essentially reverse-engineered out of how windows driver worked.
And many other drivers are "some vendor made it, and they won't tell us how the chip actually work, just give us driver code".
Then as other people mentioned hardware isn't exactly 100% predictable, HW bugs happen, firmware upgrades change how stuff works etc. so your test suite might be entirely unable to catch that.