They would be beholden to political machinations which get them elected and not do a very good job at it.

There are a lot of things with HoL but being unelected is not one of them.