A few years of work (10k edits) and a few years of dwindling participation on my side someone noticed that quite a few of those early admins never faced a vote at all. The process had re-elections when 25 wikipedians asked for a vote, took them almost three weeks, I got that treatment as well in 2009. Indeed someone had enough time to dig through and find a discussion where I wasn't the nicest person (at the same time writing and discussing on Wikipedia help me a lot to develop a healthy social skill). Well, I didn't use the admin rights anymore so I rather resigned before someone dug even deeper ;)
For security reasons those admin rights should be time limited anyways.