The two biggest surprises to me so far:

1- About 80% of tests are producing results that don't show increase / decrease

2- Assuming things will work better is completely off. I was convinced my lightbox idea from the article was easier for the user but turned out it was annoying.

One thing is a balance of making things too annoying vs getting features that help us grow. An on going challenge of always trying to do right by our visitors and customers.

It's interesting where about 80% of our test don't produce any significant results. One test nearly doubled our conversion of email and another one that is on-going now has done about a 30% increase.