Yeah that "premature optimization..." trope always bugs me. It is not a license to write the most non-performant code you can think of and then spend hours and hours tuning it later. That phrase was coined back when optimization meant inlining assembly, not simply putting some thought into your algorithms.
Part of the problem is TDD, people see the green bars fill but the customer regects it because guess what. In real world usage we arent running the method once, we are running it millions of times.