The real issue is that you shouldn't have much code to clean up in the first place.
If you occasionally need to hack something out to ship it before its effect is lost — go for it. Clean it up the next time you touch it.
If you're constantly writing code under pressure like this, you have a management problem. Your boss probably doesn't code. Sucks for you.
If someone constantly ships poor code, you have a talent problem. Mediocre talent can kill a startup. Get rid of the bad apple ASAP.