And beyond this you're also speaking of something different. You're talking about taking some output assembly and then modifying it in 'negligible' (as a percent of the whole) ways, as opposed to writing assembly oneself. For a program of minimal complexity, not only would it would require an unreasonable amount of effort, but you'd also invariably write inefficient assembly relative to a compiler. Compilers produce code that is better, faster, and cheaper.
---
Shifting outside the abstract, let's just do an example. Imagine you have a processor running at 3 billion cycles/s and you have a program that takes up to 1 million single cycle ops/s. And then some ultra clever human tunes a bit of the assembly and we get it down to 0.95m cycles per second. In reality you're probably not going to be able to tune your way to a generalizable 5% boost, but even if you do in this scenario - what have you achieved? Literally nothing. Of course this argument, taken to extremes, is why it now takes more processing power to run a text editor than it did to put a man on the Moon, but we're not talking about extremes.