It's not shifting the goal posts, because when the cost you pay has zero measurable effect, there is literally no difference. It's like if you shaved a milligram of material from a basketball player's shoes, they'd probably be able to run and jump slightly faster, but immeasurably so to the point that people would be much more inclined to look at you like an idiot rather than go grab a razor, if you ever suggested such.
And beyond this you're also speaking of something different. You're talking about taking some output assembly and then modifying it in 'negligible' (as a percent of the whole) ways, as opposed to writing assembly oneself. For a program of minimal complexity, not only would it would require an unreasonable amount of effort, but you'd also invariably write inefficient assembly relative to a compiler. Compilers produce code that is better, faster, and cheaper.
---
Shifting outside the abstract, let's just do an example. Imagine you have a processor running at 3 billion cycles/s and you have a program that takes up to 1 million single cycle ops/s. And then some ultra clever human tunes a bit of the assembly and we get it down to 0.95m cycles per second. In reality you're probably not going to be able to tune your way to a generalizable 5% boost, but even if you do in this scenario - what have you achieved? Literally nothing. Of course this argument, taken to extremes, is why it now takes more processing power to run a text editor than it did to put a man on the Moon, but we're not talking about extremes.