It's a strategic decision. If you punt you are signalling to the other team that you don't think you can get a first down in one more down.

Example: It's fourth down, (your last chance), and you have eight yards to go, that means you went two yards in three downs. You can punt and push the opponent back further, or go for it and risk giving them the ball where you are now. Most of the time you would choose to punt. You give up.

I usually interpret it to mean something more like "Cutting your losses".

It's because you still have the one down left to try to get a first down, but instead you're giving up on the offensive drive by punting.

Also, I wouldn't say that it's completely synonymous with "giving up". It's more like "giving up for now".

In the context I usually hear it (features vs. time) - you typically "punt" on something when you're no so much giving up, but putting it off for a later time or release.

I've also seen it used to mean letting someone else deal with it.