It's not just the loss of an architectural register, it's also the added cost to the prologue/epilogue.
Even on x86_64, it can make a difference, in particular for small functions, which might not be inlined for a variety of reasons.
Aee also this benchmark from Phoronix [0]:
Of the 100 tests carried out for this article, when taking the geometric mean of all these benchmarks it equated to about a 14% performance penalty of the software with -O2 compared to when adding -fno-omit-frame-pointer.
I'm not saying these benchmarks or the workloads I've seen are representative of the "real world", but people keep repeating that frame pointers are basically free, which is just not the case.