Here's how it compares on standardized llama2-7b 4K context testing vs some Nvidia cards: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp...
Not on the same platform so hard to make any solid conclusions, but vs the 4090 on a perf/$ basis right now, it's not too bad - about 60% of the inference performance of a 4090 at 45% the price. Still, if ML is your main use case, I think it's hard to argue that a used 3090 wouldn't be the way to go.