if you are just using it for inference, i think an appropriate comparison would just be like a together.ai endpoint or something - which allows you to scale up pretty immediately and likely is more economical as well.
i do think that cloud GPUs can cover most of this experimentation/learning need.
This is obviously because their are forced to use high memory cards.
Are there ideal cards for low memory (1-2BN) models? So higher flops/$ on crippled memory