For my recommender/object sorter I have not been in a hurry to upgrade because I have other things to think about. This table should give you some idea of the time-space-accuracy trade offs

https://huggingface.co/spaces/mteb/leaderboard

In a lot of cases you will see two models with a huge difference in size but a tiny difference in accuracy. I could fit either the big or small Stella on my 4080.