So does it make sense to try and run models on older 4gb GPUs? I assume the energy efficiency isn't worth it?
It makes more practical sense today to just go into 4gb GPUs and many people are working on this already.. on my side of things, it's more on the exploration side for now, until we see what good models get released that can use this capability.
There are many people on github and x running decent models especally MoE models on older GPUs and it's doing nice, I'm personally tinkering with this at home.