I'm not saying there's no benefit to using local models. My point is still the same as before: you have to be willing to sacrifice both performance and quality even when just comparing to free models that are available today.
I'm not saying there's no benefit to using local models. My point is still the same as before: you have to be willing to sacrifice both performance and quality even when just comparing to free models that are available today.
I spent some years in the pharma industry, where AI came really late, as there was (justified) concern that sensitive data might leak - even by accident. Eventually a solution was implemented - it was some kind of open model (they didn't say which) running on-premises.
You couldn't tell these people that this and that powerful model is free or inexpensive, because it's useless to them if it's running on someone else's computer i.e. the cloud.
Either way, my initial reply was responding to the idea of these 15 year old GPUs as potential "capable processing units" for LLMS. My argument is that they are not particularly capable at the current moment in time.