In the last few weeks, the initial prompt evaluation step got much faster with llama.cpp so everything that depends on it like the python library and lmstudio are faster as well. I’m very happy with my purchase at this point and it seems to keep getting better.
Where can I learn more?
I don't understand how one can ship a computer today with only 8G of mem. My phone has more than that. Are developers able to work with this amount of mem, or are they forking out $3k + for a Pro with 24G?
I went with a Framework because I got good hardware for way less money
I don't know how anyone can use a laptop with only 8GB of ram. Maybe if all they do on it is Facebook and Youtube ¯\_(ツ)_/¯.
Ok, I'll take the bait and disagree with your agreement.
Apple has a good thing going for now. However, their tightly coupled HW/OS and insular corporate culture limit their reach. LLM is a good example of this. They may have their own cooking in some secret room. Or not. What is the value=(reward/risk) of making something like an on-device LLM that Apple might do? Maybe Apple will acquire you or maybe you get Sherlocked.
As an example, Siri was an acquisition and neat trick, but it's barely improved since 2011 (I'm still a basically satisfied many x daily user). Although Apple has changed lots since 2011, it hasn't demonstrated an ability to execute something like meaningful integration and improvement of an LLM. Therefor I would not agree that "Apple is in a prime position to make strides in language model development on their hardware."