Hacker News Reader

Never thought I'd run an LLM released in 2025, on my phone, in full BF16. With ~80tps on an iPhone 16 pro btw.

How do you actually run this on an iPhone?

With something like PocketPal AI (https://github.com/a-ghorbani/pocketpal-ai). I'd love hear HN'ers opinions on the "best" LM Studio-like option for iOS devices.

For iOS, OpenCat. Has iCloud sync, and one universal app for MacOS and iOS devices.

I use PocketPal. Can run any gguf model off hf.

you can also run it on Cactus - either in Cactus Chat from the App/Play Store or by using the Cactus framework to integrate it into your own app