Vicuna on iPhone | Hacker News Reader

Vicuna on iPhone

mlc.ai

90 pointstosh15 comments

This is pretty fun. It's pretty competent for a model that runs on a phone - I did get a great hallucination out of it when I asked it about myself: https://twitter.com/simonw/status/1652363265979318272

Encode 20.5 tok/s and decode 6.6 tok/s on my 14 Pro (non-Max)

Did pretty good on an okonomiyaki recipe

Edit: the coding capabilities are pretty hilarious. I asked about partials in Turbo streams and it correctly answered, but then when I asked about a code sample it gave me a PHP+MySQL query? Who knows what happened there.

My iPhone 12 carries 3.62 gb of RAM, won't run :(. Maybe I will spend time and fix the code. Big maybe very on the fence.

Hmm. I just get it to consistently crash on my 14 Pro Max, latest stable iOS build. At least TestFlight will send those reports on over.

Me too. I thought it was my crap ass iphone 11, but if you're also suffering on a 14 pro... maybe it's a bug, not a hw constraint.

Same thing on 12 mini. I think 6gb of ram is a little low for this.

13 Pro Max, no crashes but definitely went OOM and slowed to a crawl. My phone also got very warm.

I wasn't able to get much useful output for the things I normally use ChatGPT to help with. It insisted on giving general steps instead of code for basic tasks I threw at it. (Certificate generation using openssl or python).

Fantastic effort, I'm on holidays but I'm going to burn through a significant chunk of my mobile internet traffic allotment to test this... I seriously thought it'd take at least until summer for someone to make this happen (LLMs on mobile devices)

Edit: too bad, it crashes all the time without generating output.

Are there any Android alternatives?

Bummer it crashes on my iPhone 11pro.

It also crashes on my SE. This is most likely due to memory as it suggests.

Update : Also other users with beefier hardware comment about it.

see https://news.ycombinator.com/item?id=35763483 also

What the heck is it?

A specifically trained Llama

A large language model; machine learning artifact.

Reply on news.ycombinator.com