Hacker News Reader

Comment by hmottestad | Hacker News Reader

With iOS 17 they added a tiny little LLM to the predictive typing. I have the newest and greatest iPhone but I feel that I very rarely see it in action. I must assume that it’s just too slow at to keep up with my typing at the moment. Or it’s just not large enough to give very many useful suggestions.

View on HN

Really? Typing in my iPhone 12 Pro has become a nightmare. I suspect it is because of predictive typing ML shit. It happens all the freaking time now. The symptom is that my whole device just froze for a few seconds while the next word is squeezed out. How do I turn it off?

Compared to Android, the iOS keyboard has always been a nightmare. However, wI feel it has been causing me fewer issues within the past month-ish. Has it been updated recently?

Thankfully, Gboard is available on iOS.

Is tiny LLM an oxymoron? I believe Apple has told us it’s a transformer language model, but not specifically a LLM.

Yeah, they meant a TLM

That's print('Hello, world!')

It's like how the "New Forest" is really old now: even small LLMs are (from what I've seen which isn't exhaustive) large compared to Markov language models.

There's no difference. An LLM is just a transformer language model that's "large".

According to this article[1] it has about 34 million parameters.

https://jackcook.com/2023/09/08/predictive-text.html

It’s probably why autocomplete got drastically worse, to the point I’m considering turning it off entirely.

Most “AI” features are so incredibly fragile they’re not worth deploying.

Autocomplete got worse because the new system in iOS17 didn’t retain training data from prior versions. It reset everyone back to untrained. I’ve been manually correcting specialized language I use daily (e.g. “iRacing”) on my 12 (non-pro) since iOS17 release, and now it gets it correct 99.5% of the time.

So, rather than turning it off, manually correct the incorrect completions and use the suggested words bar frequently and it will learn how you type. It’s just having to start over after tossing out several OSes worth of training that makes it feel worse.

It seems to have reset, but I find it’s actually much better than before after some intervention/training when I first updated.

It's a GPT2 model. It hasn't changed the autocomplete experience that much (occasionally I'll see a word completion).

I have noticed Siri now describes pictures sent to Messages.

Reply on news.ycombinator.com