Hacker News Reader

In 5 years phones can easily have 1 TB flash

Parameter count is basically storage. Including persistent like flash memory. Which (still) is subject to Moore's law.

That's discounting architectural innovations & improvements which are bound to happen somehow.

Do we actually know enough about neural networks to be able to say how many parameters you need for a given performance? I’m under the impression that we’re still in the phase of trying things out and we can’t rule out different architectures massively reducing computational needs.

We can't rule out different architecures, but absent some architecure or hardware or quantisization breakthrough, frontier models will simply not be fitting on phones.