I suspect the same will/is happening with AI. Either you will pay for it, or it will be so ad infested that it will become useless.
I have only 48gb of ram, so can fit only 80k context max, so good compaction is must.
I initially used the Claude Code harness on 3.6 A3B, but found that tooling would break as Claude released new versions and things would go weird. I've since written my own harness which has basic operations: read, find, bash (which can write files, python etc...) & web_fetch, all within a mac container. Works amazing. You don't need anything complicated to go very far.
Low hanging fruit would be Pi or OpenCode. If you really want a much better understanding of what your hardware is capable of then give writing your own a go.
Additional tip: Low Power mode reduces some token speed, but stops the laptop over heating and the fans going crazy.