Hacker News Reader

Do I understand right this requires access to internals of the LLM and can not be used with todays models behind an API like ChatGPT or Claude?

Innovation that would be applicable to open weight models running locally only would be awesome.