Hacker News Reader

I am new to Amdahl's law, but wouldn't a rearchitecture make it less relevant. For example if instead of growing an LLM that has more to do in parallel, seperate it into agents (maybe a bit like areas of the brain?). Is Amdahls law just a limit for the classic LLM architecture?

View on HN

I don't think it can ultimately be escaped but the cited Vegh et al exactly proposes that, the bioinspiration, as a means to surpass those limitations.

However, in this article I contend that those limitations have posed little adversity in the field given the success of the latest models. As a result, it may be a bit premature to be concerned about it.

Reply on news.ycombinator.com