Your tone comes off as very adversarial/angry. Intentional or not? The claim is about the likelihood of more efficient approaches existing, not about the subjective qualifier preceding it (misunderstanding?). Yes, NNs have existed for a while now, but they will also exist for a long time forward in the future (hence the perhaps poor choice of word: infancy).

"if you want to know whether neural nets are in their 'infancy' you shouldn't make 'observations'" -> I think you have understood this as "neural networks did not exist before" when what I meant is "neural networks will still change a lot in the future"

I was interested in discussing, specifically: will NNs continue scaling up in size in the near/long term. In answer to the statement "the training costs will exceed all the compute capacity in existence", I added "it is very likely that more efficient approaches exist.". You replied that NNs have existed for a long time, and that because of this, it is a complete fantasy that more efficient approaches exist. Do you feel like this is a fair assessment or no?