You could argue that many aspects of human cognition are "hacks" too.
The brain is not a monolith.
I struggle to imagine how much further a purely text based system can be pushed - a system that basically knows that 1+1=2 not because it has built an internal model of arithmetic, but because it estimates that the sequence of `1+1=` is mostly followed by `2`.
https://transformer-circuits.pub/2025/attribution-graphs/bio...
Keep in mind that is a basic level of understanding of what is going on in quite a small model (Claude 3.5 Haiku). We don't know what is happening inside larger models.