Just like adding code to textual models helps the model develop its reasoning capabilities, it seems like adding more languages helps in other areas too. What is needed is more good quality data to train on...
These architectures are less capable than brains in many ways. So, we should expect them to have such trade-offs. An efficient one should work fine on English, mathematical notation, and a programming language. Maybe samples of others that illustrate unique concepts. I’m also curious how many languages or concepts you can add to a given architecture before its effectiveness starts dropping.
Some kind of diminishing returns asymptote from text volume alone must have been hit a long time ago.
To be clear, LLMs are not capable of reasoning.
Is it reasonable to show interest in something you call uninteresting?
Was Gödel a reasonable man, starving to death in fear of being poisoned?