You don't just learn a language by osmosis. You need resources, and people to work with you.
It's proven to be harder as we age, but it absolutely does work like that. I would assume Toma didn't more because of little interaction, not hearing dialogue around him, etc.
(And also just because it typically works like that doesn't mean he couldn't have been unusually predisposed against it, or genuinely ill, or too traumatised/uninterested/deliberately disengaging.)