I think you're overstating your case. "although the gorilla learned a large number of signs she never understood grammar or symbolic speech" - https://en.wikipedia.org/wiki/Great_ape_language#Koko
> humans that don't have language
Which humans are those? Language arises pretty spontaneously among humans even if they're deaf or blind - https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language
Yet no amount of exposure will make a dog or ape understand language.
> There is, literally, a part of the brain that if you shut it off, you lose language.
You mean the entire left hemisphere?