It helps that they used NaCl. When cryptographers looked at it, they found an issue: People can impersonate your friends if they get their hands on your private key.
This is pretty bad, but so is having had your private key compromised in the first place. It should be fixed next time they do a flag day.
Other than that, and the rekeying issue (keys are only renewed when the client is closed. They should be on a period of time, to make forward secrecy really effective), nothing else bad was found.
Notice that, for a couple years now, effort has gone into polishing (toktok) toxcore and documenting things, rather than else (eg: adding fancy features). That's a good thing.
Tox might be understaffed and progressing slowly, but there's nothing fundamentally bad about it. They got a bunch of important things (really distributed, DHT, public keys as addresses, temporal keys for forward secrecy, always end to end encrypted) right. I'm not aware of any other project that got this much right, unfortunately.
If only if Tox got more attention, it'd gain developers, donations, and the possibility of getting a proper audit done.