Probably should have used higher-level functions. Maybe next time :). I don't think I can change the protocol and break compatibility at this time lol.

The crypto_kx functions seem to generate 2 symmetric keys for 2 directions. That creates complications in the code and that's why I didn't use it initially..