At >400 operations per second, that doesn't explain most of the 2/second (500ms) operations the discoverer apparently observed. Were they running on an old Pi and you on a currently high-end CPU or so? Basically, what hardware is this benchmark on?
However yes, even on this, not so powerful CPU, it doesnt take 500ms so I dont think it explains it.
Curious that asymmetric crypto isn't even the slow part here. Feels like they just messed up somewhere (but I don't have the low-level asm/C skills to check that without considerable time investment)