It also depends on exactly which version of OpenSSL. We ran similar tests with multiple versions of openssl. Openssl 3.0, which was the version in the official packages of our LTS distro was unacceptably slow. With 3.4, it was still worse than 1.1.1 but the regression was an acceptable level.

It also definitely depends on your hardware, and how you use it. The main source of the performance regression is more lock contention, so it is more noticeable if you have more threads running on more cores.