Our assumptions were that the GIL was getting in the way with our excessive concurrency. gevent + PyPy is not really a safe thing yet.
This entire project was also stood up in a week, so it's not like this service is some monster service with a lot of work invested into it. It was small enough that instrumenting and figuring out where the problems may lie and figuring them out would have probably taken just as much time.