In fairness, the developers of CPython don't prioritize making it fast very much. Pypy has comparable performance to LuaJIT though. Do you happen to know how Pypy represents ints internally?
No idea, but packing an int in a single 32/64 bit value without indirections is how V8 JS does it as well, hence the speed. I'm going to guess that pypy does it too.