How far can you get with JS/other interpreted things in e.g. optimizing for cache access etc? Sounds like you're at the mercy of the JIT compiler (which may go far, but still).
The aspect of the language you're using, if optimized, is virtually always optimized for the most common use-case. If your use-case isn't the most common use-case, you must account for this.