Going above 32G doesn't kill performance per se, it just means that there is not point in using heap sizes between 32G and 48G due to the larger pointers. If you go above that, it's all good again.