Just tune the kernel watermarks - vm.min_free_kbytes and vm.watermark_scale_factor
One day I will probably see if I can still reproduce the original problem and be more methodical about it. More likely on list of things I might not ever get around to.