Encrypting data on a VPS, for example using LUKS encrypted volumes, does not provide any meaningful security if the host can take a memory dump of the running machine.
On a physical machine, you have to plug in a dumping device into a DMA-capable port, cool down the RAM and move it to another machine as fast as possible, or reset the machine and boot it from another medium (hoping the BIOS didn't override anything useful). In many jurisdictions you also have better protections in regards to required warrants and such for your own hardware.