Generally I try to avoid VPSs. They are long lived, it's easy to accumulate undocumented state, and have substantial memory, disk I/O, and network I/O overhead. You may end up with a years old VPS with many changes over the years, and ugly things accumulate. Maybe you built a package from source, but now it's got a security vulnerability, but your forgot it's there. So apt upgrade doesn't help. Not to mention any security fix assumes you aren't already compromised. A rebuilt container (I suggest proxmox or docker) rebuilt will remove any previous existing compromise. Sure you can clone an old VPS... but you still don't know exactly what changes have been made, and if it was compromised the clone is as well.

Much better to have docker (or similar) build script, a ansible play, or a puppet manifest. They will clearly list needed packages, config files, services, at least in puppet's case you'll have the "trifecta". The trifecta is a mapping between a service, the required packages, and the config files. So Puppet can notice a config file change, restart a service, and if a package is missing install it. So puppet is to a degree self healing, even when an admin edits a file on the guest/container/VPS instead of in puppet and it's fixed on the next puppet run.

That way years later it's not big deal to upgrade the OS, or replace one piece of the stack with another, then apply ALL the changes you've done so far, and test the result.

I can tell you from experience it's a nightmare to inherit somebody else's VPS that has 3 python installs, tons of services, and requires manual intervention on boot to start/stop services. Would be MUCH better if it was just managed by config management and everything except the base OS is controlled by ansible/puppet.

Often custom VPSs are called pets or snowflakes (with a negative connotation), much better to make them easy to replace as needed, like cows (with a positive connotation) as the saying goes.