I think SQLite can be a good idea, actually.
When I host small side-projects, I have PostgreSQL instance running on the same server and using the ident auth strategy so I avoid managing and keeping passwords and access the DB via SSH.
If you are going to make a separate VM, then don't forget on SSL.
p.s. I wrote Deployment from Scratch (https://deploymentfromscratch.com) which has a full chapter on PostgreSQL and Redis. It also has a demo that sets up a standalone PostgreSQL cluster on a VM (including SELinux, SSL, attached storage).