A physical box with some hacks, potentially - though even then there are challenges.
https://www.havenco.com is planning to get into that later this year in well-connected collos in the US and EU, with cameras and pluggable USB displays to allow users to verify machine identity.
The target is $50/mo per machine and potentially lower (though that may require some kickstarter magic to do a Lima-like low-end hardware with serial console). For super low end, mount local LAFS or encfs+NFS or encfs+fuse+S3 storage, depending on the use case.
On top of that though you'd have to set paranoia vs convenience tradeoffs - do you trust a KVM? Want to drive to the machine and do serial console yourself? FOSS KVM or cheap serial console box?