* Don't underestimate the number of services you can having running in containers on a Raspberry Pi 4B with 8GB of RAM, sipping just 2.1W (yes, that little!) at idle over PoE or as low as 1.35W over WiFi powered by USB-C. With distributions like DietPi you can have a mere 10 processes at startup (using just 44MiB of RAM), but still all the flexibility in the world.
* Also don't forget you can cluster them. Want Kubernetes experience? Perfect opportunity to have a cluster than pulls at most like 17W under extreme loads.
* Have an old laptop? Fantastic. Great second life for it. You get the bonus of an integrated keyboard, UPS and display if and when you need it, even if it's normally headless.
* Don't forget Mac Minis and 1L form factors. I have an 1L form factor (that weighs about 1.3kg) 8-core Zen 3 Ryzen 7 PRO APU that boosts to 4.6GHz, with 64GB of memory, a 1TB SSD and 2.5GbE that sips a mere 10W at idle, and pulls maybe 45W at full bore. Its silent. takes up as much space as like 25 paper napkins, and I can pass the GPU cores through to VMs.
* Don't forget hypervisors as well. For truly ephemeral stuff you can just spin up virtual machines on your main desktop or laptop -- just stuff more RAM in it.
-- ----- -----
What's great is you can put the power sipping stuff basically ANYWHERE. It can be easily accessible, but entirely hidden from view. It makes no noise. It's not spewing heat or demanding cooling. There's no blinking lights. You're not driving up the electricity bill. Yet you can do very real work on them.
For cost? Raspberry Pis aside (sorry, use rpilocator!) check eBay or other marketplace completed listings to establish a price floor, and just be patient and only grab quality stuff that approaches the price floor. Nothing better than getting new (but open box) or barely used modern hardware for peanuts (60-70% off) because the seller is impatient or ignorant.