It's really easy to set up FreeSWITCH to do that. I prefer FreeSWITCH to Asterisk because the XML config is way easier to grok. FreeSWITCH peeps also made Signalwire and it's pretty nice.

One problem with this method is that your burner will still accept incoming calls from any number, so you'd lose caller ID information if you want to make sure only your VOIP server can call you. Of course you can just set it up to send you the caller ID through some other channel - like a text message.

But anyway you can make a really simple lua script that looks at a text file to see if the number is allowed, then forward it to your burner. In FreeSWITCH you just do something like <action application="set" data="allowed=${lua(allowlist.lua)}"/> and act on that variable accordingly. Then you can use whatever command line tool you prefer to add and remove numbers from the file.

If you need some help, drop a burner email and I can reach out.