Can one do QFT in an ultrafinitistic foundations? My guess is no.

Also, I don’t think ZF sans the axiom of infinity works as an ultrafinitistic theory? It still has every natural number, just not the set of all of them.