They released different sizes from "small" to "xxl", and at least "base" should be small enough to fine-tune virtually anywhere.
I spent several hours on this today. Currently stuck at "out of memory" error trying to fine-tune the base model.
Worked a bit with the "small" version of Flan T5 today. I was able to fine tune it, and the results were not complete garbage. About half were non sensical but the other half were passable.