>>Not only could Congress pass a law recalling any FISC judge

I'm not sure this would be constitutional. Courts have rules that the term limits on FISA judges don't impact their standing as regular article III judges. Arbitrary removal almost certainly would.