So if you do not want to allow this, then you must use something other than the AGPL. (Or maybe you could do some funky patching of the AGPL's terms, but you'd have to be careful to only do so by reference, since its text is copyrighted by the FSF, who only permit distribution of verbatim copies.)
Also, you should probably avoid calling it "fully free & open source" as you did in your original comment, since you intend for it to be neither Free nor Open Source in the sense ordinarily meant by FOSS.