I can see that. Without knowledge of cellular level biology, sponges appear to be plants or fungi.
The roots of "animal", "fungus", and "plant" are completely arbitrary; anything outside the chosen root can be called a "protist" (many marine "plants" are counted as protists nowadays).