I'm not sure how you are judging "activity". I suspect you are talking about chatrooms. In this sense, yes, IRC probably would look pretty active - it is built around the concept of channels, the majority of which tend to be public. XMPP is far more diverse, rooms are spread across multiple servers with no central way to list them. A project that aims to spider the network and help find rooms is at: https://search.jabber.network/ (note that this doesn't currently index jabber.org, traditionally one of the larger hosts, due to a technical glitch I believe).

My mention of thousands of servers is not related to public chatrooms however. There are a bunch of public servers which you can register accounts on, but many more private ones (that federate) run by businesses and individuals and other organisations. A quick shodan search reports 55,760 unique IP addresses speaking XMPP publicly on port 5269 (the default federation port): https://www.shodan.io/search?query=port%3A5269+http%3A%2F%2F...

XMPP is different from Matrix.

On Matrix practically everyone is matrix.org. The other public servers I tried are relatively slow and suffer hickups when joining rooms.

On XMPP, many people run servers in their companies, projects, or homes. Which are the "top 3-4 XMPP servers"? I probably do not even know them.

But you are right: XMPP is used mainly for IM, i.e. one-to-one dialogues or smaller, non-public chatrooms. I'm not aware of any huge, public chatrooms in XMPP, and for that I'm on IRC — via a gateway from XMPP called biboumi.