Unless you're self hosting or using an alternate server Matrix.org will run into much of the same. Delivery can already be quire delayed occasionally on the main server, and it's barely popular in terms of big picture right now.