As someone who uses the paid Azure Pipelines agents I can say I really like it generally but their hosted build agent performance is ... adequate ... at best.
I'm a product manager on Azure Pipelines. I'd like to learn more about your pipeline so we can improve the performance of our hosted build agents. It's something we're improving right now. Feel free to email me. jepling at microsoft dot com
I noticed Travis builds complete faster once they're running, by something like 20% in my case, but that's more than offset by having twice as many parallel builds by default. Never dug into why it's slower, though, at a guess something to do with the filesystem
I'd like to learn more about why Azure Pipelines was slower for your build. Feel free to email me. jepling at microsoft dot com.