Whilst there is nothing I particularly dislike about Spotify, I found it's 'Offline Mode' particularly clunky.
When I play a song on Apple Music that I have in a playlist or album, I can replay tracks without the need for a persistent 3G/4G/WiFi connection. Whilst you can do this in Spotify, I understand you have to manually click a button to enable offline playback.
Furthermore, Apple Music will shows me which tracks are cached for offline playback, since those which are not are greyed out and un-clickable, whereas I don't recall seeing anything like it in Spotify. This feature is particularly useful on the tube (London Underground for people in the USA/Europe/Rest of World) where you have intermittent WiFi at stations (in fact, by the time your phone has connected to the access point on the platform the train is already pulling out of the station - so you've got to be fast!).
However, I feel like Apple could have done better with the Apple Music experience on iPhone and iPad. For example, what is the 'For You' tab all about? Just some bouncing spheres with labels of different genres? I quite like the 'New' tab since I can quickly find popular music, but it feels too much like the iTunes Store, where you click-to-buy.
Yet, I really like the artwork they've set up for artists, and I think the player is excellent. I can't say I use 'Connect' that much, but it is an incredibly convenient way to follow artists you like and, if you're lucky, hear some B-sides that didn't make it onto the album.
I haven't yet decided whether I'm going to continue my trial with Apple Music or go back to Spotify. I just want a solid music streaming service that will 1. let me listen to the artists I love and 2. help me find other artists who I will love. I feel like neither of the two fully satisfy those requirements just yet.
Edit: I forgot to talk about Beats 1. I'm really enjoying listening to Zane Low on Beats 1 radio after his move from Radio 1.