Sure, not saying there aren't any benefits but as a casual user they don't feel big enough to justify a switch. I may be wrong though (hence why I'm asking).
It took less than 20 minutes for me to change over a decently large project. The builds on the CI server increased by about 25 seconds by switching from npm install to yarn install. It was definitely worth it for me.
Wait, your build times increased on the CI server? What makes that worth it?
Depends on how casual :P But probably not.
(Edit: Note: I mean you don't _have_ to. You might like the performance increase, but if you only use it casually, even that might not be worth the trouble.)