If the rest of your stuff is also Apple, browser sync is a pretty solid Just Works thing, which can be a benefit of using it, but you're not all-Apple, so that benefit's out.
[EDIT] "So what are the downsides?" worse dev tools, worse add-on ecosystem. Sometimes it's missing engine features that FF and Chrome have (this never hinders me at all in practice, with my usage patterns, and in fact I'm usually glad when they choose not to uncritically and promptly implement every single feature Google adds to Chrome, but I understand it bothers some people a lot, so would definitely count against Safari if those features are things you really want/need)