Now. You might read this as me saying that Bayesian statistics is all made up, and that's not what I'm saying. I'm saying that if you go Bayesian all the way, and your result is even remotely controversial, someone could easily challenge you by saying "this result would have been different with different priors, and why won't they just come out and say p < 0.01? What are they hiding?"
When you do frequentist statistics, the equivalent of priors were, instead, made up for you by long scientific tradition. They're not very good priors, as the relevant xkcd illustrates. But at least it's not you making them up.
In a more complex model, there's also the thing where you can't calculate the result exactly, you have to approximate it with Markov Chain Monte Carlo or something, leading to another way to doubt your results (did the MCMC converge correctly?)
So instead you do frequentist statistics. You use your favorite stats package and it spits out a nice comforting p-value that will satisfy the reviewers. You have tons of guidance about how to do things. It's not a great thing, but it is a definite advantage of using frequentist statistics.