Here's an old article from the time of the Higgs Boson discovery that had a great explanation:
https://blogs.scientificamerican.com/observations/five-sigma...
> Chances are, you heard this month about the discovery of a tiny fundamental physics particle that may be the long-sought Higgs boson. The phrase five-sigma was tossed about by scientists to describe the strength of the discovery. So, what does five-sigma mean?
> In short, five-sigma corresponds to a p-value, or probability, of 3x10-7, or about 1 in 3.5 million. This is not the probability that the Higgs boson does or doesn't exist; rather, it is the probability that if the particle does not exist, the data that CERN scientists collected in Geneva, Switzerland, would be at least as extreme as what they observed. "The reason that it's so annoying is that people want to hear declarative statements, like 'The probability that there's a Higgs is 99.9 percent,' but the real statement has an 'if' in there. There's a conditional. There's no way to remove the conditional," says Kyle Cranmer, a physicist at New York University and member of the ATLAS team, one of the two groups that announced the new particle results in Geneva on July 4.