A better interpretation might be "probability this would happen if results were governed purely by chance". Note that the distinction is important if the process is in fact governed by chance!

Lets say I roll a dice two times and get a six both times. The probability of this happening is 1/36 or about 3%.

Would you say I have established with a 95% confidence interval that the particular die I'm using always rolls six? No, because you have good reason to believe that the the results are in fact random. Or in other words, you have a strong prior belief that the hypothesis you're testing is false.