When you are reporting, say, a p-value from your ANOVA F-test, you are making formal mathematical assumptions, such as normal marginal distribution of your dependent variable. A lot of Frequentist methods (hypothesis testing) are really just mathematical shortcuts from times when computation was more expensive. The problem is, many people tend to misuse the tests where they are not appropriate either because those give "better" answers, or simply out of ignorance.