The linked paper has some tables that explain why this matters: How not to Lie with Statistics: The Correct Way to Summarize Benchmark Results https://dl.acm.org/doi/pdf/10.1145/5666.5673