Exactly. If you renamed flu to FLU-21, did extensive testing for the flu (doesn't really matter how reliable the tests are), you'd have a pandemic every year.
The definition for a flu epidemic is two standard deviations above the expected cases for influenza that year, as estimated by a Serfling-type regression model using a negative binomial distribution.
Influenza just doesn't breach that barrier for more than a few weeks most years, and rarely in widespread geographic areas at once.
Compare that to SARS-CoV-2, where the expected number of cases is zero.
"Epidemic" means something more than "Lots".
You can have very serious endemic diseases - malaria is one example. But "Occurring to a level above expected" is an important piece of information all its own - for example, five cases of Smallpox, while only five cases, is something of an emergency, whereas five cases of influenza is not a serious crisis.
That's all based on the people who would use the technical definition.
Source: https://www.nytimes.com/interactive/2021/04/22/science/flu-s...
My wife is also an ER doctor and agrees that there were times that they didn't perform many/any flu tests--mostly in the spring of 2020 and after none of the flu tests were returning positive for a while. But this past winter people have definitely been tested for the flu. Rapid influenza tests are also usually performed first, only sending to labs when confirmatory testing is desired.