This would be consistent with Omicron having certain immune-escape features. People who previously had strong, neutralizing immunity (from vaccination or prior infection) may experience reinfection/breakthrough infection, but because of T-cell immunity or residual antibody immunity the variant is less likely to cause severe disease than it would in a non-immune person.
If the above hypothesis is true, that doesn't necessarily make Omicron "milder" than Delta. Omicron would be infecting people who would not be infected by Delta, so in those people (some disease) is more severe than (no disease). I have not yet seen evidence one way or the other regarding severity for a first infection in a non-immune person.
That is to say, "Omicron is milder!" could be an artifact of Simpson's Paradox.