Highly recommend reading the various interviews and papers by Bruno Putzeyes. He’s been pushing the state of the art in PDM / PWM / Class-D amplification for the last 20 years or so.
Here’s a white paper of his specifically about direct PWM amplification.
https://www.researchgate.net/publication/242013327_All_ampli...