Eventually I went back to just eating regular food and within a month it was coming back along with not feeling as great.
I have still never been able to nail it down, and my dermatologist is pretty dismissive of it.
This stuff about the gut-skin axis has only been coming out the last 2-3 years, although there are a few lone researchers who have been thinking this for decades.
For myself, I experienced roughly 80% reduction in symptoms after switching to a strict vegetarian diet two years ago, and it's stayed that way.
Some people have had luck using this method to find issues that doctor visits weren't able to surface.