I don't think I'd inherently read all the conflicting reports from people as obviously mis-attributed noise. I don't think there's strong enough reason to expect everybody to response similarly to the same diets.

Not only does everybody have unique genetics, environments, and histories, but how sure are we that all the people saying they do keto, for example, are doing the same thing?

It's a giant complex mess, even if you assume humans are prefect and infallible observers and attributers of cause to effect. Which, as you pointed out, they absolutely are not. At all.

So not only do you not have the expectation that individuals will respond similarly, the difficulties in ascertaining what any individual is actually doing, you also have to deal with misattributed experiences, and the ability for our brain to play tricks on us.

And that's even assuming everybody is acting in good faith. Which they almost certainly are not, considering money can be involved.

I've yet to come up with any decent way of approaching these topics. There are so many confounding factors I don't even know where to begin.

Turning yourself into an endless experiment and figuring out ways to blind yourself and measure results (which is yet another rabbit hole...) is honestly starting to look like the most viable way to get to some kind of 'truth' for this stuff, at least for you, now. Though even at this extreme, time is limited.

Otherwise... I guess you just do what you feel like and do something different if you feel bad. If you feel good doing whatever you're doing, maybe that just has to be good enough. Because I don't know how to actually get any deeper than that in any reasonable way.