Is this something new? I've seen streamers produce a https://en.wikipedia.org/wiki/Droste_effect with OBS self-capture many times. (Unless that only happens with screen capture?) It doesn't seem like this is something you'd want to do intentionally, anyway.

This is actually intentional, at least on Windows. At the time it was to fix a crash of some kind, many years ago.