The limiting factor for old consoles is largely RAM. The NES only had 2 KiB of RAM, making it pretty infeasible to even run anything CP/M-like. Its 6502 CPU also doesn’t have things like privilege levels/memory protection.

Moving up to the Sega Genesis with its 16-bit 68000 that many old UNIX systems were built around: it still only has 64 KiB. But, hey, that’s a lot better than 2 KiB, and it has memory protection. But the 68000 can’t do virtual memory (unassisted) and I don’t believe the Genesis has a timer interrupt, which seems like it’d complicate multitasking (I guess you could run the scheduler when the graphics system raises the vertical retrace interrupt?) but IMO it’s about the minimum level that could run something like what we tend to think of today when we talk about an operating system.