It looks like a decade long game of trail and error.
I suppose you start at the basics, getting an interpreter purely for CPU code. So PPC to x86 code. Then onto more complex stuff like floating point VMX to x87. Once you have basic code processing, every time a line of code comes up that cannot be interpreted, you figure out what it does and then build the appropriate interpreter. Do this again and again and you would gain some basic function.
Basically just try running game code and keep plodding away like this until it works. Like giant bug hunt for 100,000 bugs. Very Tedious but necessary work. As for optimizing for performance, that is just black magic.
[0] https://github.com/abaire/nxdk_pgraph_tests
[1] though nxdk/freexdk/openxdk had to "compete" with the leaked XDK that a lot of homebrewers used so there was much less incentive to contribute to the open source one