> I don't even know what the serial port was actually meant for.

The rarely-used Playstation Link cable, ostensibly:

https://en.wikipedia.org/wiki/PlayStation_Link_Cable

Never used it myself, but it allowed a really weird first-person multiplayer mode in Bushido Blade:

https://www.youtube.com/watch?v=-obFMn_u3U4

Unpatched v1 models can be softmodded no hardware modding (picofly) needed. For those you only need to enter the recovery mode which can be done with a paperclip > sorting the pins on the right joycon rail. Though it's also not recommended because you can fry the whole Switch with the paperclip. Nowadays people use 3D printed cheap RCM jigs.

First models of the switch can be jailbroken by connecting pins accessible without disassembly (used to connect joycons), lookup switch jig RCM.

You short pins on the right joycon connection at boot time to enter recovery control mode in the boot firmware which (via an unpatchable exploit) gives you full access to the hardware IIRC.

(I presume the picofly & similar devices are just doing this automatically so you can boot the hardware under user control without requiring other hardware to make the exploit work?)

The paperclip RCM was an oversight by Nintendo on the early switch models. It is not possible to do on later firmware's, I would HIGHLY recommend not trying to install the Pico fly yourself, it is much harder than you might think and the first time I tried I screwed up the capacitors on the APU and bricked the console.