3.5mm is an analogue signal that can only output stereo. Its quality will be limited by the device it comes out of.

SPDIF (both optical and coaxial) support sending multiple audio channels in their original encoding, to support surround sound. The receiving device needs to support decoding of the audio, which is why there is often an option to force the SPDIF output to use PCM stereo instead of surround sound.

If you have a cheap TV and a good hifi, you'll want to use SPDIF or HDMI so that your audio isn't ruined by poor quality of the audio chipset in the TV.

My assumption was just that it was something about how the EQ is mixed for that jack, because it is labelled specifically as headphones. The other replies about surround sound are true, as well, but I don't think should apply to my tv -> stereo soundbar set-up.

But the difference is definitely noticeably, even to my relatively forgiving ear.

The biggest difference is digital interconnects can carry extra data for surround sound.

(Also, unique for optical connections, it's easier to avoid ground loop hums.)