"so if you have a packet destined to fe80::4, how do you disambiguate it?"

Routing tables get you to the destination but I think the question is about which source address to use ie which network card/interface to use as source - after all, they are all in fe80::.

For a destination in fe80:: the OS will pick the one on the right interface (in effect the IPv6 version of ARP).

You never use fe80:: as a source for a network beyond fe80:: because it and they are link local addresses. You'll send to the default gateway/GoLR/etc unless you have more explicit routes and set your source address as your IPv6 "identity" which might be one of many.

Anyway, here's your problem:

"But if you try to parse this as a URL in Go, you get an error:"

Go needs fixing!