At this point it seems like the biggest limiting factor of scale is the vias, not the LEDs themselves. If they made LEDs with 3 pads, where the outer 2 pads are connected together, you could make a matrix on a single-sided board with no vias.
How about OLED microdisplays? You could have full HD and thousands of nits of brightness.
I wonder how it would look to connect the columns with 0201 zero-ohm resistors between the LEDs (jumping over the rows lines). If you want a square pixel pitch it might work out nicely.
You don't really have the clearance for that. The gap between the two pads of the resistor should be at most 0.3mm, and more realistically 0.2mm - any larger and they're going to tombstone. 0.1mm traces with 0.1mm clearance are manufacturable so it could fit in a 0.3mm gap, but realistically you probably want to do 0.15mm traces & clearance to keep the cost acceptable.
For one-off, I wonder if you could just glue the leds together and hand-wire (very carefully) with some magnet-wire. Throw in some kapton tape to insulate everything.