They did, jut not mainline. People forget these are embedded chips - they are intended to go inside of something and do one thing. These chips lack auto hardware discovery because the manufacturer assumes the customer will only turn on the hardware peripherals they need for their specific use case and build a static kernel image to meet that requirement. They ship a product that will likely see few, if any software updates and end up in a landfill.
It's because of the Raspberry Pi foundation we have this perception that embedded Arm chips are like general purpose desktop computers that run Linux desktops. The original Pi SoC was designed for TV set top boxes, STB's hence the loopy booting from GPU which was likely part of some obfuscated secure boot chain to thwart STB hackers. The Pi was a throw away hobby toy based on a chip Broadcom was going to scrap so they got a dumpster deal. It took a lot of effort for the community to fully reverse the Broadcom SoC and bring all the Pi hardware to mainline.