The sad part is that on ARM the kernel is usually also custom compiled for the board. So what happens is that Armbian ship a different image for each board.
If you go and look in https://github.com/torvalds/linux/tree/master/arch/arm you see a zillion "mach-xxx" directories for different SoC architectures, even if they all use Arm.
Device-tree is a partial solution, but no-one seems to have an incentive to finish the job and let a single image run on any (sufficiently recent) arm board. It's difficult for the community to fix because most people have only their own board. Someone would need to pay for a CI rig with every board, and some kernel devs to do the work of building a single kernel to run across everything. (I think that's originally what Linaro was for - not sure why they didn't finish the job)