The primary factor with using a Desktop Linux is GPU support.
The Videocore IV in the Raspberry Pi is as close as to open source as you're going to get on an ARM board. Plus it's been around since 2013 according to Wikipedia: https://en.wikipedia.org/wiki/Raspberry_Pi
Compared to this to the Mali-T720 MP2 in the Orange Pi 3, GPU support in Linux is virtually non-existent. https://linux-sunxi.org/Mali
This article is a little outdated but is still very relevant: https://forum.armbian.com/topic/7037-multimedia-related-stuf...
Also, note that GPU support on Android is usually a lot better compared to Linux. This is because they are using an older, more mature kernel.