Sure, but your gonna be able to max out the ethernet port on an OrangePi Zero, whereas none of the Raspberry Pis can even fill half of a 100Mbps ethernet port. Printing a case isn't all that hard, or you could always build it from thin plywood, cardbord or something else if you want a case.
> It doesn't support POE out of the box
No, but you can wire it for any type of POE you want, whether that be 5v, 12v to 24v, or proper 48v POE. The traces are already there on the PCB, and at that price point I doubt anyone could fit in a buck converter to make it do 48v POE without increasing the cost by a few bucks.
> WiFi on the Orange Pi Zero is a piece of shit.
That and every other SBC, I've got a Raspberry Pi 3 on the shelf that I wrote off due to poor wifi performance. SBCs with small integrated antennas just aren't gonna have amazing wifi. Most of what was in the BSP is now mainlined in kernel 4.10 by the way.
Armbian is decent, I've been quemu-debootstrapping my own images though since I'm not a fan of all the modifications they make to Debian.