Having an intel GPU is nice enough I guess. I might get one just for that.
You may have easier time trying to run some alternative OS such as one of the BSDs, Haiku, ReactOS, or even Windows.
For example, I use my Intel-based SBC as headless server. The only thing I need is two SATA ports available (which the SBC I own has), and the condition that's it's a small and cheap machine.
For a use case like mine, it's perfect. I don't need to think about using a certain distribution and ultimately depending on a producer. I used to have ARM boards, so I know :-)
It's important not to underestimate the dependency on the distributions. Intel x86 last virtually forever¹, ARM boards have a very short life². It's fine to use ARM if somebody wants a mid-term throwaway toy, but if one plans to have a stable [server] machine, it's a problem³.
¹=in the sense - as much as realistically possible
²=in comparison; the ARM boards tend to get discontinued quicker.
³=finally, consider that not every ARM board is [as supported as] an RPi.
This is true though I believe it's mainly due to them being newer developments in a field (mass produced embedded boards) where everyone and their cat makes something new every month. It's a moving target that will never settle (luckily) but given enough time it will slow its pace to sustainable levels both for manufacturers and users/developers. Give it some time, the kid is young.
As an example, I'm still waiting for a cheap enough board around which I would build what I consider a vital brick in every home network: something with enough SATA ports to make a RAID enabled server, then would cut costs in all other corners. Say 4 to 5 SATA ports, no video, no audio, no WiFi, no multiple NICs (I'd use a separate board for firewalling), no more than a serial port for monitoring and few GPIOs for blinking some leds or drive fans, monitor UPSs etc. It would be the best possible platform to make cheap non trivial file servers.
My current configuration is 2+2 disks arranged as RAID1 (I don't trust anything else) kept in sleep mode when not in use for more than 2 hours plus a fifth disk used very often as temporary storage before things are being stored on the RAID. I use a cheap Atom Mini ITX board I purchased used on ebay plus a rack enclosure that weighs like a ton. Nas4Free is the OS, which is perfect for the task although RAM usage with XFS reaches the maximum available on that board (4GB), but for home use it's still ok. It works perfectly, but I would happily swap it for a lighter ARM system in a smaller box. Unfortunately it looks like there's no solution to be used to build a similar file server, except costly boards with a lot other peripherhals or much faster CPUs I wouldn't have any use for. So why nobody makes a board like that? I would guess they analyzed the market and concluded there'not enough demand. A board like that would mean an excellent home/SOHO file server at a third of the price, but apparently the small number of people who build their own file server doesn't justify any risks in that field.
The Celeron J4105 (which powers the ODROID H2), is technically an Atom, and it's massively faster than an RPi 4 (2-5x), but I don't know about the Z8300.
Based on Passmark (which is an arbitrary choice), Rock Pi X and RPi 4 may have comparable performance; if that was established, and I wanted a cheap SBC, I'd definitely go with the former.
https://openbenchmarking.org/result/1907128-HV-1907127HV99
it's hard to make an absolute conclusion; it seems that the Pi 4 is faster than the Z8300 in more cases, but not all.
I wouldn't categorize these results as "definitely faster" though.