Even the better low-end N100's usually idle at 5-6W, which is double the Pi 5.
SoC idle temperature depends on environment and cooling solution; the N100 would quickly thermal throttle if you don't have a comparatively large heatsink attached. The Pi 5 will actually give you full performance for a minute or two before throttling (assuming no heatsink).
Other Arm chips are much better, efficiency-wise, but the Pi 5 is still more efficient than any low-end x86 build, especially used.