I think you're underselling the small cores a bit. Their microarchitecture is roughly comparable to a Cortex-A75, which was the high-performance cores in Snapdragon 845 phones like the Samsung Galaxy S9. But Apple clocks them about 26% slower than the SD845 did, so maybe we should be comparing against Galaxy S8 instead: flagship Android performance from 3 years ago. That's capable of a lot more than just casual background processing. Samsung introduced their DeX docking station for smartphone-powered desktop usage back with the S8 generation, when their phones only had as much computational power as the Apple M1's smaller cores.