Even on mobiles, GPUs have much more computing power. They have architecture designed for transforming vertices and computing lightning, very fast and highly parallel.

In best case, SVG will take input triangles and submit them to GPU. Very small overhead (slightly more GPU bandwidth because lightning, when it’s that simple, bandwidth is often more expensive than compute). But in worst case, SVGs will rasterize on CPU, very slow in comparison.

If you rasterize on the server e.g. send jpeg images to browser, it becomes complicated, and depends on many factors like scene complexity and server hardware. Rasterizing triangles is much cheaper on GPU, but JPEG compressor needs data in system RAM not in VRAM, need to download back, relatively slow. Also on many public clouds virtual GPUs are expensive.

I would expect huge difference for similar operations. Current-gen $300 GPU consumes 120W doing 4.6 TFlops. Current-gen $300 CPU consumes 65 W doing 16 flop/cycle * 3.6 GHz * 8 cores = 0.4 GFlops. GPU cores don't spend electricity minimizing latency e.g. predicting branches, and they work at much lower frequencies.

Also some parts of the pipeline are implemented in hardware (rasterizer, texture units, alpha blending, Z buffer), they don't cost flops and hardware implementation is likely even more efficient than shader cores. E.g. if you want a gradient, load/compute per-vertex colors and the rasterizer will interpolate in hardware.