Even then, while there are some amazing FPGA-based synths available, companies like Korg just put their code on a raspberry pi and call it a day. The same is true for emulators (SNES Mini etc. are also just raspberry pis under the hood iirc)
You get an FPGA for timing. They're less capable, but (in many common design architectures), they output their results once per clock, every clock, on time, every time. If you can hit a fabric clock of say 100MHz, clocking all the weird logic you can stuff in there, it gives 100 million outputs per second, never skipping a single one for any reason (short of total failure). The penalty is that making a small change to your desired "program" can be very expensive, and many things won't be realistically possible at all. Or at least won't fit into a part that you can buy. But things like audio, video, and high-frequency trading love being able to guarantee timing.
(Of course there are other ways to write your FPGA HDL, but that's one of the more common ones. And you do see DDR-style clocking, and similar, every now and then.)
It depends. For some things, CPUs don't even come close. An XCVU13P FPGA can handle 1.2Tbps of full-duplex Ethernet @ 1 billion pps. And that part costs less than a grand at moderate qty, and with significantly less power consumption than a CPU that'd be capable of operating a dataplane at these speeds.
The point I was trying to make is that the CPU is a general-purpose creature and doesn't really care what you want it to do. If you had a CPU that could handle 1.2Tbps of Ethernet packets at 1Gpps, it could do a whole lot of other things involving 1.2Tbps of data flow too, very easily, if someone wrote the software. And more. (But you're probably not getting 2.4Tbps out of it, no matter what you do.)
An FPGA can not. There's plenty of things that those XCVU13Ps just can't do, or would do worse than a $1 microcontroller. (Setting aside for a moment implementing a CPU inside the FPGA... which does actually happen in just about every large-enough FPGA design, which is its own discussion....)
To give a concrete example, suppose you are doing some image processing and you need to write out something like a BMP image file from the pixel data in memory. Suppose, also, that it's dynamic-ish and you can't really precompute the header and just copy it around. This is a horrifying nightmare to do in an FPGA, generally involving writing a custom state machine and populating different bits of the structure on each clock and so on and so on. (This is not an arbitrary example, sadly; I've had to do almost exactly this.) You can do it but it's just plain hard.
On a CPU you're done in five minutes, for a BMP file.
Now imagine you want to write out a JPEG or PNG. On a CPU this is hard enough that you'll probably grab a library... and then still be done in about five minutes. Maybe thirty if something's unusual or you're being extra careful. Maybe a day or two if you decide to write it yourself (don't do this).
On an FPGA? Holy hell you're in pain. You're either going to spend $$$ licensing something or you're going to be there for weeks.
That is why there are jobs that FPGAs don't do. They're not technically impossible, they're infeasibly impossible.
If it is your core business, yes, great, do them. But you'd never decide to stop making wasteful unoptimized BMP files on an FPGA "just because". It's a major chunk of work. But on a CPU, five minutes.
That would be better suited to FPAAs (field programmable analog arrays). FPGAs can usually only work with clocked digital signals.