I remember a CS prof saying he moved from hardware to software because the speedups possible in algorithm design dwarf anything you can achieve by improving the bare metal.
Case in point: none of the algorithmic improvements to matrix-multiplication has had any real effect on deep-learning etc (except Winograd convolution).
The advantage of speeding up hardware is that it scales to most present and future programs. Though it's a bit of an economic activity. As the hardware gets faster, software developers de-prioritize algorithmic speed in favour of faster product development or research.
Is implementing an algorithm in hardware ever done before all of the obvious software optimizations have been exhausted? If not, how would you know you're implementing the best known version of the algorithm?
It seems to me that hardware is a route to improvement, but only after all of the software improvements have been done. If you want to spand your career making computers do stuff faster there'll always be more work to do in software, simply because if you can make something fast enough there you don't need to implement it in hardware.
Then again, there is this crypto-mining boom, where custom ASICs and GPUs literally pay for themselves. There, the associated algorithms were specified to be slow, and no improvements on them can be reasonably expected unless quantum computing takes off for real and opens new venues.
That said, when you implement specialized algorithms in hardware, it's likely because it's good enough for the task it's designed for.