That's because a lot of people refuse to use status codes properly, like just using 200 everywhere.
There is a difference between /api/entity/123 and /api/search with a payload of 123, though.
The request couldn't be processed due to semantic errors... perhaps such as not being mapped to a handler :->
I suppose from the right point of view that could also be likened to a reverse proxy not being able to send the request on (502 bad gateway), but sane people would probably find that even more confusing.
There were also attempts to use 204 no content for "I successfully confirmed that what you asked for doesn't exist", but I think I managed to shoot those down.
Back-and-forth conversion is a very poor idea. It works for what was “internet resources” initially (basically files and folders), but later people stretched that on application data models and that creates constant issues because people naturally can’t understand the mapping, cause there’s none. This is not a good idea. Talk to http hosts with http and talk to your client with a language you designed specifically for talking to it. 200 vs non-200 is http level and orthogonal to in-service statuses.
The HTTP server, when it detects "URI handler not found" condition, builds an 404 HTTP response and sends it as a normal payload through the underlying connection instead of turning it into an TLS error packet or an RST packet on TCP level (that's the TCP's standard response for "port handler process not found", after all) or something silly like that, and that is absolutely fine, because the application-level (HTTP) error messages should be transmitted by the transport level (TLS/TCP) just as normal messages would.
The same reasoning holds just the same when we consider the usage of HTTP as a transport-level protocol for some higher-level RPC exchange. Yes, HTTP has some assortment of error codes that superficially look like they can be reused to serve as the upper-layer errors as well but that's a red herring.