As someone also dealing with "your search sucks", I felt all of this pain.
I think sticking with Postgres was a mistake, though. Denormalizing into something like ElasticSearch would probably have gotten you further (and it's so fast).
Searching the filters is pretty clever, though.