qdrant vs pgvector remains to be benchmarked for a range of relevant sizes, please drop any if you see them!
This is when the relational nature of postgres really shines - you can even do in-mem calculations for distance.
You could put different categories in different shards though, resulting in the same effect.
You can also specify exact=true to do the same on any size, but this can get rather slow when your collection grows.
Disclaimer: I'm part of Qdrant.