Abusing vector search for texts, maps, and chess

Why the need for new technologies then? Databases are well studied. Vector search is relatively easy to implement. Sure, there are some new insights to be gained by respecting a hybrid approach - but they are clearly overvalued.

Machine learning is supposed to make things easier. If you implement vector search across your company's data, there's no reason a LLM couldn't simply do the various SQL-style operations on chunks of that data retrieved via KNN. I'm not aware of this approach being used in practice - but I still think the obvious direction we are heading towards is to be able to talk to computers in plain english, not SQL or some other relational algebra framework.

Here's an architecture diagram for a production-ready vector database https://milvus.io/docs/architecture_overview.md. Not exactly something you can build in a month.

It's much easier to start from a database and add vector search as one of the features, then to go backwards. We have spent 7.5 years on the DBMS part, while the vector search can literally be added in a week...

And that's why every major modern database is now integrating such solutions :)

That being said the basic idea is really interesting. I'd love to see a fully opensource competitor to chessbase. For people who don't know, chessbase is a subscription service which gives you a windows-only chess analysis platform and game database. It allows you to do advanced searches (eg I want to find games by 2500+ rated players in the Caro-Kann where black has a pawn on c5 and a bishop on e7 or whatever) which advanced players use to do "prep" (deep positional analysis typically of opening positions) which they save in reams of files prior to memorization. It's probably not an exaggeration to say nearly all strong players and serious improvers subscribe to it.

Chessbase made themselves persona non grata in the opensource world by apparently ripping off parts of stockfish and selling it under the names "Fat Fritz" and "Houdini"[1] and even were that not the case it would be great to have mac and linux opensource options.

[1] https://stockfishchess.org/blog/2022/public-court-hearing-so...

A better criticism might be: this metric defines position as close even if can only be reached by reversing moves. There's some discussion in the HNSW paper https://arxiv.org/pdf/1603.09320.pdf of working with non-symmetric metrics but I haven't read further.

> Alternatively, you can design a custom scheme to weigh pieces differently, assuming pawns’ positions affect the game less than those of queens.

...no? A pawn being blocked vs. passed can completely change who's winning, and if the queen is hanging it doesn't really matter where. The chess section is strange.

It's interesting to think of positions that can be reached in fer moves being close to each other in search space, but that seems like it would just become standard BFS.

    8 ♜ ♞ ♝ ♛ ♚ ♝ ♞ ♜
    7 ♟ ♟ ♟ ♟ ♟ ♟ ♟ ♟
    6 
    5 
    4 
    3 
    2 ♙ ♙ ♙ ♙ ♙ ♙ ♙ ♙
    1 ♖ ♘ ♗ ♕ ♔ ♗ ♘ ♖