The spirit of this is correct, but a better approach to this is going for a walk with just your thoughts.
Yes, that means no phone, no headphones, just you and your brain enjoying a walk. Let your mind wonder and be free.
1,018 karma · joined February 9, 2016
Yes, that means no phone, no headphones, just you and your brain enjoying a walk. Let your mind wonder and be free.
grug very elated find big brain developer Bob Nystrom redeem the big brain tribe and write excellent book on recursive descent: Crafting Interpreters
book available online free, but grug highly recommend all interested grugs purchase book on general principle, provide much big brain advice and grug love book very much except visitor pattern (trap!)
Grug says bad.
In all seriousness, the rough argument is that it's a "big brain" way of thinking. It sounds great on paper, but is often times not the easiest machinery to have to manage when there are simpler options (e.g. just add a method).
It's not bad if you need something quick. I haven't had a large need of ANN in duckdb since it's doing more analytical/exploratory needs, but it's definitely there if you need it.
Instantiate all of your metadata once, and then send that logger down, so that anybody who uses that logger is guaranteed to have the right metadata... the time to add logging is not when you are debugging.
But ANN search is still a sledgehammer and building out hybrid solutions that help bridge the gap between this and traditional data stores still have room for innovation.
* filterable ANN, decomposes into prefiltering or postfiltering.
* dynamic updates and versioning is still very difficult
* slow building of graph indexes
* adding other signals into the search, such as query time boosting for recent docs.
I don’t disagree these systems can work but innovation is still necessary. We are not in a “data stores are solved” world.
Especially in the context of embedding search, which this article is also trying to do. We need database that can efficiently store/query high-dimensional embeddings, and handle the nuance of real-world applications as well such as filtered-ANN. There is a ton of innovation in this space and it's crucial to powering the next generation architectures of just about every company out there. At this point, data-stores are becoming a bottleneck for serving embedding search and I cannot understate that advancements in this are extremely important for enabling these solutions. This is why there is an explosion of vector-databases right now.
This article is a great example of where the actual data-providers are not providing the solutions companies need right now, and there is so much room for improvement in this space.
Can’t speak for the code since I haven’t peaked into it
A fascinating memoir by a philosopher turned brain surgeon, facing a terminal cancer diagnosis. A person who spent their entire life pondering the morality of life being faced with their own ultimatum.
I reread it once a year, at minimum. A deeply moving book.
Still my favorite class and professor from my time at school.
At one point you'll have to reckon with IO costs and storage once that is over and then you have no choice but to exploit data locality more heavily.
Just saying, YMMV depending on how much data is in your database.
Most homeowners (product managers) want the cheapest (fastest) thing that fixes (delivers) a working home (feature) to them. When given the options those are generally what people choose.
Sure some contractors (software engineers) always cut corners, but I believe many take pride in their work and given the option would prefer a solution they take pride in.
Life’s too short to let the few negative people out there bring you down. 1% or 5% whatever lets you realize that and move on
This is my favorite way of writing software as well. My current gig has a ton of microservices, and when a feature comes up that requires changing one of them, I much prefer to make a couple other, smaller changes that help keep the service operational and easier to maintain with it.
One issue is that this often times bring out the yak-shaving, but I think it's a fair tradeoff and helps reduce the time burden of doing large migrations.
However, with Postgres its pretty trivial to get a JSON format of the output if you needed a machine parseable answer. I don't know of a library that does this for you, but it wouldn't surprise me if one existed.
explain (format json, analyze, buffers) select 1Use One Big Database.
Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care).
It is so much easier to do these joins efficiently in a single database than fanning out RPC calls to multiple different databases, not to mention dealing with inconsistencies, lack of atomicity, etc. etc. Spin up a specific reader of that database if there needs to be OLAP queries, or use a message bus. But keep your OLTP data within one database for as long as possible.
You can break apart a stateless microservice, but there are few things as stagnant in the world of software than data. It will keep you nimble for new product features. The boxes that they offer on cloud vendors today for managed databases are giant!
* Grafana 8 alerts removed the Image Preview, which was extremely useful during issues. [0]
* Grafana 8 alerts don't have any way of being stored as code. In fact the API that they provide in their docs [0][1] doesn't work, or isn't up to date.
* The expression languages have zero documentation about them, so aren't exactly useful for things that might get a developer out of bed in the middle of the night.
[0] https://github.com/grafana/grafana/discussions/38030#discuss...
[1] https://editor.swagger.io/?url=https://raw.githubusercontent...
[2] https://community.grafana.com/t/posting-an-alert-using-grafa...