tacoooooooo

This is a wildly out of touch thing to say

correct afaik :(

https://github.com/timescale/pgvectorscale/issues/113

the main issue with pgvectorscale is that it's not available in RDS :(

This probably doesn't count as an "app" in terms of what you're looking for, but was a fun little project

https://alexjacobs08.github.io/lobsters-graph/

(i built this in search of a lobste.rs invite if anyone willing and able sees this--email in my bio :)

This looks awesome. ai-sdk is an excellent library. excited to see it proliferating

Standard benchmarks (like BEIR/MS MARCO) are great, but they are likely already in distribution for foundation models training sets, and crucially, they lack the complex, structured metadata needed to test real-world filtering scenarios (e.g., "Find docs from region X, between dates Y and Z, with tag A").

datasetFactory is an orchestrated LLM pipeline that turns a single natural language prompt into a (potentially) massive, structured evaluation dataset.

it's an odd choice. I'd be curious why they picked that. it's not the cheapest, most expensive, best, or worst.

It does have a relatively large context window, and ime is very good at format adherence

well said! we demo'd milvus (or zilliz i should say,) and while we didn't ultimately go with it--it seems like a great option

Author is a human :). Performance and semantic accuracy are both important. The point about pre-filtering _youre still searching millions of vectors_ is important because once you apply a filter you can no longer use your vector index. And doing a full scan on millions of vectors is quite expensive

pgvectorscale is not available in RDS so this wasnt a great solution for us! but it does likely solve many of the problems with vanilla pgvector (what this post was about)

this is at the expense of precision/recall though isn't it?

i discuss that specifically!

> The problem is that index builds are memory-intensive operations, and Postgres doesn’t have a great way to throttle them. You’re essentially asking your production database to allocate multiple (possibly dozens) gigabytes of RAM for an operation that might take hours, while continuing to serve queries.

> You end up with strategies like:

    Write to a staging table, build the index offline, then swap it in (but now you have a window where searches miss new data)
    Maintain two indexes and write to both (double the memory, double the update cost)
    Build indexes on replicas and promote them
    Accept eventual consistency (users upload documents that aren’t searchable for N minutes)
    Provision significantly more RAM than your “working set” would suggest

> None of these are “wrong” exactly. But they’re all workarounds for the fact that pgvector wasn’t really designed for high-velocity real-time ingestion.

short answer--maybe not that _hard_, but it adds a lot of complexity to manage when you're trying to offer real-time search. most vector DB solutions offer this ootb. This post is meant to just point out the tradeoffs with pgvector (that most posts seem to skip over)

guess it depends on your scale? for some, 10+ GB of RAM being consumed on an index build is > 25% of the DB's RAM. apply that same proportion to your setup and maybe it'll make more sense

for sure people are running pgvector in prd! i was more pointing at every tutorial

iterative scans are more of a bandaid for filtering than a solution. you will still run into issues with highly restrictive filters. you still need to understand ef_search and max_search_tuples. strict vs relaxed ordering, etc. it's an improvement for sure, but the planner still doesn't deeply understand the cost model of filtered vector search

there isn't a general solution to the pre- vs post-filter problem—it comes down to having a smart planner that understands your data distribution. question is whether you have the resources to build and tune that yourself or want to offload it to a service that's able to focus on it directly

> Anyways, all this vector stuff is going to fade away as context windows get larger (already started over the past 8 months or so).

We're searching across millions of documents, so i doubt it

I mention this towards the end of the post. it looks like a good solution, but it's not available on RDS

how are all of the mentioned issues resolved?

some fair points points on the specifics.

> maintenance_work_mem

sure, but the knob existing doesn't solve the operational challenge of safely allocating GBs of RAM on prod for hours-long index builds.

> REINDEX CONCURRENTLY

this is still not free not free—takes longer, needs 2-3x disk space, and still impacts performance.

> HNSW vs B+tree

it's not that graph updates are uniquely expensive. vector workloads have different characteristics than traditional OLTP, and pg wasn't originally designed for them

my broader point: these features exist, but using them correctly requires significant Postgres expertise. my thesis isn't "Postgres lacks features"—it's "most teams underestimate the operational complexity." dedicated vector DBs handle this automatically, and are often going to be much cheaper than the dev time put into maintaining pgvector (esp. for a small team)

We actually looked into vectorchord--it looks really cool, but it's not supported by RDS so it is an additional service for us to add anyways.

I hate this so much and also great job

its 100% the correct take

the "AI lies" takeaway is way off for those actually using these tools. Calling it a "junior dev faking competence" is catchy, but misses the point. We're not expecting a co-pilot, it's a tool, a super-powered intern that needs direction. The spaghetti code mess wasn't AI "lying", it was a lack of control and proper prompting.

Experienced folks aren't surprised by this. LLMs are fast for boilerplate, research, and exploring ideas, but they're not autonomous coders. The key is you staying in charge: detailed prompts, critical code review, iterative refinement. Going back to web interfaces and manual pasting because editor integration felt "too easy" is a massive overcorrection. It's like ditching cars for walking after one fender bender.

Ultimately, this wasn't an AI failure, it was an inexperienced user expecting too much, too fast. The "lessons learned" are valid, but not AI-specific. For those who use LLMs effectively, they're force multipliers, not replacements. Don't blame the tool for user error. Learn to drive it properly.

GeForce Now https://www.nvidia.com/en-us/geforce-now/

nvidias cloud gaming offering. it works pretty well

you misunderstand infinity

not hard to see it

with open('wat_source.py', 'w') as f: f.write(zlib.decompress(base64.b64decode(code.encode())).decode())

i think pookha is saying their reformation was possibly inspired by the book

The title of the article is wrong, and the author describes the distinction perfectly (so weird they chose that title)

You can pick whatever words you want to describe the distinction, but it will remain. We need way to discuss products and whether or not the end user can work at the command line or need a user interface. If you're offended that you can't work at the command line (or think that makes you less valuable--I don't) then learn to work at the command line.

I was once asked in an interview if I was comfortable in 'text-only land'. How offensive!

Money

you're off by a bit, the announcement of Sam returning as CEO actually increased OpenAI valuation to $110B last night

Arr, me hearty, once I boarded the ChatGPT ship on the advice of a chirpy shipmate. Thought it'd help me navigate the treacherous seas of code, but nay, it didn't make sailin' faster. So I marooned it—left the subscription high 'n dry.

But listen ye, I boarded 'er again, findin' she be more of a muse than a swab. Aye, not for treasure maps or writin' the next epic of the sea, but for polishin' me own log entries and givin' structure to me scatterbrained musings.

Yarrr, it be like handin' a rough chart to a trusted navigator—ye get it back, sharper and clearer. While I don't take her words as gospel, that ChatGPT wench helps me tack in the right direction. She be a fine first mate for reworkin' yer sentences, bringin' clarity and firepower to yer words. And reckon it be quite the tool for rephrasin' yer thoughts as if ye be a pirate too, savvy?