AlloyDB AI: Generative AI applications with PostgreSQL | Hacker News Reader

AlloyDB AI: Generative AI applications with PostgreSQL

cloud.google.com

How are people actually using vector databases?

The closest explanation to a use case architecture I've seen recently was https://mattboegner.com/knowledge-retrieval-architecture-for... - it basically describes doing knowledge retrieval (keyword parsing) from LLM queries, feeding that to a vector db to do similarity search to get a top K similar documents to the parsed keywords, then feeding that list that back into the LLM as potential useful documents it can reference in its response. It's neat but it seems a bit hacky. Is that really the killer app for these things?

Yes, GPU poor people are just using top k semantic search to try to fix the issues will low ram low knowledge LLMs. It's OK for some applications, but other methods need to be investigated.

We used it in an e-commerce application.

Apparently, one of the hardest things to do is to match a product name + description to a product taxonomy.

There are multiple taxonomies. Here's Google's for example: https://www.google.com/basepages/producttype/taxonomy.en-US....

Amazon has their own. Walmart has their own. Target has their own.

Given a list of tens of thousands of products, how can you automatically match the product to a merchant's taxonomy?

I started with a "clever" SQL query to do this, but it turns out that it's way easier to use vector DBs to do this.

    1. Get the vector embedding for each taxonomy path and store this 
    2. Get the vector embedding for a given product using the name and a short description
    3. Find the closest matching taxonomy path using vector similarity

It's astonishingly good at doing this and solved a big problem for us which was building a unified taxonomy from the various merchant taxonomies.

You can use the same technique to match products with high confidence across merchants by storing the second vector embedding. Now you have a way to determine that product A on Target.com is the same as product A' on Walmart.com is the same as product A'' on Amazon.com by comparing vector similarity.

Could this strategy work to match products across retailers? If so, any tips on getting started with vector databases? I've heard of them but have yet to try one out.

Yes. You compute the embedding for the product name + description from Target.com and then the embedding for the product name + description from Walmart.com. They'll have a very close vector similarity.

The easiest way to get started is with Supabase since it has a free tier and the pg_vector plugin built in.

You calculate the embedding using OpenAI's embeddings API and store the result. Then it's just a vector similarity query in Postgres (trivially easy).

Another way to do this is using the pgml extension. You can run huggingface embedding models, which have surpassed OpenAI's at this point. It's pretty fast if you run it on a machine with a gpu for acceleration. I've created embeddings on my local desktop with a 3090 for ~2,000,000 tokens in chunks of ~100 (450 characters). It took around 20 min using the gte-base model including insert into indexed table.

Still uses pg_vector.

Very cool that there's a "preview" downloadable version: https://cloud.google.com/alloydb/omni

> AlloyDB AI allows users to easily transform their data into vector embeddings with a simple SQL function for in-database embeddings generation

Slick!

Is/will AlloyDB be open source?

No, it’s Google’s proprietary product meant to lock you into their Cloud offering.

As another person noted, there’s a downloadable version at:

https://cloud.google.com/alloydb/omni

"Available at a fraction of the cost of legacy databases, with a free developer edition"

So I guess there is some kind of licensing model even though you can download it and run it on your own servers.

Yes, perhaps there is a free developer edition, but for commercial deployment you have to use their cloud. That'd be a bummer..

Disclaimer: I work on Google Cloud

Yes it’s commercial but you can run it on any cloud or on prem.

"Free while in developer preview, check back in the future for pricing". Too weird for me.

After that you'll be able to pay for it for a year or two until they realize that running a business is a whole thing and then they'll just deprecate it and kill it off.

Is AlloyDB basically like Aurora DB for AWS?

Yeah its closest analogy is Aurora, but no serverless option, payment is per vCPU, memory, and storage.

Is Google Cloud SQL a more direct alternative to Aurora? I'm having a hard time keeping all these database services straight.

Cloud SQL is RDS; AlloyDB is Aurora.

Architecturally yes but with an extension of in-memory columnar format as well. This makes it an HTAP (Hybid Transactional and Analytical Processing) database.

on steroids it seems

> AlloyDB AI allows users to easily transform their data into vector embeddings with a simple SQL function for in-database embeddings generation, and runs vector queries up to 10 times faster than standard PostgreSQL. Integrations with the open source AI ecosystem and Google Cloud’s Vertex AI platform provide an end-to-end solution for building gen AI applications.

- Embrace [X]

- Extend [X]

- Extinguish [?]

Will they allow it to use custom embeddings?

lol couldn't you say that Postgres is EEE for SQL? It embraced SQL, has added new functions and incompatible constructs to SQL... is it going to extinguish SQL next?

It does provide plenty of PG specific capabilities, but PG has always had much better conformance to SQL standards and support for all the standard features than other DBs like MySQL. It's one of the most compatible DBs from a few points of view.

Yes, that's the "embrace" part.

I guess my point is that "Embrace, Extend, Extinguish" is a really outdated saying and I hate it. Embracing and Extending are really good things.

The "Extend" stage is the key: it's actually supposed to refer to attempts to introduce "better" / "more advanced", but proprietary and inherently implementation-locked technologies into an implementation (and thus into the platform the implementation implements); where these technologies don't complement, but instead supersede the use of the platform's existing open-standards capabilities, for those using this implementation.

In the "Extinguish" stage, the open-standards capabilities are then deprecated in favor of these proprietary technologies — in theory just for the implementation itself, but in practice ecosystem-wide, due to the implementation's majority user share. As the locked-in nature of the component makes it impractical to offer any other implementations of it, all other implementations "fall behind" due to lacking this component, supporting fewer and fewer greenfield projects, until users give up on them, "Extinguishing" their market-share.

The "Extinguish" stage only makes sense insofar as the technologies introduced in the "Extend" stage have inherent implementation lock-in. Otherwise, it's just a regular open-standards dueling-RFC-banjos scenario, not EEE.

With Internet Explorer, the "Extend" phase was when Microsoft attempted to get people to write ActiveX components (that could only ever possibly work on Windows, since the ActiveX "platform API" is just "the entire Windows API") instead of writing JS. If doing this had become popular, no website that did it would have worked on any other browser than IE (and so there would have had to be "the ActiveX-enhanced IE version" and "the regular HTML version" of the site. Which was actually a thing for a short while... though only on Microsoft's own web services, AFAIK.)

I think what you're trying to say is that just because someone - especially [large company] - tries to improve/integrate popular open projects doesn't mean it's always EEE.

Which I doubt EEE is purposeful the majority of the time initially, even if has the potential to become that later. In the case of google I think this would be a case of "how do we add value to our product to sell" followed by "this feature costs us too much resources to maintain, let's cut it and focus on [new feature]"

That being said Postgres is not a good example because AFAIK they're both not a commercial entity (or any registered entity at all?) and they're true FOSS. At the moment they have no commercialization strategy (sponsorships maybe?) and no ability to extinguish anything given their open license and lack of any proprietary services.

Postgres is open source and free, so accusing them of EEE wouldn't make any sense.

It's not like Google is famous for dropping projects.

Reply on news.ycombinator.com