Hacker News Reader

Comment by dennisy | Hacker News Reader

Does this mean we will have people “vibe coding” data warehouses now? Might cause a few issues…

We say "data vibing" so it feels unique to the data community! But in all seriousness, this is already an issue, people are already asking ChatGPT (or Cursor/whatever else) to generate SQL for them, but the next steps do not exist, if you "vibe code" for data you want to have the easiest feedback loop you can get to check if the output is good, and that's what we are working on: identifying the downstream impacts in the IDE and proposing fixes, a table diff view, new UI/UX to test your outputs.

The goal for us is to be the best way to do data with AI.

Ok, but how do you know it’s good?

With data I think that is very hard, I wrote a SQL query (without AI) which ran and showed what look like correct numbers only to years later realise it was incorrect.

When doing more complex calculations, I am not clear how to check if the output is correct.

Usually what we've seen is data people having notebooks/worksheets on the side with a bunch of manual SQL queries that they run to validate the data consistency. The process is highly manual, time consuming. Most of the time teams knows what kind of checks they want to run on the data to validate it, our goal here is to provide them the best toolbox to do it, in the IDE.

Tho, i'd say this is like when writing tests in software, you can't catch everything the first time (even when going 100% code coverage), especially in data when most of the time it breaks because of upstream producers.

It will still require live observability tools monitoring data live in the near future.

Reply on news.ycombinator.com