Hacker News Reader

Comment by arauhala | Hacker News Reader

I can see, why having predictive queries in solutions like Snowflake or Postgres would be extremely tempting.

Still, the problem of integrating the Aito like functionality in existing databases is that it requires lot of specialized data structures to work fast enough. While getting it to work in existing DBs is plausible, at minimum it would require completely new storage engine, or at least a wide refactoring to the old ones.

Regarding the analyst workflows: could you tell more about the main use cases? We haven't had Analyst / BI customers (yet), while they seem like a plausible audience for an Aito-like solution.

View on HN

Main use case for BI analyst is generally something along the lines of integrating data from a few data sources (CRM, e-commerce etc) into a single data warehouse and then building analytical investigations on that data, most typically to track and distribute KPIs.

Typically in this workflow there is some or other use case that involves some minor predictive element (eg: who are the most likely leads to convert to sales) which then requires some light ML to make a prediction. This often results in very little in the way of actionable outcomes, but doesn't seem to stop people wanting to do it.

I wrote a blog about the stack and workflow [1]. This is quite an established domain.

Probably an easier route for a snowflake customer is to call the ML function using this new Snowflake feature: External Functions[2]

[1] https://groupby1.substack.com/p/data-as-a-utility-tool

[2] https://docs.snowflake.com/en/sql-reference/external-functio...

You work on this?

Writing storage engines for MySQL or even Postgres isn’t that hard.

Would you be our first customer, if we do it? ;-)

The fact of the matter is that, while it is a tempting idea it's far from easy. The interfaces may not be that hard, but the storage itself will have its challenges and building a fast ad-hoc inference & representation learning layers on top of it is a huge project.

After working on Aito's DB and ML parts for several years, I can promise: it's more work & harder than it looks :-)

Yeah I trust you, I use and contribute to rdbms and am a bit familiar with their innards.

The key thing you want to enable is eg Tableau. Building classifications and predictions into something business people rather than devs use would be a promising strategy.

Recently I’ve been using presto to make various things appear to be conventional db tables, and getting computed data into tableau that way.

Reply on news.ycombinator.com