Hacker News Reader

Comment by llarsson | Hacker News Reader

I don't see it as a problem that this will likely just work for, and thus correctly target, "easy" problems. Nobody looking at this blog post should think "hmm, sober of this lets me implement an autonomous vehicle in a single query", but rather "this looks like a great way to explore data without requiring any data science expertise". Many businesses have data and could gain insight and improve their business if they just ran some rudimentary queries to understand customer patterns etc, without needing a data science PhD to crunch numbers for them.

This seems to provide that type of value.

If I were developing this, I would make sure to make this able to read from Excel sheets and integrate with popular invoicing services etc, because that is where I believe the primary customers are in technological maturity.

View on HN

Agree entirely.I see this being implemented as a column type on analytics DBs such as Snowflake or Bigquery (feature of a bigger product) rather than a specific DB designed for ML.

The reason being that as you pointed out, this would be most useful for "easy" problems. These problems live with analysts who are using analytics oriented DBs as part of their Business Intelligence workflow.

I can see, why having predictive queries in solutions like Snowflake or Postgres would be extremely tempting.

Still, the problem of integrating the Aito like functionality in existing databases is that it requires lot of specialized data structures to work fast enough. While getting it to work in existing DBs is plausible, at minimum it would require completely new storage engine, or at least a wide refactoring to the old ones.

Regarding the analyst workflows: could you tell more about the main use cases? We haven't had Analyst / BI customers (yet), while they seem like a plausible audience for an Aito-like solution.

You work on this?

Writing storage engines for MySQL or even Postgres isn’t that hard.

Would you be our first customer, if we do it? ;-)

The fact of the matter is that, while it is a tempting idea it's far from easy. The interfaces may not be that hard, but the storage itself will have its challenges and building a fast ad-hoc inference & representation learning layers on top of it is a huge project.

After working on Aito's DB and ML parts for several years, I can promise: it's more work & harder than it looks :-)

Yeah I trust you, I use and contribute to rdbms and am a bit familiar with their innards.

The key thing you want to enable is eg Tableau. Building classifications and predictions into something business people rather than devs use would be a promising strategy.

Recently I’ve been using presto to make various things appear to be conventional db tables, and getting computed data into tableau that way.

Main use case for BI analyst is generally something along the lines of integrating data from a few data sources (CRM, e-commerce etc) into a single data warehouse and then building analytical investigations on that data, most typically to track and distribute KPIs.

Typically in this workflow there is some or other use case that involves some minor predictive element (eg: who are the most likely leads to convert to sales) which then requires some light ML to make a prediction. This often results in very little in the way of actionable outcomes, but doesn't seem to stop people wanting to do it.

I wrote a blog about the stack and workflow [1]. This is quite an established domain.

Probably an easier route for a snowflake customer is to call the ML function using this new Snowflake feature: External Functions[2]

[1] https://groupby1.substack.com/p/data-as-a-utility-tool

[2] https://docs.snowflake.com/en/sql-reference/external-functio...

llarsson, I think you are spot on with this comment :-)

Also thank you for the advice. We have indeed gotten a good response in the less-technical, but more business oriented, RPA developer community, as an addition to appealing to the traditional software developers. (Many data scientists can obviously appreciate the idea, but the added value of this kind of tool is not that high, if you already have a routine for doing custom models & analytics)

Still, we have also faced situations, where the users have been too non-technical. I think the jury is still out on, whether these kind of solutions can be easily used by the average accountant or a business analyst.

For those last ones, try to figure out if there are a few systems that they commonly use and make integrations with those and be able to provide some kind of answer to interesting queries.

It's great that you have a general solution, but even those need to be made into specific products and services that non-technical people can appreciate for the value they bring. And the message needs to be crystal clear for them, because a general solution is too broad.

"This will analyze your customer behavior and tell you which ones will likely order new units from you next month or quarter, and how much."

A broad and general tool like yours does not, in itself, solve the problems for them, so you have to help them understand what it can do.

Reply on news.ycombinator.com