Hacker News Reader

Comment by mble_ | Hacker News Reader

Author here.

One of the main things here is that you should know your data well enough to articulate the right request from BI. In my experience, BI often end up as pure order takers - if you ask the wrong question, you get a lovingly formatted but wrong answer.

The other thing is that this assumes you have a BI team at hand - smaller teams/orgs often don't! Perhaps I should make this a little more explicit.

My central thesis, also not made explicit, is that leaders should be appropriately curious _and_ leverage the tools they have to be able to do things like "hey, this looks weird, what's up?" and share the data and their methodology - that way it can be corrected/investigated etc.

View on HN

Thanks for chiming in, great post, I like the premise - I just think we must have completely different working experiences. I'm typically in a larger org that has multiple systems feeding data into a data lake or something similar that has been normalized but also can still usually has some quirks. Articulating the right request to BI is certainly a skill, but my approach/experience is that I try to paint the picture of the end goal and let them fill in the gaps as needed. Sometimes that's literally drawing out a graph or chart that I want to exist.

Even when no BI team is dedicated, there's usually someone that's wearing that hat. Someone setup those schemas and data pipelines, etc or is responsible for maintaining them. That person is probably the one that knows "make sure you exclude the NULL items" or something similar.

I do like being in touch with changing data trends from a leadership perspective. It's either real and could be a valuable insight or it's a bug that needs to be addressed before any ill advised decisions are made from the 'info'. I find this can often be setup proactively and put into a dashboard. In that way, identifying it and raising concern can be 'my job' but when investigating it, it could be a team effort.

> I just think we must have completely different working experiences.

Likely! I've generally worked in smaller orgs (including as part of a much larger org, as with my current employer) and there is less access to dedicated resources.

> Even when no BI team is dedicated, there's usually someone that's wearing that hat.

100%. Unfortunately, this has commonly be me from my personal experience.

> In that way, identifying it and raising concern can be 'my job' but when investigating it, it could be a team effort.

Totally agreed.

For some additional context, I've spent my working career on data systems so I likely feel a much stronger affinity to this type of self-serve analysis than your average bear.

Confirms my initial impression that the the author, you, were likely on the receiving end of these requests and would rather teach people to fish than being the cook. Which is a great thing and certainly has a place, especially on smaller teams/orgs. So I think the bias is strong (desire for others to self serve) and ignores a lot of the realities of trying to 'manage up' in this way (the risks and inefficiencies and skills gaps of having managers exercising technical chops). For that reason, I feel like promoting usage of Retool or something more GUI based would be more successful than promoting that managers should start using DuckDB and Python, et al.

From my POV I have a choice of a database or a tool like pandas. Anybody who is interested in this sort of work has a choice of doing it with databases or with a specialized data analysis tool. What's your take on that?

Why not both?

There are times when pushing the work down to the database layer is appropriate - databases are quite good at a lot of these operations - but if you need more nuanced approaches (e.g. ANOVA, ARIMA, other kinds of forecasting or analysis), leverage the appropriate tools.

Reply on news.ycombinator.com