HNHacker News
TopNewBestAskShowJobs

mjirv

1,361 karma · joined June 21, 2014

Software and analytics engineer

https://delphihq.com

submissionscomments
mjirv··on Show HN: Acolyte – Query your data warehouse in plain English
Ah yeah that’s a good example of something it can’t do well. Right now you have to give more context, e.g. “How many orders were placed by someone named Howard?” works for me.

Thanks for sharing about Seektable. Neat that you’ve already built out a search function, and even aside from that it looks like a really interesting tool.

mjirv··on Show HN: Acolyte – Query your data warehouse in plain English
For those having trouble with the URL, you can also use http://acolytehq.com
mjirv··on Show HN: Acolyte – Query your data warehouse in plain English
Hi all,

I've spent the past 4 years working as a data analyst and analytics engineer. One of the biggest pain points for analysts is that, even with tools like Looker and Tableau, they still spend a ton of time answering fairly simple questions from less-technical users instead of doing high-value analytics.

Often as a data analyst I've felt like a computer translating requests from English into SQL. That got me wondering - could an actual computer do it instead?

For a while, I thought the answer was "no" (or at least not in the near future). But then OpenAI's Codex[0] was released. I started using GitHub Copilot[1] every day while writing SQL in Visual Studio, and it felt like magic. The autocomplete was getting my code right a lot of the time, even for complex SQL queries.

So I decided I'd give this a go. I've built a demo using the Codex API and dbt Labs's "Jaffle Shop" example database, which will be familiar to many in the Analytics Engineering community. It's a simple database, but the results are promising. The demo app gets a lot of queries right, and it does so without any database-specific logic.

The only input needed is a schema.yml file describing the tables and columns (which any dbt project would have) and 2 or 3 example queries, so I am hoping to open up testing to more data warehouses and users in the near future.

I hope you enjoy using the demo. If you have any feedback, ideas, or would like to test it out on your own data warehouse, my email is in my bio!

[0] https://openai.com/blog/openai-codex/ [1] https://copilot.github.com/

mjirv··on Show HN: Low-Code Metrics Store
As a current Looker power-user, this looks really solid.

One thing I’m not sure about though: can you use the metrics outside of the native tool, and if so how?

That is, I see Looker as a BI tool, not a metrics layer, since you mainly use the metrics you define inside Looker, not in other tools. On the other hand, something like MetriQL[0] is a pure metrics layer that can supposedly be used anywhere.

Is this both? If so, some better documentation around how to use the metrics layer would be helpful (or maybe I just didn’t look in the right place).

[0] https://metriql.com/

mjirv··on Eradication of Covid Is a Dangerous and Expensive Fantasy
At this point, Covid is with us to stay. The authors are clearly right about that.

But I'm not sure that has the policy implications they think it does. No one in the US (or outside of Australia/NZ/parts of Asia) have really been _trying_ to eradicate Covid.

But at some point, we will have to figure out what the goal is of our Covid policies.

At first (Spring 2020) the goal was to "flatten the curve" i.e. prevent hospitals from being overwhelmed. We largely did that.

Next, the goal was to postpone as many infections as possible until vaccines were widely available. They are.

So the question is: what comes next? Perpetual restrictions and lockdowns every time there is a flare-up to try and prevent as many (mostly unvaccinated) deaths and hospitalizations as possible? Or do we just accept that everyone is going to get it eventually and we've done all we can to minimize the risk (to people who choose vaccination)?

mjirv··on Show HN: Work with CSV files using SQL. For data scientists and engineers
I like that this lets you join CSVs, which a lot of tools in this space don’t.
mjirv··on Building a data team at a mid-stage startup
Analytics Engineer is a clear one for this, as teej said.

The title is strongly associated with the dbt community, so it could imply you’re using dbt for your data modeling (not necessarily a bad thing, as it sounds like it would be a good tool for your use case).

mjirv··on Launch HN: Polytomic (YC W20) – Get internal data to your business teams
I’m happy to see this launch. We use Census for the same thing (mostly to get data from Redshift into Salesforce), but there don’t seem to be that many other companies who do this well.

I am interested to see how long it’ll take someone like Fivetran to buy one of these new “reverse ETL” companies so they can be a full end-to-end solution.

mjirv··on Contextual: Advertise without 3rd-party tracking
Hey everyone! I started working on the idea behind Contextual after becoming a bit disillusioned with the current state of AdTech, especially the tech being created as a response to GDPR and browsers dropping 3rd-party cookies.

I used to work in AdTech and keep an eye on the industry. Most of the responses to those developments have been around consent management and first-party authentication, which are disappointing both from a privacy lens and as an advertiser (they often rely on signed-in users which are usually going to be a small % of traffic).

To me, this shift towards privacy feels like an opportunity to reinvent online advertising.

My core question is: what if contextual advertising (based on the content of the site/page you’re on) could be as good or better than all the creepy tracking that happens now? I think the answer is that with recent developments in natural language processing (NLP), it can be.

Thanks for reading, and I’m happy to answer any questions!

mjirv··on I asked an online tracking company for all of my data (2018)
These kinds of articles always make me think about how big a privacy risk the laws that make these possible are. We're worried about user privacy, so we mandate that companies build a way to connect PII to otherwise-pseudonymous browsing data in case the user asks for it. What if someone else asks for it? Huge security risk.

That's not to say these laws are bad, just that the giant additional privacy risk they pose is kind of funny to think about.

mjirv··on The evidence which suggests that Covid-19 is not a naturally evolved virus [pdf]
It is worth clarifying that she was not responding to this paper specifically, just to general arguments of the type “this must be man made because it has so many features that make it well-suited to spread in humans.”

The fact that researchers in Wuhan were studying (some of?) these features is stronger evidence when combined with the location. However, it also seems reasonable that features of viruses researchers study because of their likelihood to increase human transmission would show up in a naturally occurring virus with a lot of human transmission.

mjirv··on The evidence which suggests that Covid-19 is not a naturally evolved virus [pdf]
Epidemiologist Ellie Murray said it better than I could, so I’ll just quote her: “Yes, the chance of a virus like SARS-CoV-2 suddenly arising is fairly low. That’s why we have so few pandemics!!

But conditional on being in a pandemic the chance that the virus that arises to cause it is like SARS-CoV-2 is super high!!

There’s no conspiracy, just statistics”[0]

That said, the fact that the outbreak started so close to the Wuhan Institute of Virology is certainly suggestive that it escaped from the lab, though that doesn’t mean it was necessarily “man made.”

[0]https://twitter.com/epiellie/status/1281960135796101120?s=21

mjirv··on Show HN: PaperPup – Discover new articles from blogs and websites in one UI
Thanks! Users can add new RSS feeds already, though it’s a bit hidden. If you click the “New Collection” Button, there’s a link at the bottom of the pop-up that says “Don’t see your favorite blog or website? Add it here” which lets you add a new feed.
mjirv··on Show HN: PaperPup – Discover new articles from blogs and websites in one UI
It’s something I’d like to do, since I’ve noticed using it on my phone isn’t the best experience (though also not the worst-hopefully it’s the same on other kinds of phones! Let me know if not).

It’ll probably be the next big “feature” I work on.

mjirv··on Show HN: PaperPup – Discover new articles from blogs and websites in one UI
Thanks! I actually hadn’t come across Memeorandum before, but it looks neat. I like its simplicity as well, and the previews it shows. One issue I ran into making this is that not all RSS feeds seem to contain descriptions, so it was hard to make something like that work and look consistent.
mjirv··on Show HN: PaperPup – Discover new articles from blogs and websites in one UI
Thank you, I’m happy you like it! Glad my laziness as a developer is also appreciated haha.
mjirv··on Show HN: PaperPup – Discover new articles from blogs and websites in one UI
Ah yes that's probably something I should add. It irked me too when I was starting out but then I got used to using the "Home" button and forgot that I needed to do something about it.

UPDATE: There is a back button now.

mjirv··on Show HN: PaperPup – Discover new articles from blogs and websites in one UI
Hey HN! I had the week off for the 4th of July holiday but canceled my vacation plans due to COVID, so I figured I'd do one of those "7 days of code" things where I do 7 different projects in the week. I wanted to create some things I would use and keep my programming skills sharp since I do data analytics rather than software engineering as my day job.

This was supposed to be Day 2, but ended up taking me about 3-4 days to get to the current state.

The problem I wanted to solve is that basically, I am addicted to Twitter. I get my news from Twitter, so I'm constantly pulling it up and scrolling through it throughout the day. But there's a lot of noise on Twitter--a ton of low quality content, mean-spirited comments, and so on. I wanted a way to stay on top of the news that met the following criteria:

1. One place to see new articles/posts from a bunch of different sources. I didn't want to have to pull up each news site individually.

2. No comments or noise--just the latest headlines.

3. Customizable - I can add new sources I come across, group them in different ways by subject.

4. Public - If someone else added some interesting news sites, it would be cool to see that and find something new to follow myself.

5. No authentication - Handling passwords and user state is a pain. Just let everyone see everything!

So I made "PaperPup" (the name came from the idea of your dog fetching you the morning paper). While there are other RSS aggregators out there, a lot of them feel clunky to me and don't necessarily have the public aspect I wanted. In technical terms, it's a Rails API backend with a "kind-of-vanilla" HTML/CSS/JQuery single-page front-end. As I mentioned I'm not really a software engineer, so this is nothing fancy; I wanted to make sure my skills on some of the basics were still up-to-date rather than experimenting with whatever the hot new languages/frameworks are these days.

Anyway, I hope you all like it! If you have any suggestions or questions (technical, feature-wise, or other apps to use), I'd love to hear them.

mjirv··on Ask HN: If you could work remote where would you live?
Barcelona. Maybe Lisbon and other parts of Spain or Portugal.
mjirv··on Ask HN: If you could work remote where would you live?
I for one am hoping that house prices drop with all the tech people moving out of the Bay Area. That way I can move back to Rockridge (North Oakland) to work remote from there without paying an arm and a leg.
mjirv··on Ask HN: What startup/technology is on your 'to watch' list?
1. Fishtown Analytics - makes dbt, a sql data modeling tool that has really caught on in the analytics world over the last year or two

2. Bubble - no-code!

3. Stripe - already big but has the potential to be the next Google/FB/MSFT etc

mjirv··on DataJet – Fully managed Apache NiFi as a service. Visual Data flows without code
For anyone who has used DataJet or knows about it - why would I choose it over a tool like Matillion?
mjirv··on Taleb: The only man who has a clue
To be a little pedantic, this isn't really how I would summarize Taleb. His main idea is that Black Swan events are extremely important because of their outsized impacts, not because they are more frequent than people think.
mjirv··on Taleb: The only man who has a clue
I recommend reading Fooled by Randomness (his first book) if you want the one that maximizes substance and minimizes the bluster.
mjirv··on Actual Causality (2016)
Anything by Judea Pearl, but especially the Book of Why[1], is good. He comes at causality from a CS perspective, which I think would make sense for most people on here.

Economists also have a big causality literature which might be less accessible for HN folks but I think is still interesting and important. For a good intro to all that, I suggest Scott Cunningham’s “Causal Inference: The Mixtape.”[2]

[1] https://www.indiebound.org/book/9780465097609

[2] https://scunning.com/causalinference_norap.pdf

mjirv··on Show HN: Visual SQL
Oh neat, I didn't know that about BigQuery. Thanks for the tip!
mjirv··on Show HN: Visual SQL
I like the user-research-first approach here a lot. Very different than many other BI tools in a refreshing way. In the past I've hesitated to consider Chartio for my organization because most people don't know SQL, but I'm excited to give it a try now.

I also love how everything is a table. Looker does this, and it's great (though Chartio's implementation looks better). Tableau does not, which makes certain things a headache.

I do wonder how well this will scale to large datasets and columnar databases, though. Showing a preview of the tables is really cool and useful, but it is not what data warehouses like Redshift are designed for. The preview probably wouldn't load in time for my users to see it unless they just sit there waiting for it.

mjirv··on An introduction to data processing on the Linux command line
This is a great list and well-written. As a data professional, I use these commands all the time and my job would be much harder without them. I also learned a few new things here (`tee` and `comm`).

I was lucky that my first job was as a support engineer at a data-centric tech company, which is where I learned these. I've often thought about how to teach them to data analysts coming from a non-engineering background. This is comprehensive but clear and would be a perfect resource for training someone like that. Thank you!

mjirv··on Ask HN: What does your BI stack look like?
Stitch -> Redshift (with DBT for data modeling) -> Looker

For a smaller company, it makes a lot of sense for us to use off-the-shelf tools like these rather than rolling our own pipelines. Every once in a while we run into issues where Stitch isn't quite as flexible as we'd like (can't set dist/sortkeys etc), but that's the tradeoff you make for its simplicity.

DBT is amazing and I can't recommend it highly enough!

Looker works for analytics, but we're starting to do more data-sciency work, and it doesn't have built-in Jupyter notebooks or anything like that. Does anyone have a solution they use and like for hosting and sharing notebooks alongside a stack like this?

mjirv··on Show HN: Ananas – a hackable data tool for beginners
Metabase is much more for data visualization. As far as I know it doesn't really have any ETL features.

I could actually see this being pretty useful as an ETL layer to go along with Metabase if someone were trying to build a free/open source BI stack.

← PreviousPage 3 of 4Next →