archiewood

They have a table comparing what they are doing vs materialize in the FAQs in the docs (not super easy to find) - https://docs.epsio.io/faq/

On a quick read, seems like the core difference is around whether you query the streaming engine (Materialize) or still interact with your db. Other differences are then downstream from this.

> Not sure the language you choose matters as much as making the API usable by a wide audience.

Fully agree with this, and having typeset my masters thesis and later my resume using LaTeX, I think that the “authoring experience” is definitely the place to focus on improving, LaTeX just takes too damn long to get something good.

If you’re interested in the “markup to document publishing” space, you might also be interested in the open-source report publishing tool I’m now working on, Evidence.dev (https://github.com/evidence-dev/evidence).

It’s similarly based on markdown, though uses code fences to execute code, HTML style tags for charts and components, and {…} for JavaScript, i.e.

    ---
    title: Lorem Ipsum
    description: dolor sit amet, consectetur adipiscing elit
    ---

    ```sql petal_vs_sepal
    SELECT
      petal_length,
      sepal_length
    FROM iris_dataset_table
    ORDER BY 1 DESC
    ```

    <ScatterPlot
      title="Petal vs Sepal Length"
      data={petal_vs_sepal}
      x=petal_length
      y=sepal_length
    />

    The longest petal in the dataset is {petal_vs_sepal[0].petal_length}.

Our design philosophy here is that the rendered documents should be beautiful by default, but highly configurable so you can get pixel perfect results.

We’re also aiming for first class output options for desktop, mobile, PDF and image export.

Previous HN discussion:

https://news.ycombinator.com/item?id=28304781 - 91 comments

https://news.ycombinator.com/item?id=35645464 - 97 comments

Really interesting read.

Have you thought about how you might identify usability ‘gotchas’ in notation.

For example two similar visualization components where the property names for a similar concept differ. Eg `title` vs `name`

Oh yeah. Any manual data update has potential to go wrong.

Especially since in my exp the most common way to do this is to paste the new data over previous sheet in an excel, and hope all the formulas still work.

Kind of fine, but let’s hope there aren’t any new categories that weren’t there last month!

> Looker (which is hot garbage for other reasons as well)

From what I remember, Looker does allow you to create pivot tables from the Explore interface?

You can then also download to csv / excel from a Looker explore.

Something missing for you there?

> This is because Power BI tracks usage metrics and almost all dashboards are completely unused.

This is what I term the “dirty secret of BI” - for most organisations, most dashboards are unused, except by the teams creating them.

This is because reports need to be useful to be used, and most are not. In my opinion useful means one of:

1) Aiding people with actual decision making power to make decisions

2) Improving the ability of “front line” staff to complete their day to day tasks better

PowerBI and similar tools are poorly suited to the above because 1) requires mgmt to understand the data they are looking at, which is difficult without context in a grid of charts, and 2) requires a thoughtfully designed, fast UX that is basically impossible to deliver in a dashboard.

I’m now working on an open source BI tool, Evidence.dev (https://github.com/evidence-dev/evidence) which is targeted at these two use cases. It enables you to add context to data inline, design the UX for the use case, and is just fast.

Previous discussions on HN:

https://news.ycombinator.com/item?id=35645464 (97 comments)

https://news.ycombinator.com/item?id=28304781 (91 comments)

Step 1: Be a front end engineer

Step 2: Wait three months

I assume this is the problem that https://what3words.com/ is trying to solve. but I guess it being in English makes it a less good solution

Edit: they support hindi (and many other languages)

i will opt for `fix: make! code! great! again!`

If you are interested in this, you might be interested in the code-based BI tool I'm working on, Evidence (https://github.com/evidence-dev/evidence)

The experience is a bit more: Start writing code -> GitHub copilot autocompletes -> Dashboard

GIF of it in action: https://twitter.com/evidence_dev/status/1631477155229237248

Previous discussions on HN:

https://news.ycombinator.com/item?id=28304781 - 91 comments

https://news.ycombinator.com/item?id=35645464 - 97 comments

it's wild to me that they don't offer colour options

This is interesting. I've built reports in Dash before. As I remember it was kind of tricky to hook up the filters etc, and "callbacks" confused me for a while. Is that the kind of thing Vizro addresses?

Do you have a "why Vizro" kind of explanation somewhere? Would be really helpful early in the docs, or on the GH readme.

I also do this. Had two bikes stolen in quick succession a while back.

Now: Duct tape all over the frame. Different handle tape on each side.

No theft since

I built a VSCode extension so you can explore and query a DuckDb.

It’s built on top of the very excellent SQLTools ecosystem.

Find it’s much easier than the CLI to work out what your columns and tables are called.

It also has autocomplete for basic keywords.

See it in action here: https://x.com/archieemwood/status/16893165620359331856

Im in Firefox and it tells me 43% of the fingerprints are Firefox.

So seems very skewed!

is this satire?

It doesn’t work on Safari, and WebKit based browsers (and therefore, all browsers on iOS)

Shoukd work on Chrome, Firefox etc on all other platforms

Having just tried out Voila, I do think these products are serving pretty different audiences on the front end.

When I open examples in the Voila gallery, it spends 15s or so building the site, which a dev may tolerate, but a business exec has already given up on.

One of the benefits of a static site for reporting use cases is that it’s just HTML and JS. It loads virtually instantly as it doesn’t hit a database - all the data is shipped with the site.

Open it on your phone when you wake up, in a cab, by the gate at the airport: you’re looking at a mobile optimized site, and 5s later you can finish writing that email with the datapoint you needed.

Only came across blazer yesterday, on an awesome-BI list on Github, but I do really dig it.

Love how simple it is, the auto-configuration of charts etc. Even just playing around with the demo there was a lot to like: https://blazer.dokkuapp.com/queries/new

100%

I'd be interested to hear who you think the audience is here that can't code?

We're positioning the tool towards data teams who already manage their ETL layer using SQL, so our take is that they _do_ know SQL.

From the business-user perspective, it's just a website. Click to navigate around, drill into data etc.

(as a side point, there is no python, but I think the sentiment of this comment is around it being code, rather than the language specifically)

Yeah it's pretty awesome. Keen to try the API at some point soon to see if I can use it at higher scale.

Very cool project - much more data than mine...

Perhaps salaries seem to be declining because it's using job post listing salaries not actual paid salaries?

I can imagine the negotiation position to get above listed is much higher now than in 2019?

I scraped salary data to work out how data scientists can earn more:

- DS salaries have been going up at 7-9% per year

- Those with a PhD earn an extra $45k than those with a Master’s

- DS in the West is paid $30k+ more than other regions; those in the Southeast are paid the lowest

- After FAANG, the highest salaries were for other Tech, O&G and Healthcare

I scraped data from Reddit, cleaned it, and analyzed it to work out what influences data engineering pay.

TLDR:

- US or remote roles typically get a $40k+ higher base

- There's a steep salary trajectory with experience, rising from $73k at entry-level to $135k by Y5

- The highest salaries posted were for Telecom companies, followed by Tech

- Roles using Apache Spark or Airflow in the tech stack have highest pay

Full dataset on Github: https://github.com/archiewood/reddit_de_salaries/blob/main/d...

I feel the pain of internet vices.

What worked for me recently (for the last 4 months only - so can't claim victory yet) was exercise and a goal (so I don't bail). Decided to run a half marathon. Found a training plan where I run 5 days a week.

- Now I have to get up at 7am (Before: 8.55am for 9am start) - If I don't get to bed by 11:30pm I feel lousy running (Before: 2-4am bedtime) - And it makes me shower and eat breakfast (Before: no shower, no food)

It's basically added a bunch of structure to my day, which has really helped with work