karakanb

Your link seems to be pointing to the older M3 model. The correct link is this: https://artificialanalysis.ai/models/comparisons?compare=mim...

Hi there, creator of Epho here (https://epho.io).

This looks very interesting, letting people run these in any box they own. I very much agree with the sentiment that there are no proper tools that let you run any coding agent without being locked to a single provider. I just want to run opencode or pi somewhere in a box without having to spin up all of them in my machine, let alone being able to trigger them from within another prouduct as a background agent.

Would pi-pod allow me to standardize pi config on a team/project level so that other people in my org can also use them on the same private infra?

Hi there, Burak here, creator of DaC and Bruin: https://github.com/bruin-data/dac

This is obviously a space we are very much interested in, so it is definitely nice to see approaches that attack the same problem. I haven't played with dbtcharts in depth yet but it looks very similar to DaC in principle, and also in the actual spec. I believe the industry definitely needs solutions like this to help get out of the legacy BI tools as Bİ is one of the biggest bottlenecks for AI adoption in large orgs.

Excited to see further competition in the space, nice launch!

I launched Epho a few weeks ago as an API like this but for all harnesses: https://epho.io

I built it primarily for ourselves: we are building an AI data engineer, and we need a way to run many of them in parallel securely. An API for this seemed like the most obvious path forward. It makes it trivial to bring agentic capabilities into any product surface without having to deal with sandboxes, reliability issues, compatibility problems, and more.

I think it also makes sense from OpenAI's perspective to do this, but also we did find ourselves needing to change models and harnesses quite a bit, which is why I think this needs to be a layer of its own above the labs. It also needs to be a layer above the sandboxes, since many of them are quite brittle.

Overall, I expect a lot of the agent implementations to move in this direction. I think this is a lot saner for engineers to implement and maintain, and it makes it trivial to build agentic stuff into products.

I wasn't expecting this, and I am pleasantly surprised. I think AWS has been a bad player in the data space and very much behind all the other warehouse providers for many years now. This could lead into interesting integrations of DuckDB on top of all the data that lives in S3 and provide a leaner, faster and a more modern alternative to Athena.

One would think. Conceptually they are nice, practically it gets very dirty across different databases + all the infra around them. My dislike stems from the operational complexity and years of trauma around it.

Disclaimer: I am cofounder of Bruin (https://github.com/bruin-data/bruin), we are a competitor to Erathos.

It seems like a relatively straightforward marketing article. I was pleasantly surprised to learn about Erathos though, nice product!

I am personally not a big fan of CDC in prod. Streaming data movement is generally prone to confusion, and it feeds into bad data patterns like hard deletes without any audit logs, no timestamps on updates or deletes, etc. which are usually the reason why batch loads cannot be utilized. They require a decent operational understanding of the underlying database, and have some gotchas like the Erathos folks mentioned in the article. We offer CDC both in our cloud platform, as well our open-source tools, but if I could, I would always pick an incremental batch load with a cursor value over a CDC connection.

I understand it is sometimes required due to organizational complexity or legacy database reasons, mine is just a personal preference.

If anyone is looking for an open-source CDC tool that runs as a standalone Go CLI, check out ingestr: https://github.com/bruin-data/ingestr

I got excited for this not because this didn't exist before, but because Docker putting their weight on this would imply a broader adoption and better integration in the industry. I am sad that they are asking for a login here though, which doesn't make any sense to me.

Hi all, this is Burak, co-founder at Bruin. We built a new feature called data apps, and I recorded a quick video showing how I built the actual Scranton office of Dunder Mifflen from the show The Office to report on data warehouses.

For context: Bruin is a data platform that does ETL + BI. We do a lot of things open-source, and recently we launched Dashboard-as-Code: https://github.com/bruin-data/dac

The idea behind DaC was that using AI agents would require a more code-driven approach. That turned out to be directionally right, however it fell short in terms of unlocking even more value as people became more ambitious.

That's why we built data apps: - These are React apps that connect to your database / data warehouse - They work with all kinds of databases + external APIs - They can make external requests in a secure way without the credentials being embedded into the app itself.

This allows building all kinds of interesting and capable UIs with them. The dashboards you build become interactive internal apps that works with your data.

I have written a bit more about them here: https://getbruin.com/blog/announcing-data-apps

I would love to hear what you all think about generating more flexible dashboards on top of warehouses and what could be different. I wanted to offer this as a Show HN for folks to try out but it is currently a bit too expensive to run this publicly for us, that's why we are initially enabling access to the feature only to a select set of customers.

Let me know if you have any thoughts or what would you imagine seeing in this space.

İngestr is meant to be a lot more simpler to operate than Debezium and the likes of it, so yes, it makes things a lot easier.

About the staging dump, I haven't tried it but I don't see why it wouldn't work? I'll give it a try.

Yes, we'll create and share a helm chart, although we already publish Docker images so it should be trivial to spin up a chart if you wanted to.

One of the reasons why ZeroFS seems interesting is they use SlateDB under the hood, which optimizes the requests that hit S3 behind the scenes.

It is not obvious from the post but it seems like the allow list for the scripts supports whitelisting packages instead of a global setting. This should make it easier to maintain org-wise rules to allow scripts only for specific packages.

Is there a linter that could be used for scenarios like this to prevent unsafe default on package manager config?

Hi, this looks interesting, thanks for sharing. I am the builder of ingestr (https://github.com/bruin-data/ingestr), so I am very much in the same space.

I really like that you did this in Go, and I'll definitely dig a bit more into the source code to see how you tackled the CDC stuff, given that there is not many reliable CDC libraries in Go, and there are quite a few gotchas when it comes to doing CDC right. We also hand-rolled ours in ingestr, or I must say clanker-rolled, and we got quite a few things wrong in the first place.

Curious about the postgres-compatible query option: what's the usecase you have in mind there? My perception is that any org that would use Iceberg also has one or a few query engines in place, is this more for debugging stuff?

Quite cool stuff, keep it up!

Zot seems interesting, this is the first time I see it. On a quickl look it seems like Pi, but in Go. I was hoping to embed Pi into some of our internal projects and the typescript stuff was blocking me, I'll definitely give Zot a look.

That's a good point, thanks!

The project has been under development for over 6 months. We just open sourced it with a clean history. I am not sure what you expect here, should a project exist for months before it is worthy of a show post?

DAC currently supports YAML and JSX, what else would be a good alternative?

Thanks! DAC does that kind of validation partially, although doesn't validate the usage of the downstream dashboards. That's a very nice idea though.

In terms of validation it will validate queries, metric definitions, chart definitions and all ahead of time, before render. That way agents tend to validate their work much quicker.

I am not sure we are on the same page, as far as I am aware Vega doesn't do layout, does it? E.g DAC could use Vega for the charts and still take care of everything else around it.

There are quite a few libraries for charts and visualization, there are not as many for actually combining many of them with layouts, different components and including the actual implementation of the backend. Dac aims to provide all that as a standard and an implementation.

Point taken, thanks! Out of curiosity, which DAC did you come here for?

Partly, yes. It is a simplification with the perspective that the agents would be the primary builders.

You write a few lines of YAML or JSX and you get a dynamic, interactive dashboard out of it. Do you have any suggestions on how to make it simpler?

That's a great idea, will do very quickly, thanks!

Hi, author here, thanks! I have used TypeScript before across various projects, but I haven't considered building CLI tooling in that before, I guess due to my prejudice against the whole JS ecosystem. I plan to give it another try in the next weeks.

I have been recently looking into extracting a bunch of details from a set of legacy invoice PDFs and had a subpar experience. Gemini was the best among the ones that I tried, but even that missed quite a bit. I'll definitely give this a look.

It seems like such a crowded space and there are many tools doing document extraction, I wonder if there's anything particular pulling more attention into the space?

This seems to undermine the engineering muscle these companies have. Fivetran is well-capable of building a query engine, and with this merger, they also get access to SDF's query engine. They have the engineering capabilities, as well as the capital to attract the talent where needed.

I would not underestimate any of these players in the space.

OpenFlow feels like an attempt at keeping customers within Snowflake boundaries, and while it might work for some, I see no way Snowflake being able to keep up with the data integration needs unless they allow another way of extending their capabilities other than pre-built integrations.

On the other hand, I do agree with you that it is quite a big challenge for Fivetran to try and become Snowflake.

Disclaimer: I am the co-founder of a dbt alternative, Bruin. (https://github.com/bruin-data/bruin)

I think consolidation in the space has been coming for quite some time now and this merger only confirms what us, along with many others, have been saying: the data tooling is in a miserable state and we had to glue together a bunch of different tools that don't work with each other.

At this point, I think it is quite obvious that Fivetran is going for Snowflake/Databricks's market share. They own the ingestion for many companies already, and they will offer a managed data lake product in order to compete with the data giants. By owning the means of bringing the data in (Fivetran) as well as the transformation layer (dbt/sqlmesh) they will aim to get ahead of Snowflakes of the world.

I think it'll be a win for the data community if they maintain and continue investing into the existing tooling, as they are running in quite a few places already, especially dbt core running in a self-managed way. I certainly hope they won't try to squeeze revenue for the sake of it from their combined users.

It's an interesting time to be in the space, and it feels great to be one of the few independent players in the market.

This seems really interesting, thanks for sharing.

I can see myself deploying something like this for some of our internal usecases, however the deployment options around such tooling seem to be rather complicated, especially when it comes to the need for RAG and stuff. Would I need to host something else on top of Upsonic for these sorts of usecases?