HNHacker News
TopNewBestAskShowJobs

timsehn

350 karma · joined February 15, 2019

CEO of DoltHub
submissionscomments
timsehn··on Dolt is Git for data
Dat is more of a distributed data sharing protocol. It does not do branch/merge which Dolt does.
timsehn··on Dolt is Git for data
Here is an earlier blog we published on comparison's to Pachyderm: https://www.dolthub.com/blog/2020-03-06-so-you-want-git-for-...

We got a blog on the storage system coming on Wednesday. It's a mashup of a Merkle DAG and a B-tree called a Prolly Tree. It comes from an open source package called Noms (https://github.com/attic-labs/noms).

I'm not familiar with CRDT. Will read up on that.

timsehn··on A group of ex-NSA and Amazon engineers are building a ‘GitHub for data’
It still cool when people know we exist. Thanks for the shout out.

Git for data to us means a SQL database with Git semantics, backed by a content addressed graph of rows. That's what we built.

Plus, we own http://www.gitfordata.com, so we win.

timsehn··on Mapping Income Inequality in the US Using IRS SOI Data
With this dataset, you can do Zip, not county :-)
timsehn··on Mapping Income Inequality in the US Using IRS SOI Data
The dataset used to create this map:

https://www.dolthub.com/repositories/Liquidata/irs-soi/

Has tax data aggregated by zip. You can get the median zip code in a state from it because it only has totals so you can only calculate averages otherwise.

timsehn··on Show HN: Version Control for Databases
We built a Git-style version controlled SQL database called Dolt (https://github.com/liquidata-inc/dolt) mentioned deep in this thread. It allows diffs and merges across schema and data based on commits.

We are focused on the data distribution/open data use case (https://www.liquidata.co/blog/2019-10-09-where-is-the-data-c..., https://www.liquidata.co/blog/2019-12-06-the-history-of-data...) but I think a number of use cases mentioned in this thread are covered by Dolt.

You do have to move a copy of your data to Dolt, it does not sit on top or beside of your current database. So, it is an extra step in your data modification workflow, similar to versioning code. You had to add a version control step into your code modification workflow.

timsehn··on Launch HN: Quilt (YC W16) – A versioned data portal for S3
The coolest thing to do is diff between the V2 and V3 branch for a label_descriptions table.

https://www.dolthub.com/repositories/Liquidata/open-images/c...

You can start to see the power of column-wise diffs. You can start to imagine what it would be like to change this table and then merge Google's changes in V3 onto your modified copy. Very powerful. We need a query interface on top of diffs. Lots to build...

timsehn··on Launch HN: Quilt (YC W16) – A versioned data portal for S3
Congratulations to the Quilt team on the launch!

Quilt reached out to me and suggested I chime in suggesting that people interested in versioning data also check out Dolt (https://github.com/liquidata-inc/dolt) and DoltHub (https://www.dolthub.com).

We've taken the Git and GitHub for data analogy a lot more literally than Quilt has :-) We are a SQL database with native Git semantics. Instead of versioning files like Git, we version table rows. This allows for diff and conflict detection down to the cell level. We are built on top of another open source project called Noms (https://github.com/attic-labs/noms).

We think there is a ton of room in this space for a bunch of tools: Quilt, Noms, QRI (https://qri.io/), Pachyderm (https://www.pachyderm.io/), and even Git. We're excited to see so many bright minds trying to solve this problem.

We're going to be populating DoltHub with a bunch of datasets we harvest from the open data community to show off the capabilities of Dolt. The coolest one so far is the Google open images dataset: https://www.dolthub.com/repositories/Liquidata/open-images.

timsehn··on Dolt. Git for data. Announcing private beta
Thanks for all the interest!
← PreviousPage 3 of 3