HNHacker News
TopNewBestAskShowJobs

trentmc

238 karma · joined September 4, 2014

Founder @OceanProtocol. AI, blockchain, data, Token Engineering. http://trent.st
submissionscomments
trentmc··on BigchainDB – A scalable blockchain database
Hi, it's Trent here, CTO of BigchainDB.

The word "blockchain" is much misunderstood. There is a ton of argument over what it actually is. Just a linked list of hashes? Bitcoin and nothing else? I could go on and on.

To me, that debate is less interesting than building systems that actually work. For this, it's useful to think about compute stacks in the past, from mainframe to desktop, from web to cloud to mobile. In each, there are core building blocks that each have their own way of instantiating the elements of computing (storage, processing, communications). Yes, let's go back to first principles:)

Take cloud, on say AWS. Here are some blocks:

* Storage:blob storage -- S3

* Storage:database -- DynamoDB

* Processing -- EC2

* And so on.

The emerging decentralized stack [1] is no different. There is no single monolithic block called "blockchain" that magically does everything, though much of the rhetoric would have you believe that. Rather, there are emerging building blocks.

* Storage:blob storage -- IPFS + FileCoin (and more)

* Storage:database -- BigchainDB

* Storage:pure-play-token -- Bitcoin (this is specific to decentralized space)

* Processing:business logic (aka "smart contracts) -- Ethereum (and more)

"Blockchain" is best treated as a label for the space of decentralization. Side by side with other fields like "artificial intelligence" or "cloud computing".

I think we can all agree that DynamoDB is not "a cloud". It's just an implementation of the "database" building block for the "cloud computing" field. Similarly, BigchainDB is not "a blockchain". It's just an implementation of the "database" building block for the "blockchain" field.

[1] https://blog.bigchaindb.com/blockchain-infrastructure-landsc...

trentmc··on BigchainDB – A scalable blockchain database
Hi, it's Trent here, I'm CTO at BigchainDB.

BigchainDB targets giving the following benefits beyond traditional database as a service:

1. decentralized - no single entity controls it, which means tolerant to malicious / Byzantine faults. Benefit: groups that don't necessarily trust each other can share infrastructure.

2. immutable - which practically speaking means more tamper resistant, e.g. it's append only. Benefit: well-defined provenance for history of assets, data, etc.

3. assets - you can create and issue assets, where you own them if you have the private key. How: each tx is signed. Benefit: moving around assets on a substrate that no single entity owns or controls. Lower friction in exchanges. And the digital signatures gives cryptographic proof that about who did what.

These are the targets. We're not fully there yet. Most notably, we still need to address some Byzantine faults as our docs ([1] above) mention. This will come in an upcoming release. We are also working on improved scalability while maintaining the security guarantees [2].

Re consensus: BigchainDB has a two-layer consensus, as follows.

* The lower layer directly uses MongoDB's consensus (which it builds on) to agree on whether a transaction should be stored.

* The higher level has federation-style voting on whether a transaction is valid or not.

Our documentation describes this further.

BigchainDB is explicitly not trying to do Bitcoin-style proof of work. PoW solves for an additional problem: (theoretical) anonymity of servers. That additional goal compromises scale. And, in practice you know who's running the servers anyway (ie big Chinese Bitcoin miners), which is why I say "theoretical". BTW I am a fan of Bitcoin, it just has different goals than BigchainDB.

[2] https://blog.bigchaindb.com/bigchaindb-developer-update-2d32...

trentmc··on Ask HN: Who is hiring? (October 2017)
BigchainDB | Python - Decentralized Software Engineer| Berlin, Germany | Full-time ONSITE | http://www.bigchaindb.com

BigchainDB is a leader in the blockchain space. We're building a decentralized database, and a data marketplace for the internet. Organizations use us to build applications that liberate personal data from centralized silos, that help artists & musicians feed their families, and to power their applications in IoT, supply chains, and AI.

If you are someone that loves to build things and cares deeply about the future of the internet, we want to hear from you. You need to have production experience in Python, and at least deep interest in blockchain or distributed systems.

You will research, design, and implement decentralized database software & networks (BigchainDB & IPDB), and data marketplaces (Ocean Protocol).

The position is in Berlin -- Europe's most vibrant startup city. You don't need to speak German. Vacation leave, parental leave, and health benefits are excellent. We supply a budget for personal growth; and trust us, you'll be growing in this fast-moving blockchain industry:). We sponsor VISAs.

Jobs page: https://github.com/bigchaindb/org/blob/master/jobs/engjob.md

Are you up for working with an awesome team, learning a ton, and shipping code that matters? Please email engjob@bigchaindb.com, with your cover letter, resume, and code samples (e.g. GitHub profile).

trentmc··on Ask HN: Basically and for dummies, what to do with blockchain?
BigchainDB is open-source; with same licensing as MongoDB which it wraps. http://github.com/bigchaindb/bigchaindb.

It has quickstart that takes about 20 min to do. By the time you're done, you'll have issued assets, transferred them, and more. https://docs.bigchaindb.com/projects/server/en/latest/quicks...

Live support is at gitter. https://gitter.im/bigchaindb/bigchaindb

(Note: I'm CTO of BigchainDB.)

trentmc··on Ask HN: Basically and for dummies, what to do with blockchain?
Hi! I've been working on blockchain tech professionally since mid 2013, and been writing software since the 80s. I'm CTO of BigchainDB.

Here's how I see things. Blockchains are distributed databases (e.g. MongoDB) with three new characteristics: decentralized (no single entity owns or controls), immutable (tamper-resistant), and assets (you own the asset if you have the private key).

Each new characteristic of blockchain tech leads to business benefits. "Decentralized" helps organizations share resources; e.g. music labels sharing a database of who-wrote-what. "Immutable" gives better audit trails; e.g. to see history of ownership of art. "Assets" can now live on the data store itself, which enables decentralized exchanges and more.

Details: https://blog.bigchaindb.com/three-blockchain-benefits-ae3a2a...

Let's use this as a basis to answer the question...

You can use blockchain tech in your existing centralized stacks. E.g. it would run side-by-side with your instances of Postgres, MongoDB, etc. Typically the blockchain would incorporated as a database-as-a-service. This would make the stack partly-decentralized.

Or you can go for a fully-decentralized stack. You'd have decentralized file systems (e.g. IPFS), decentralized processing aka "smart contracts" (e.g. Ethereum), and decentralized database (e.g. BigchainDB).

trentmc··on Blockchains for Artificial Intelligence
Articles like the "AI + Blockchain" one are meant to be forward-looking, to help inspire people to build. Call it hand-wavy if you wish; I call it laying out a vision:)

But we're all about real apps getting built. Many comanies are building on BigchainDB. For example, I describe some of them here: https://blog.bigchaindb.com/where-does-blockchain-scalabilit....

There are many, many more companies that have not publicly announced what they're building yet. Stay tuned:)

trentmc··on Blockchains for Artificial Intelligence
To me, incentives can be intrinsic (built into the system) or extrinsic (outside the system). Each has pros and cons. Bitcoin is intrinsic, based on mining reward and tx fees. Traditional private blockchains are extrinsic, where the folks running the network are incentivized to pay for hardware based on cost savings, higher profits, etc, ie something extrinsic to the network itself. IPDB is also extrinsic where the folks running the network participate because they care about the future decentralized internet, and their costs of running a node are covered by SaaS-style transaction fees.
trentmc··on Blockchains for Artificial Intelligence
> You're the one who asserted that CAP-style consistency is required to prevent double-spends

Could you point me to where? I like my thinking and expression to be consistent (pun intended:)

> I asked how the nodes federate .. who controls identity?

Each node has a list of the public keys of other nodes. There are various ways to handle key distribution, of course.

trentmc··on Blockchains for Artificial Intelligence
> have something close to a consensus mechanism

For this context it's really about whether or not it can prevent double spends. Currently, it can't. Though its protocol stack has a place for consensus algorithms to achieve CAP-style strong consistency; and the IPFS team is working on consistency algorithms.

> It arguably is when ...

Point taken. The post is about things that many people consider to be blockchains or at least blockchain-like; BigchainDB, Ethereum, and others are in that category.

trentmc··on Blockchains for Artificial Intelligence
To clarify: "distributed" and "decentralized" are different. "Distributed" spreads compute among many physical resources, but may still have a single entity controlling it. "Decentralized" has no single entity controlling it. Paxos distributes but does not decentralize.
trentmc··on Blockchains for Artificial Intelligence
Hi, this is the author:)

I've been using Python in enterprise-grade, large-scale production systems since the early 2000s. And it works, well. Most python libraries where performance matters have C under the hood. E.g. numpy matrix manipulation. Parallel computing got way better with the async support.

As for BigchainDB itself: it wraps RethinkDB (and soon MongoDB) which is where much of the compute intensive stuff happens. Also, we leverage these fast Python+C libraries, and parallelization. If/when needed, we can always convert some of our core stuff to C or another fast language.

trentmc··on Blockchains for Artificial Intelligence
We acknowledge this attack vector and others; and list them in github. It's in our roadmap to address them. The current schedule is the following: we're currently wrapping up the a base MongoDB integration (next couple weeks); on the heels of that we'll improve the MongoDB replication (a couple months); and on the heels of that we'll address the "drop" issue (and other unwanted commands) via a wire protocol firewall. More security stuff to follow that.

BTW thanks for the kind words about our team. I think they're amazing too:)

trentmc··on Blockchains for Artificial Intelligence
Great question. As the article says: there are many ways that AI can help blockchains, such as mining blockchain data (e.g. Silk Road investigation). That’s for another discussion:)

I'd love to see more suggestions here or elsewhere. It would be my pleasure to write an article on this.

trentmc··on Blockchains for Artificial Intelligence
I'm looking forward to seeing these. It's gonna be fun:)
trentmc··on Blockchains for Artificial Intelligence
IPFS is very close to what you describe (Merkle trees! Globally-identified hashes!) but it doesn't claim to be a blockchain because it doesn't have a consensus mechanism, let alone a way to prevent double spends.

And that's ok! It's designed for different problems, like file system style blob storage. It's not about "whether something is a blockchain", it's about "what's the right tool for the job"? I see value in complementary decentralized pieces of file systems, databases, processing, and more.

trentmc··on Blockchains for Artificial Intelligence
As the article discusses, it's not an either-or. Blockchains are simply databases with specific properties; if some of those properties help your app, consider using one. They are sometimes deployed to the public, sometimes private, with appropriate models. Something like Tay would be higher in the stack - we're gonna see a lot more, some good, some bad, some ugly.
trentmc··on Blockchains for Artificial Intelligence
Oh cool! Thanks for the link. I just added it to the "further reading" section on the bottom.
trentmc··on Blockchains for Artificial Intelligence
Thanks for the thoughts. I'll respond to each point in turn.

> I don't understand the assertion made regarding blockchains/digital signatures supposedly increasing accuracy or trust in IoT sensor-reading reliability.

If each IoT sensor has a known public key then we can know that the data came from that sensor.

It doesn't solve all reliability problems of course. E.g. if you have a hardware failure, a digital signature ain't gonna help:)

> Also, how do you respond to the inevitable criticism that advertising your solution as a "shared global registry" invokes the "How Standards Proliferate" XKCD[1]?

I get this. What's cool is that IPDB doesn't need to be (and shouldn't be) the "one network to rule them all. The path is to leverage other existing standards and new connectivity standards, so that IPDB (or whatever registry) plays well with other emerging & existing registries.

The main lesson is from the internet itself, which was born by combining together disparate networks (ARPANET, NSFNet, etc) that didn't communicate previously, via the invention of TCP/IP. All the networks had to do was alter their top-level protocol and suddenly they could talk to other nets. There's a similar protocol for blockchain tech, called Interledger (https://www.interledger.org). Whereas TCP/IP connected networks with data, since you can re-send packets etc it doesn't account for double-spending; Interledger does this. You can view it as a way to connect networks of value.

Interledger has already been used to connect Bitcoin, Ethereum, and many centralized payment networks.

There are other protocols that are closer to the data level. E.g. simply using JSON-LD helps. And ILPD on top of that (roughly, a Merkle-ized JSON-LD). And domain-specific value transfer protocols above that, like COALA IP for intellectual property (which plays well with existing protocols like DDEX for music, PLUS for photography).

> Finally, I like the idea of creating a marketplace or exchange for AI training information, but it's strictly not necessary to use a blockchain for this purpose

Agreed; and I wrote about it: "An exchange could be centralized, like DatastreamX already does for data. But so far, they are really only able to use publicly-available data sources, since many businesses see more risk than reward from sharing. What about a decentralized data & model exchange? ..." and I went on to list some benefits.

> unless a very large portion of your users are hyper-concerned about their anonymity.

To me, there are much greater benefits. I'm hoping that the biggest benefit will be to further catalyze a truly open data market.

> It's also misleading to try to build a marketplace of signed IoT data, with the assumption that it is more trustworthy than unsigned IoT data, when that is simply false.

I'm not sure why you say this; knowing the provenance of the data has clear trust-related benefits in some cases like I described. Obviously just standing on its own, it has no benefit, it's all about usage farther along in the pipeline.

trentmc··on Blockchains for Artificial Intelligence
You're absolutely right - there is a lot of room for improvement. One of the big research threads in ML is better unsupervised learning - to learn from data without labels. Another hot topic is adversarial networks, which generate their own training data by playing against each other. (15 years ago this was "competitive co-evolution";)

> If the key to making ML useful has been huge sets of training examples

BTW this is only for some ML approaches and problems. There are some problems where getting more data is just way too expensive; e.g. building a model from silicon manufacturing where each mfg. run costs $50M; i.e. each datapoint costs $50M. Guess what - there are still useful ML-y things you can do here. Less extreme, there are many problems with only 100-1000 training points available, and you can still do a lot.

trentmc··on Blockchains for Artificial Intelligence
> increasingly a literally meaningless marketing buzzword Agree, "blockchain" is getting more diluted.

To me, it still has enough information content to be useful. Just because different people have different definitions doesn't mean the information content is zero. People may agree on the broad ideas and disagree about the specifics. And as discussed in another comment, definitions are nonetheless useful to help summarize a set of properties, help communication, and more.

For example, I think most people would agree a blockchain needs to have the property of having a consensus mechanism. Versus say a file system which typically doesn't.

trentmc··on Blockchains for Artificial Intelligence
Ditto on agreement about what properties (including guarantees) a system provides.

To me, definitions are nonetheless useful to help summarize a set of properties, help communication, and more. Even if people have different definitions, the definitions typically have similar themes; they're not totally arbitrary. For example, with "decentralization" you'd find a lot of people agreeing that "no single entity owns or controls" within the same theme as other definitions.

trentmc··on Blockchains for Artificial Intelligence
> Care to link to sources for server-based and server-free decentralisation that aren't your own blog posts?

The best precedent is simply the long-standing difference between servers and clients in computing systems.

In blockchain discourse, this difference had not been acknowledged as much; though of course a similar pattern exists between full nodes vs light/SPV clients.

To my knowledge, no one else had made the clarification of "server-free" vs "server-less" as different types of decentralization. It is a useful distinction as the article discusses.

> Also, that blog post you linked is flat out wrong -- Bitcoin, Ethereum, et al are all instances of eventually consistent systems

As the article states, they can and do prevent double spends, so we agree there. But that's not what "consistent" means in a CAP setting. As the article states, "they they never have a deterministic guarantee of a consistent order; they’re only eventually consistent (in a probabilistic sense). But let’s be generous and call them consistent, because in practice they are used that way, the workaround being higher latency as one waits for a sufficiently high probability of avoiding inconsistency."

> how do your nodes federate without a central authority?

Each node votes on any transaction coming through. The transaction only clears if it gets enough positive votes.

> just adapt a run of the mill distributed log-based db instead ...

Well in some cases that's all that people actually need; sometimes find myself referring people to Kafka and the like.

But Kafka and the like are still controlled by a single admin; you can do more to decentralize. As for immutability, it's all shades of grey, and certainly being a log-based db (read-only) helps a lot. You can do more with Merkle DAGs, continuous backup to write-only media, etc.

To me it's not about "eroding" guarantees. It's about saying "ok, I have this database, what properties do I want?" The potential results might be blockchain-like or not. If decentralization, immutability, or assets are potentially interesting, then a blockchain technology could be interesting. Otherwise it comes down to other questions to choose among traditional DBs.

trentmc··on Blockchains for Artificial Intelligence
I define "decentralized" as "no single entity owns or controls". It can be further distinguished with "server-based decentralization" and "server-free decentralization" [1].

You only need to be Sybil tolerant if you want your validating nodes to be anonymous. There's certainly some applications where that's useful. But it's not a requirement for being decentralized. (Some will argue otherwise, and that's ok; once again it depends how you define "decentralized"; my approach is about what benefits come to the application.)

[1] https://blog.bigchaindb.com/the-dcs-triangle-5ce0e9e0f1dc

trentmc··on Blockchains for Artificial Intelligence
Hi, it's Trent here, I'm CTO of BigchainDB and also the author of the blockchains-AI article.

Yep, it's a [database] cluster, and proud to be that:)

But, it's special in that there isn't a single organization that owns or controls it. That is, it's decentralized.

And in fact it draws on traditional distributed DBs by building on them, to get first-class scalability and querying abilities. The first version was built on RethinkDB and we're working on wrapping MongoDB too.

There's also a public version of BigchainDB that's getting rolled out, called IPDB (http://ipdb.foundation). With IPDB, people building apps don't need to roll their own cluster, they can just talk to it via http.

> was disappointing

How? We're always looking for feedback.

trentmc··on Blockchains for Artificial Intelligence
Throw a rock in the blockchain space and you'll find a different definition of blockchains, or what it means for one thing to be a blockchain or not. It doesn't really help anyone to argue for hours on end over this, when it's largely a matter of opinion. To me, what's more interesting is to identify what are the new characteristics (in terms of benefits) that blockchains have, above and beyond traditional distributed databases; which in turn unlock new applications or improve existing applications. To me, those characteristics are: decentralized, immutable, assets; I describe them in the article, and also in more detail in https://bigchaindb.com/whitepaper.
trentmc··on Blockchains for Artificial Intelligence
It's the author here. (Hi!)

I agree, for the use cases that involve sharing, how do you incentivize this when many see data as the moat? This is why I wrote the section about "Hoard vs. share?". In some applications, there will be more incentives to hoard; in others, to share.

Note that one thing that softens this impasse is the idea of data exchanges. So rather than "hoard vs share" you can think of all data as potentially "for sale, but with a price" and where the price could vary dramatically. And that's where a data exchange could come in to reduce friction for price discovery.

trentmc··on Blockchains for Artificial Intelligence
Hi, it's the author here.

> This really doesn't make any sense.

Disagree. Below, I respond to each of your points.

> Blockchains aren't immutable, they are just expensive to mutate.

Agreed; very little is truly absolutely immutable. It's all shades of grey. I actually prefer the word "tamper-resistant" and I usually say that next to the "immutable" definition, such as in the first paragraph of https://bigchaindb.com/whitepaper. But "immutable" makes for a good shorthand, especially because that's the label that the community uses.

> Blockchains aren't centralized.

Oops, that was a typo. I meant to say "decentralized". Fixed it. (That was a pretty big oops!)

> Blockchains didn't introduce audit trails ...

Correct.

> ... [status quo] require[s] trust in the central authority

Exactly. And it's crucial to note that once you don't have to trust a central authority to store your audit trail, you have a way more trustworthy audit trail that improves many applications and unlocks new ones.

> The author has a poor understanding of both AI and blockchain technology ... "treatise on altcoins"

Disagree. First, you don't need an "altcoin" to have a blockchain. To understand of what's special about blockchains, you first have to understand what already exists for distributed databases, then what the delta is.

Second, consensus in a distributed database has been around for decades; Satoshi did not invent it. Lamport laid down much of the theory for FT and BFT consensus in 1982. What Bitcoin brought forward, in addition to BFT-ish consensus, was Sybil tolerance (addressing attack-of-the-clones).

Third, as for my understanding of AI: I've been doing it professionally since the late 90s; here are my publications: http://trent.st/publications. Doing AI in the 90s was one of the least popular things one could possibly do, so I certainly didn't do it for the hype.

> vaguely proposing using a blockchain as a mass datastore (with ownership labels)

Obviously I gave much more specific proposals than that. I have other writings that dive into more detail on some of the use cases, such as an IP registry [1] and for AI DAOs [2].

[1] https://medium.com/ipdb-blog/a-decentralized-content-registr... [2] https://medium.com/@trentmc0/ai-daos-and-three-paths-to-get-...

trentmc··on From Web 2.0 to Web 2016: The Need for Public Platforms for Digital Ownership
Here's a whitepaper from early 2015 that explores the challenges faced by digital artists and creators of the Internet, and points to a solution that is a combination of legals and technology.

https://www.ascribe.io/app/editions/1iYjGaiKPSm3uw3bqZoTWBEk...

Abstract: One long-standing drawback to the Internet is the hidden ‘artist penalty.’ The very strengths of the Internet make it difficult for creators of digital content to be fairly compensated for their work. This paper describes our implementation and proposes a fix that makes it easy to control one’s intellectual property by constituting a new “ownership layer” on top of the existing Internet. The approach has two pieces: a registry with easy, secure legals; and visibility into usage / provenance of the content. The legals formalize existing copyright rights of digital objects, making them easy and fluid for a creator or collector to use, transfer or modify. The bitcoin blockchain is used to securely record ownership transactions that are impossible to later repudiate or manipulate. Internet-scale media search provides visibility into usage of the media by crawling the web, applying machine learning to identify similar or identical media, and subsequently reporting their existence and location to the registered owners. Taken together, these pieces constitute “ownership processing” – a simple tractable solution to make ownership actions of digital property universally accessible.

trentmc··on Show HN: WhereOnTheNet – Trace how images spread on the Internet over time
Yes, in the bottom image, if you hover above the leaf nodes (farthest from the center of circle), you will see the link and can simply click on it.
trentmc··on N3uro: A marketplace for brainwaves of people thinking about things
Dude. This is art. No more, no less.
← PreviousPage 2 of 3Next →