Comment by patio11 | Hacker News Reader

Comment by patio11 | Hacker News Reader

Parent Full thread

In 2013 when v0.8/v0.7 difference in database engines caused half of the network mine parallel blockchain due to obscure limit on number of file handles.

Can I just highlight, for generic web developers who might not parse this sentence, that this is like saying "One of the ways we make sure all clients agree on HTTP is by making sure that all clients and servers stay on the same point release of Oracle DB because when they didn't for a few minutes the entire Internet died in fire. We put the fire out by reverting the Oracle DB version on a few big sites, telling everyone else to downgrade as well, and posting on an out of band channel that if you were doing anything important with the Internet you might want to stop for a few hours because there were going to be two separate and very mutually incompatible Internets and one of them was going to suddenly vanish a couple hours later and it sure would suck if your mail/web browsing/Dropbox/etc went into that one."

Seriously. That actually happened.

That's because in HTTP, only the client and server involved in the transaction need to agree on what's going on. But in bitcoin, every client in the network needs to be able to verify your transaction, so they all need to agree on what transactions are valid.

That's not really what happened. The majority of Bitcoin clients need to agree whether or not to accept a block. In 0.7, a limitation of the database library was causing a very large block to be rejected, whereas in 0.8 the block was correctly accepted. This caused a blockchain split, with the 0.7 clients going one way, and the 0.8 clients going the other.

This sort of problem doesn't really occur on the web, so trying to draw an analogy to HTTP isn't going to help people understand the issue.

The point of the post is to make web developers say "But wait, Patrick, if HTTP actually worked like that that would be batshit insane. I mean, separation of concerns. Having to coordinate patch schedules worldwide across billions of clients. Being to-death-do-us-part committed to every library choice made by the first HTTP client. Wait, there could never be a second HTTP client, because you'd never be sure it worked warts-and-all like the first one. You'd probably have to do something like extract the One True HTTP Client into a wrapper program and, I don't know, call out to it from your web browser, so that changes in web browser chrome weren't tightly coupled to the One True HTTP client. Developers on the One True HTTP Client would probably refuse to help people who made competing HTTP clients, because they'd be building potential existential threats to the Internet, and they'd probably gloat when those other developers suffered misfortune.

Please tell me, Patrick, that you're exaggerating and that Bitcoin doesn't work like this."

Guess what?

HTTP isn't trying to maintain a consistent, decentralised database. Bitcoin is. HTTP doesn't have a problem maintaining data consistency because that's outside the scope of the protocol.

You're suggesting the Bitcoin protocol is badly designed because it's had problems that HTTP hasn't had. But this is like saying aeroplanes are poorly designed compared to cars because the latter don't fall out of the sky when their engines die. Planes have to deal with problems cars don't, not because of poor design, but because of the nature of what they're made to do.

Bitcoin concerns itself with providing data integrity across a distributed system. The only way of doing this, without a centralised server, is to ensure that all the clients are performing the same calculations. If it turns out a significant proportion of clients are performing the wrong calculation, then you have a problem. This isn't a problem inherent to Bitcoin; it's a problem with any distributed system trying to maintain a consistent database.

Patrick is a very smart guy, which is part of the reason I (like you) am so confused as to why he thinks HTTP makes for a good analogy.

I'm pretty sure that by HTTP client he meant rendering engines. And in this analogy the data integrity is "whatever crap I have to send to the client to get the expected output on all client machines". So it's somewhat fair to say that if you have wildly divergent interpretations of HTML rendering then you have a data integrity problem, since you can't guarantee that the same data will be interpreted the same way by all clients. None of that has to do with HTTP the protocol though... so I'm confused as to why HTTP the protocol was mentioned instead of HTML and rendering engines.

I can't tell if you meant for this statement to be as insulting to Patick as it seems at first glance. If you did, I will point out that an alternative outlook is to realize that as Patrick is a very smart guy you may have simply misunderstood, misinterpreted, or misapplied his analogy.

I wonder if anyone with some experience with bitcoind's source code could explain why the network layer and the data storage layer are not (or cannot be) sufficiently abstracted to prevent some low-level data storage implementation detail like this from affecting the correctness of the propagation algorithm's behavior. This seems like an architectural flaw, but maybe there's something inherent to the bitcoin protocol that makes this type of incident inevitable?

Early days of exciting new tech are early. And, yes, Blockchain compatibility requirement is a new phenomenon, unlike HTTP or even TCP/IP, so your comparison is not very relevant (although useful to highlight possible difficulties). We have to get used to it and learn how to deal with the risk of unintentional forks, or stay away from the game.

Deleted comment

With the exception of the integer overflow none of these problem seem directly related to the language being used (and integer overflows are a problem in many languages besides C).

"v1.0" is meaningless label. Bitcoin already works and cannot be changed willy-nilly in a glorious self-proclaimed version 1.0 of a single particular implementation.

bitcoind is C++.

Reply on news.ycombinator.com