Would be interesting to see a breakdown of the differences.
Would be interesting to see a breakdown of the differences.
https://www.philipotoole.com/9-years-of-open-source-database...
https://www.philipotoole.com/replicating-sqlite-using-raft-c...
https://www.philipotoole.com/rqlite-v3-0-1-globally-replicat...
The post makes a fairly big point of the shortcomings of statement-based replication (like what rqlite does).
What they state is correct -- non-deterministic SQL will be a problem, but it's not a difficult problem to solve. Just parse and rewrite the SQL statement before sending it to Raft. It's exactly what rqlite does[2] (though there is more to do). However they claim that there are two concerns with this approach.
>This solution introduces a runtime delay because each SQL statement must be parsed and analyzed.
The delay is insignificant compared with the delay introduced by Raft consensus. This is not a valid concern. I'm surprised they even mentioned it.
>More importantly, it increases the difficulty of staying up-to-date with the new versions of the underline SQL engine (SQLite in this specific case) because the custom parser must be updated each time a new SQL function/feature is introduced.
Theoretically somewhat true, but not true in practice. Any competent programmer can update any parsing code in a matter of hours. Also just because a new feature is introduced does not mean that the parser needs to be rewritten. That's because SQLite will not introduce breaking changes into their SQL specification. Any parser needs to be updated only if you want to support any new, non-deterministic feature.
Is that supposed to still apply when a small cluster (eg three nodes) is reduced to a single node? eg during node failures, maintenance, node failures during maintenance ;), etc.
When I was trying rqlite last week - eg prototyping a new SQLite "cloud" offering for DBHub.io - a remaining single node would consistently refuse to elect itself as the leader. Using default settings for everything.
Tried it out several times, though I didn't give it more than a minute or two to get itself happy.
Anyway, had to go with an initial non-RAFT approach for the prototype, but will likely investigate rqlite again in the future if I was just doing something dumb / fixable with tweaking. :)
However if you create a 3-node cluster (or add 2 nodes to an existing single-node cluster) then at least a quorum of nodes (N/2 +1) must be up and running for the cluster to be available. That's the way Raft works -- Cockroach, etcd, Consul -- all of them will work the same way.
So in your case a 3-node cluster with 2 nodes subsequently going down (or deliberately taken offline) is a cluster which is offline.
There is extensive documentation available on clustering, which you should review:
It's just it wasn't clear from the initial docs I saw that a 3 -> 1 cluster would be considered any differently. Clearly, an error on my part. :)
I'll circle back to rqlite in a while, after I get this existing prototype code rolled out and see what people make of it. Hopefully it turns out to be useful for people.
https://rqlite.io/docs/clustering/general-guidelines/#recove...
While this talks about "dealing with failure" using the config file described above basically allows you to force the cluster into any configuration you want (including a single-node cluster).
https://rqlite.io/docs/faq/#how-can-i-change-my-multi-node-c...