5,215 karma · joined April 3, 2014
https://www.philipotoole.com
[1] https://www.rqlite.io
rqlite will give you HA, but won't help with load (rqlite replicates for reliability, not for write performance). But I like to think the it'll give you solid -- and simple to use -- HA. Replicated just replaced their use of Postgres with rqlite for this reason[2].
[1] https://github.com/rqlite/rqlite
[2] https://www.replicated.com/blog/app-manager-with-rqlite/
For the system you describe to have the practical challenges of a distributed system it would also need experience some of the following:
- some of the "different processors" would need to fail at seemingly random times, only to come back again in a few minutes.
- the speed at which those "different processors" respond would need to vary widely, and different times of the day.
https://www.philipotoole.com/a-prayer-for-distributed-system...
"The idea of monitoring the responsiveness of a read-modify-write cycle is obviously nonsense from a programmer’s point of view (though not so for the microprocessor designer!)."
[1] https://www.philipotoole.com/rqlite-7-7-0-released/
[2] https://github.com/rqlite/rqlite/blob/master/DOC/NON_DETERMI...
That said, there are some things that would be much easier to do with some changes to the SQLite source. But I think the message that rqlite sits on top of pure SQLite makes is still the right choice.
I can make that clearer.
So projects like mvsqlite are solving a real need. I just disagree with some of its definitions.
There are a couple of statements on that page I disagree with.
"mvsqlite is a distributed database, while dqlite and rqlite are replicated databases"
I disagree with this statement, and consider its definition of "distributed system" to be incorrect. rqlite[1] is a distributed database. A "distributed system" is simply a system that splits a problem over multiple machines, solving it in a way that is better, more efficient, possible etc than a single machine. rqlite uses distribution to provide fault-tolerance and high-availability. It uses distributed systems technology i.e. Raft, to make rqlite appear up-and-running, even in the face of node failures. That it replicates a full copy of the SQLite database to every node is correct, but that doesn't mean it's not a distributed system. Is Consul a distributed key-value store? etcd? By the definition quoted on that page they are not, but no one would actually agree with that.
rqlite is not just about replicating a SQLite database. I understand what the page is trying to say, but the point is that rqlite is distributed, just for fault tolerance and high-availability.[2]
"(+): mvsqlite runs on a production-grade distributed key-value store, FoundationDB, instead of implementing its own consensus subsystem."
rqlite doesn't implement its own consensus system either. It uses the same Raft consensus code that powers Hashicorp Consul. I don't see how this is any different, in principle, than using Foundation DB's consensus system.
[1] https://github.com/rqlite/rqlite
[2] https://github.com/rqlite/rqlite/blob/master/DOC/FAQ.md#rqli...
So it's actually pretty simple for a given node to contact the Leader. If a node receives a request which must be performed on the Leader, and that node is not itself the Leader, it can do one of the following things:
1) reject the request with an error, but this isn't really a production-viable option.
2) reject the request with an error, but tell the client where the Leader can be found, so the client can retry the request, this time sending the request to the Leader.
3) transparently forward the request to the leader, wait for the Leader to execute the request, get the response, and return the response to the client. In this case the client doesn't even know the forwarding to the Leader happened.
rqlite supports mode 2 and 3, client can choose which behavior it wants, on a request-by-request basis. Option 3 is the default.
https://github.com/rqlite/rqlite/blob/master/DOC/DATA_API.md...
https://github.com/otoolep/hraftd
This is kind of a reference use of Hashicorp's Raft.
Architecturally (in rqlite's case) a node looks something like this: https://docs.google.com/presentation/d/1Q8lQgCaODlecHa2hS-Oe...
[1] https://github.com/jackc/pgproto3
[2] https://github.com/rqlite/rqlite
Disclaimer: I am the creator of rqlite.