ngaut

Cool, it uses a distributed database TiDB as the meta data storage.

See Kyle's report here:https://jepsen.io/analyses/tidb-2.1.7?from=timeline&isappins...

slides: https://gitlab.com/frondeus/meetup-2019.02/blob/master/Meetu...

The project URL:https://github.com/UWSysLab/tapir

Prototype integration with Spark on a 25-node cluster shows that Decima outperforms several heuristics, including hand-tuned ones, by at least 21%. Further experiments with an industrial production workload trace demonstrate that Decima delivers up to a 17% reduction in average job completion time and scales to large clusters.

Features

Compatible with redis protocol

Full distributed transaction support

Strong consistency

High availability

Here is a blog post: https://blog.steve.fi/so_i_wrote_a_basic_basic.html

Paper: https://www.usenix.org/conference/usenixsecurity18/presentat...

Slides: https://www.usenix.org/sites/default/files/conference/protec...

Data is generated by gopherstats(https://godoc.org/golang.org/x/build/cmd/gopherstats)

The paper link: https://www.sigops.org/src/srcsosp2017/sosp17src-final23.pdf

This paper implements a subset of the SQLite virtual machine directly on the GPU, accelerating SQL queries by executing in parallel on GPU hardware.

This paper focuses on accelerating SELECT queries and describes the considerations in an efficient GPU implementation of the SQLite command processor.

Results on an NVIDIA Tesla C1060 achieve speedups of 20-70x depending on the size of the result set.

Features:

Small

Support Tracing-JIT compiling

Thanks for providing more details.

Features

Source is a wasm32 binary module. The application embedding the compiler decides what import functions it provides.

Supports x86-64. Support for 64-bit ARM is planned. (Support for non-64-bit or non-little-endian CPU architectures isn't planned.)

Single-pass, low-latency ahead-of-time compilation. Early functions can be executed while the latter functions are still being compiled, even while the source is still being downloaded.

Generated code requires minimal runtime support. It may be run e.g. in a strict seccomp sandbox. Note: calling standard library ABIs is not supported.

Supports snapshot-and-restore across compiler versions and CPU architectures.

Rusty rail is a line rate distributed load balancer It is inspired by the Google Maglev paper(http://research.google.com/pubs/pub44824.html). Here is the video talk: https://www.youtube.com/watch?v=_GRM1Ij_3t0

The source code is available on GitHub: https://github.com/AlexKashuba/SQLite_JIT

"We create a set of automated tools to create a runtime code generation engine and integrate such an engine into SQLite which is one of the most popular relational databases in the world and is used in a large variety of contexts. Speedups of up to 1.7x were observed in microbenchmarks with queries involving a large number of operations"

Thanks, done.

Supported Sources & Sinks: Amazon (S3, SQS), Elasticsearch (output only), File, HTTP(S), Kafka, MQTT, Nanomsg, NATS, NATS Streaming, NSQ, RabbitMQ (AMQP 0.91), Redis, Stdin/Stdout, Websocket, ZMQ4.

And distributed database: https://github.com/pingcap/tidb

Features: Compliance to Apache TinkerPop 3, supporting Gremlin Schema Metadata Management, including VertexLabel, EdgeLabel, PropertyKey and IndexLabel Multi-type Indexes, supporting exact query, range query and complex conditons combination query Plug-in Backend Store Driver Framework, supporting RocksDB, Cassandra, ScyllaDB and MySQL now and easy to add other backend store driver if needed Integration with Hadoop/Spark

TiDB is a distributed HTAP database compatible with the MySQL protocol(https://github.com/pingcap/tidb)

Nice reference, thanks for sharing.

Almost there.

If the clock offset is too high (more than 250ms), we should use another transaction model, Google Percolator is a good fit before the unforeseeable improvement in the hardware. Based on the monitoring of clock offset on cloud, TiDB chose to use the timestamp oracle to allocate timestamp, which is much faster.

RocksDB is widely used in many of projects for years, i think it's quite mature. Not just TiDB and CockroachDB, MyRocks and MongoRocks are using RocksDB as storage engine too. Also there are plenty of other projects are using RocksDB. If you are worry about the adding new feature of RocksDB, yes, you are right, but all of the storage engine are changing rapidly(even InnoDB which is used by MySQL as default engine), Anyway, happy hacking :)

Thank you,We are still working on the next RC release.

Yes, it does. Thanks to MVCC, TiDB supports the repeatable read isolation level which guarantees that any data read cannot change, if the transaction reads the same data again, it will find the previously read data in place, unchanged, and available to read.

So we can use any MySQL tools such as mysqldump or mydumper to backup the database consistently.

See the MVCC implementation here: https://pingcap.github.io/blog/2016/10/17/how-we-build-tidb/

Hi, TiDB maps SQL table to K/V table and store the SQL table to K/V store(TiDB store these K/V to TiKV), Please see https://pingcap.github.io/blog/2016/10/17/how-we-build-tidb/

(TiDB developer here)

Actually we are using RPC to call Rust from go, and it has nothing to do with Go or Rust, because either way we have to do RPC call(by encoding and decoding data), currently we depend on protocol buffer and customized RPC , and we are trying to migrate to gPRC. We know Go doesn't have pattern matching and generics, and it's kind of pain sometimes. But still we like Go very much because of simplicity and concurrency.