HNHacker News
TopNewBestAskShowJobs

ideal0227

830 karma · joined April 2, 2013

submissionscomments
ideal0227··on Bizur: A Key-Value Consensus Algorithm for Scalable File-Systems
This (like a sharded Raft group) is somewhat correct :P.

If you can sacrifice consistency (causal instead of external), you can get EPaxos. If you can sacrifice more, you can get Single Paxos. Both of them can increase concurrency to some degree, and can improve availability over a leader based RSM system like Muti-paxos or Raft.

I believe there are other ways to make the trade-off, even making the trade-off dynamically. Hopefully, I can talk about it more soon.

I work on etcd/raft, which is a shared Raft implementation by TiDB(TiKV), Cockroachdb, etcd and many more.

ideal0227··on Bizur: A Key-Value Consensus Algorithm for Scalable File-Systems
v3 does not have a native HTTP API. I suspect you still use v2 API. You need the gateway to make v3 works with HTTP: https://github.com/coreos/etcd/blob/master/Documentation/dev...

But that might impact performance quite bit depending on your workload.

ideal0227··on Bizur: A Key-Value Consensus Algorithm for Scalable File-Systems
etcd v3 read is linearizable by default.

Also I believe the authors of the paper tested against the old etcd v2 API/backend although they use etcd3.

The new API/backend has significant high performance.

I work on etcd.

ideal0227··on Announcing etcd 3.1
etcd3 provides a HTTP+JSON API as a gateway: https://github.com/coreos/etcd/blob/master/Documentation/dev....

There are users we know successfully using it. If users are unhappy about the HTTP API, or it is too hard to use, we might consider to provide a native HTTP API.

ideal0227··on Announcing etcd 3.1
> the "ease" of etcd seems to have gone

Can you explain this more? One of the goals of the new API is to make etcd easier to interact with.

Can you provide a few examples where v3 is more difficult than v2 to work with? (etcd-dev or github might be a better place to discuss though) We would love to hear and improve.

ideal0227··on The etcd operator: Simplify etcd cluster configuration and management
Yea. They are similar in functionality.

But they work differently. The operator does not really “schedule” containers. It finishes the controlling logic by using Kubernetes APIs. For example, it uses native Kubernetes health checking, service discovery, deployment. It works completely on top of Kubernetes API, so no specialized scheduler, executor or proxy are needed comparing to https://github.com/mesosphere/etcd-mesos/blob/master/docs/ar....

The advantages of Mesos is exposing lower level APIs and resources to allow more control. The etcd operator we built does not really need that. Building this kind of application operator may be simpler on k8s than on native Mesos.

Disclaimer: I work at CoreOS on Kubernetes and etcd.

ideal0227··on The etcd operator: Simplify etcd cluster configuration and management
The "Chaos Monkey" lives inside the project as a sub-pkg right now: https://github.com/coreos/etcd-operator/tree/master/pkg/chao....

We plan to make it a separate project once we feel good about its functionality and reliability.

If you have any potential use case, requirement in mind, please tell us. :)

ideal0227··on Infrastructure discovery with etcd
etcd use bi-directional streams for watchers. One. TCP connections can maintains multiple streams. No matter what you need to keep at least one connection. ZooKeeper is not an exception.
ideal0227··on Infrastructure discovery with etcd
There is a dashboard for publishing the testing result at realtime at http://dash.etcd.io/dashboard/db/functional-tests. The result is not super clear right now, we are improving it.
ideal0227··on Etcd v3: increased scale and new APIs
We are working on it (https://github.com/coreos/etcd/issues/5067). Probably we could work together on the Java client first? It should not be hard given that gRPC supports Java.
ideal0227··on Etcd v3: increased scale and new APIs
Yes. Here: https://godoc.org/github.com/coreos/etcd/clientv3/concurrenc...

It would be great if you can provide opinions, comments or help on these high level APIs. We also might move these to an internal proxy layer, so that other clients in other language can use it more easily.

(some more here: https://github.com/coreos/etcd/tree/master/contrib/recipes)

ideal0227··on Etcd v3: increased scale and new APIs
It really depends on how you view the problem. Yes, the latency of agreeing a proposal is similar, which is limited by physical (network latency + disk io). However, there are ways to put more stuff into one proposal (batching) and submitting proposals continuously (pipelining). These optimizations highly depend on the implementations.
ideal0227··on Etcd v3: increased scale and new APIs
Consul: https://github.com/hashicorp/consul/blob/master/bench/result...

etcd: https://github.com/coreos/etcd/blob/master/Documentation/op-...

Note that the testing environments are not exact the same in the doc I listed, but comparable. Also Consul performance is improved after a few releases.

So we did the benchmark internally on the same environment. The result is still comparable to what I listed in the two official docs.

The best way to compare performance is still probably to run the benchmark on your own environment.

ideal0227··on Etcd v3: increased scale and new APIs
etcd is based on Raft. Raft has a TLA+ spec. But do note that the implementation usually diverges from its algorithm [0].

For etcd, we try to keep the core algorithm as self-contained and deterministic (no I/O, no timer) as possible. So it can be very close to the pure algorithm. We are very confident about it since we throughly tested it, and the implementation is shared with other consistent large scale database systems too (cockroachdb, tikv).

ZooKeeper uses ZAB [1] under the hood. I do not think there is a TLA+ for ZAB.

[0] https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/...

[1] http://www-users.cselabs.umn.edu/classes/Spring-2016/csci821...

ideal0227··on Etcd v3: increased scale and new APIs
etcd3 has similar performance compared to ZK for small scale. For large dataset, etcd3 does better since it does incremental snapshot/ smaller memory footprint, when ZK does full snapshot that takes a lot of resources. For watches, etcd3 does streaming + TCP multiplexing to save memory.
ideal0227··on Etcd v3: increased scale and new APIs
It should not be hard. For v2 API, there are several Java bindings (https://github.com/jurmous/etcd4j).

For the gRPC API, it is easy to generate a Java gRPC client based on the defined service. We have plans to make that experience better.

ideal0227··on Etcd v3: increased scale and new APIs
I think the two projects focus on different things.

Consul provides features like health checking, failure detection besides its consistent key-value store. It aims to provide an all-in-one solution[0].

etcd focuses on the consistent key-value store. The key-value store has more advanced features like multi-version keys, reliable watches, and provides better performance. People build additional features on top of etcd's key-value store/raft.

(I work on etcd)

[0] https://www.consul.io/intro/vs/index.html

ideal0227··on Torus: A distributed storage system by CoreOS
Have you ever thought the etcd issues was just because how you operate it? Or maybe you were running an old version of it? Have you reported to upstream? If there is a very common issue that you can meet frequently, it should have been fixed or you should have reported it.
ideal0227··on Tikv: Distributed key value store powered by Rust
tikv's raft is a port of etcd/raft, which is used by a few projects in production. So we can expect rust raft to be mature soon.
ideal0227··on Improving Kubernetes Scheduler Performance
Placing quality and throughput are always a tradeoff. Kubernetes, now, uses a naive algorithm that goes over all its nodes and finds the best placement. This is simple and effective for most web workload, when most of your scheduled jobs will take "forever" to run. And as it states in the blog post, after some simple optimization, now the scheduler can schedule tens of jobs per second. There are other limits make it slower overall. So we stop optimizing scheduler to avoid adding unnecessary complexity in its early days.
ideal0227··on Scaling Raft
You are thinking about consistent RSM. Paxos and Raft are not the similar thing and not even at the same level.

Epaxos, along with other master-less RSM, does not require heartbeat. It is not about smart people, it is about how you want to implement your system based on the requirement.

ideal0227··on Scaling Raft
> The problem is the heartbeat; you can't have every node talking to every other node every few seconds.

The node does not have to talk with EVERY other nodes.

> What we really need is someone smart to come up with a consensus algorithm that doesn't need a heartbeat.

Well. Most consensus does not require heartbeat. Heartbeat is just the simplest way to detect a potential failure.

> if you want a system that runs fast

Consensus might affect the commit latency. It wont affect throughput that much though. But if you do not care, you can simply write to a consensus system without waiting for the commit. You can still get the serialization and replication.

ideal0227··on Is gogs dead?
Github is not blocked.
ideal0227··on Etcd Clustering in AWS
We have fixed the SSL issue in 2.1. We are also considering back-port it to 2.0 release if possible.
ideal0227··on China
As a native Chinese, I doubt anyone in Chine knows China good enough.

But as a native Chinese, what you should have known is China is growing faster and better in the recent 10 years.

ideal0227··on China
> implicit discrimination towards engineers

Never. There is discrimination on social status in China, which is determined by wealth.

> In Chinese college, the "elite majors" including cooperation and government management usually have the highest bar of admission.

This is not true anymore.

ideal0227··on [dead]
Your comment is not accurate. And your GF cannot confirm this either. There is a news said the airplane landed at Nanming(in Vietnam) rather than Nanning(in China).
ideal0227··on ZooKeeper vs. Doozer vs. Etcd
One thing is not correct. Raft allows to read from followers. You will never be able to read uncommitted value in raft.
ideal0227··on Etcd v0.1.0 release
People usually do not need to worry about implementation difficulties, unless they want to contribute. If you want to, I am happy to talk with you more about details.

I do not think it is difficult to use, unless people do not want to look at the docs. If you go through the docs, and still cannot figure out how to use I would like to help.

If you do not want to know the context, then raft/zk/etcd will probably remain mystified to you for a long time.

ideal0227··on Etcd v0.1.0 release
If you want to ask thing you do not know very much, you should be more humble. 1.etcd is not a key-value store like redis or memcached, whose aim is to provide high performance. etcd is aim at providing high reliable storage for small amount consistent data across multiple machines.

2.1 We have cmdclt tool, which will not return redundant information. 2.2 I do not think you know exactly what index is.

3.raft is a distributed consensus protocol, which help with fault tolerance and consistency. For sharing configuration using etcd, we can assume the conf server will be robust. The essential goal is to remove spof.

4.It is not. It is hard for our usage. Because it is HTTPS, .

← PreviousPage 2 of 3Next →