HNHacker News
TopNewBestAskShowJobs

majidazimi

700 karma · joined August 7, 2013

submissionscomments
majidazimi··on Open-sourcing a 10x reduction in Apache Cassandra tail latency
Because GC related issues don't undergo from "problem" state to "solved" state. It is just a never ending stream of issues, that the team need to resolve, specially in a database realm in which metrics are hugely workload dependent.
majidazimi··on Open-sourcing a 10x reduction in Apache Cassandra tail latency
Based on previous research [1], almost every part of a DB is in the hot path.

[1] http://15721.courses.cs.cmu.edu/spring2018/papers/02-inmemor...

majidazimi··on Open-sourcing a 10x reduction in Apache Cassandra tail latency
Well, other than storage engine, the next big part of a database software is the query planner/optimizer which Cassandra doesn't have (due to simple KV nature of it). So there isn't much remaining. In a long term plan, rewrite them all and you have single code base and you'll benefit from mighty C++ in other components of the database. And there is still room for more optimizations: SIMD, ...

The GC problem is not limited to C*. This shit(virtual machine) is hitting the whole Hadoop stack: HDFS, Hive, Spark, Flink, Pig...

Immense number of tickets in any fairly large cluster is related somewhat to GC and JVM behavior.

majidazimi··on Open-sourcing a 10x reduction in Apache Cassandra tail latency
> Well, Java has the advantage of being platform (and to a certain degree, runtime) independent

What is the benefit of that? Who on earth runs a DB written in Java on windows? Any useful server software will end up using platform native features, be it SQL server, MySQL, HBase, ...

majidazimi··on Why I hate all articles about design patterns (2016)
I don't want to start a war. I'm neither an expert programmer nor a smart person. But isn't it better to put data and validation all together? What if someone forgets to call the validation function? I mean in some complex projects, there may be tens of those validation functions that need to called, probably in a special sequence.
majidazimi··on Why I hate all articles about design patterns (2016)
Except that one could unexpectedly/unintentionally pass "#@%^%#$@#$@#.com" to email function, but Email class works perfectly fine.
majidazimi··on MongoDB gets support for multi-document ACID transactions
We are using it for building aggregate tables from raw table. We insert data into raw table. And triggers propagate running Min/Max/Avg for hourly/daily/monthly tables. After couple of months we truncate our raw tables but keep aggregate tables. Duplicates are easily removed based on insert. And we get instantaneous result on our aggregate tables (No hourly batch job)
majidazimi··on We should welcome the advances that are making hospitals less important
The title should be: We should welcome the advances that are making hospitals less important in US.

There is nothing wrong with hospitals in EU.

majidazimi··on MongoDB gets support for multi-document ACID transactions
Call me when they added Triggers, Stored Procedures, Common Table Expressions, Window Functions,...
majidazimi··on Anatomy of a Ceph meltdown
I can't emphasize this more. Being able to comprehend the system is my first design choice, no matter how the alternatives are great/stable/fast, ... You are going to pay the price some day as long as you don't know how stuff works or interacts with each other.
majidazimi··on The Kubernetes Effect
> A consequence of the "Kubernetes Effect" is that while distributed systems are easy to build and use, a lot of developers lose sight of the fundamental problems which make distributed systems difficult.

I would extend this to cloud as well. The more prevalent cloud becomes, the more ignorant developers become. It's like: I have Mathematica license, who cares how to calculate function derivative?

majidazimi··on Designing Distributed Systems E-Book
A while ago, I answered the same question on Quora[0]. I listed my favorite ones there. And if you are interested into database systems checkout [1].

[0] https://www.quora.com/What-are-some-good-books-on-Distribute...

[1] https://www.quora.com/What-is-a-reading-list-for-distributed...

majidazimi··on Designing Distributed Systems E-Book
I still prefer classical scientific books. This book is too much tool driven. I need the actual concept.
majidazimi··on CMU 15-721 Advanced Database Systems [video]
Spring 2018 is coming in couple of weeks:

http://15721.courses.cs.cmu.edu/spring2018/

majidazimi··on Building a Distributed Log from Scratch: Scaling Message Delivery
> In Kafka and NATS Streaming, reads (and writes) are only served by the leader. Similarly, Amazon Kinesis supports up to only five reads per second per shard (a shard is Kinesis’ equivalent of a partition). Thus, if we have five consumers reading from the same shard, we’ve already hit our fan-out limit.

Long live DistributedLog. One of the great aspects of DistributedLog is the read scaling. Start reading from any replica. Even read proxy node caches records for all consumers. Launch as many read proxies as you need, since read proxies are stateless.

majidazimi··on Germany vs. Elsevier: universities win temporary free journal access
Basically everyone... All top universities are in the list...
majidazimi··on Building a Distributed Log from Scratch: Data Replication
Distributed logs like kafka/pulsar/nats are essentially a blockchain technology in trusted environments.

Raft/multi-paxos are already the cheapest communication bound consensus. Theoretically you can't achieve any better with current network guarantees.

majidazimi··on Building a Distributed Log from Scratch: Data Replication
The doc says:

> The replica that registers first becomes the new leader. The new leader chooses its LEO as the new HW.

Isn't it a better idea to elect a process who has the largest offset? Basically during leader election, a process should get confirmation of majority of correct processes. If another process observes that it has larger offset in the log, rejects the election and starts another round of election. Eventualy a correct process who has largest offset will become leader. Regarding correctness, this sounds to me as a safer election.

majidazimi··on Building a Distributed Log from Scratch: Data Replication
Regarding Kafka replication: What if the following scenario happens?

1. Leader replicates to followers

2. Followers get the messages and send ACK to leader

3. Leader gets the confirmation from followers, increments high watermark (HW) and replies client that messages is commited.

4. NOW: Leader fails before it could piggyback to followers that HW has been incremented.

5. The question is: Since potential leader is not aware of the fact that HW has been incremented, it becomes the new leader and truncates the log to old HW, which means we have lost an entry in the log that has been confirmed by previous leader.

As a result, client has been confirmed that the message has been successfully written, but it has been lost.

majidazimi··on Why Don’t Americans Understand How Poor Their Lives Are?
Isn't the maximum percentage for singles with no kids in tax category 0, which is around 40%?
majidazimi··on Implementing Malloc: Students and Systems Programming [pdf]
Well, as a CS student in a not so highly ranked university, it's not surprising that grads from top CS schools end up in Google, Microsoft Research, etc. They know how stuff work much deeper than others. Of course this is not a generalization. There are some crazy dudes in other universities or even not in university at all.
majidazimi··on PostgreSQL HA cluster failure: a post-mortem
I use two different controllers in JBOD mode. Each controller manages half of HDDs. Then I build cross controller software RAID 1. That is, I choose two disks from separate controllers to build raid 1. Last but not least, all HDDs are mixed from different vendors across controllers.

For CPU, RAM there is no other option. you have to replicate stuff.

majidazimi··on PostgreSQL HA cluster failure: a post-mortem
It's gossip. There is no precise gossip protocol. I've just mentioned one approach. It's not difficult to modify it such that each node gets one message multiple times, to ensure everyone will see the changes. Centrally configuration management systems like Zookeeper/etcd,... will also suffer from the same problem: message propagation takes time. Even with floating IP, a couple of requests may fail until routing tables converge. So there is no atomic configuration switch in either case anyway. The argument was if floating IP is easier than pure software approach to deploy, which is not the case. both of them are equally challenging. One of them requires experienced network engineers, the other one requires experienced distributed system designer. In case of failure the effort to switch from failed node is zero in both approaches.
majidazimi··on PostgreSQL HA cluster failure: a post-mortem
Always use software RAID. I'd rather loose 30% on efficiency than creating a single point of failure with a random hardware. Software RAID on modern OSes (Linux, FreeBSD, ... ) are pretty darn reliable and fast.
majidazimi··on PostgreSQL HA cluster failure: a post-mortem
It's not hard. The problem is operation team rarely exercise failures. Configure a test HA cluster in lab, and test it. If it works, push it to production. The production system, should be continuously tested with real failures to see whether fail over mechanisms actually taking over the responsibility or not.

OF COURSE every thing is going to work in the lab, BUT MAY BE there is some other corner case in the production that you haven't considered yet. --- Louis C.K. after a sleepless night of switching primary DB to secondaries.

majidazimi··on PostgreSQL HA cluster failure: a post-mortem
Zero. Update configuration in a central location. Push an update to configuration manager agent on each node (using a dead simple gossip protocol). The agent wakes up, pulls new configuration, writes it and reloads pgbouncer. Using an aggressive gossip protocol, the whole process shouldn't take more than 5 seconds on a large cluster, let alone couple of nodes.
majidazimi··on What MongoDB got right (2015)
SQL is most sophisticated language ever built by mankind. It is centuries ahead of the second most productive language. Mix Window Functions, Common Table Expressions, Qube/RollUp, Materialized Views, ... and holy smoke...

You should try really hard to be unproductive with SQL and any relational system.

majidazimi··on Linux kernel: multiple vulnerabilities in the USB subsystem
Considering scientific statistics, which says there are 0.5 to 25 bugs per KLoC in delivered software, we are really lucky that stuff still work... :D
majidazimi··on Stack Overflow reducing headcount by 20%
I'm a student assistant at SAP in Germany. Even I (as a part time hourly-paid employee) have a one month notice period.
majidazimi··on Apache Kafka Goes 1.0
The biggest differences:

- rebalancing: Bookkeeper (which is the storage backend of Pulsar) manages rebalancing way better than kafka.

- Read scaling: Bookkeeper uses consensus to store log records, which means you can read from whatever shard you want. Whereas in kafka, reader has to connect to partition master.

← PreviousPage 4 of 6Next →