Hacker News Reader

I'm reminded of the (long winded) article that popped up on here some time ago about a person who was suggested to modify redis' source code to fix a problem they had: https://medium.com/@paydro/hacking-redis-to-save-money-64723... (slightly scathing discussion: https://news.ycombinator.com/item?id=10835950).

I wonder if redis can be convinced to flush data to disk. AFAIK it does it periodically?

If only SQLite was a tiiiny bit more scalable. It's the only DB where you have to explicitly disable force-flushing of transactions to disk.

Creating a test-db can be even simpler:

   $ createdb mytest
   $ psql mytest

This assumes that you already have a data directory and a running pgsql instance. Mine doesn't. :)

Simply having the binaries is enough- this is something that gives pgsql another edge on mysql.

That's why I said it can be simpler ;)

May I ask what company you are working for?

Ubisoft. :)

How about the performance? Is it a good fit to use postgres as a k/v store?

> But getting started with PostgreSQL is easier and it scales better, so it's hard for me to agree with using it in 2017.

Facebook and Youtube use MySQL, YT via Vitess, in 2017. I am just curious if you could exactly point out what part of PSQL scales better.

The memory locking which I mentioned.

Arguing "x uses y therefore it's good" is a poor metric, I personally spent months with a lot of very expensive hardware finding out if we could go forward with mysql (there was a strong push for mysql), but I found PGSQL not having any of the same single server scaling issues (IE; all 40 cores could be uses, all memory could be used, all disk channels could be used) with no hacking of the kernel or memory allocators.

To say that Facebook uses MySQL is a bit of a misnomer. No one at Facebook outside of a hundred or so people on the db team uses MySQL directly, it is just an on-disk storage system with a basic SQL front-end that can be used as an intermediate translation layer and even then it was more of a legacy issue than anything inherent to MySQL as a technology. Please don't use it as an example of why someone should consider MySQL, as there is almost no way anyone else is going to be using it in a similar fashion.

Facebook has spent literally thousands of man-years working to make this system scale and perform in the way they need it to, but there is nothing about this effort that required MySQL and in fact a lot of effort was expended working around glaring deficiencies in MySQL. If they had it to do over again I think that no one at Facebook would select MySQL at this time.

Plus there's the whole MyRocks thing too...

Just about everything you've said here is factually incorrect.

There are far, far fewer than 100 people working on MySQL at Facebook. I say this as a former member of the team, and I am still in close contact with several team members.

Additionally, every engineer at FB has the ability to issue ad hoc SQL queries, and many teams do use special-case MySQL DBs that they interact with directly. I was the lead on FB's DBaaS so trust me that I know what I'm talking about here. Over 1000 engineers had used the DBaaS at the time I left the company.

There is no legacy issue involved. FB is changing storage engines in MySQL right now; they have the resources to completely move away from MySQL if there was a compelling reason. Your statement of "If they had it to do over again I think that no one at Facebook would select MySQL at this time" is completely incorrect.

Even if you ignore FB, a much larger percentage of the largest sites on the internet use MySQL than Postgres.

As a former manager on the ops side who worked closely with the managers of the db teams, and the hadoop/hbase teams, and the presto folks, wormhole, scuba, a few other teams that may not be public knowledge, etc. I can say with absolute certainty that the only people in the company who thought it was not a sad joke that we used MySQL down in the bottom layers of the stack were the people on the db team. And why wouldn't that team like MySQL? Knowledge of its intricacies and bugs and how to work around its problems is what got them this high-paying gig.

I will simply state that having also spent a long time inside FB I disagree with your assessment regarding the high regard in which you think MySQL is held within the company.

> I can say with absolute certainty that the only people in the company who thought it was not a sad joke

You can say with absolute certainty that you know the opinion of thousands of engineers at Facebook? This is clearly hyperbole. It does not appear you are interested in having a meaningful discussion on this topic.

> I disagree with your assessment regarding the high regard in which you think MySQL is held

Where did I say it's held in high regard? I simply said there is no compelling reason to move away from it, and also said your numbers were way off base in terms of how many people are working on MySQL at FB, as well as how many people directly interact with MySQL at FB.

I also greatly question how closely you were interacting with the managers of the db teams if you think there were 100 people on these teams!

[EDIT: It's also possible that we just have very different definitions of people "on the db team". For people "working on MySQL" I would only include MySQL Eng, DB Client, MySQL Infra PE, and the majority of Data Performance PE. Arguably you could include some RocksDB people at this point, though their work is used in many non-MySQL ways too, and even then that brings the total to still way under 100 people. Overall I would not include teams like Wormhole or TAO as "working on MySQL", though even if you add them in, still significantly under 100.]

In retrospect my tone should have been softer and less combative here. My apologies. But to give some perspective on why this hits a nerve: By greatly overstating the number of engineers who worked on something, you're directly undermining the effort and ingenuity of the engineers who worked on it. This stings coming from a former coworker.

In your second reply (sibling to this one) you seem to state that Facebook's db engineers are motivated by self-preservation (re: staying with MySQL), and you imply they are over-paid. If self-preservation was a motivation at all, FB wouldn't be migrating major data sets from InnoDB to MyRocks, which has a completely different set of performance characteristics and administrative/operational/automation concerns. As for pay, yes the db teams are very well-compensated because they are literally among the top experts in the field, working on an absolutely massive billion-daily-user OLTP system and all of the insane challenges that entails.

Put yourself in my shoes. You worked in security, yes? If I posted some factually incorrect comments about your team, and the engineering motivations and talents of your team, on a public forum like this, how would you react?

I actually started at FB by managing SRO prior to joining the security team, so I am quite sure that I was over-estimating my total number of people by including the teams in Dublin and probably everyone who needed to hop on to a udb directly for some reason. As you can imagine, from that starting-point in my introduction to MySQL at FB it was all about dealing with various problems and slapping down hundreds of db replication error and similar alarms every hour in the pre-FBAR/pre-db-automation world. I was also probably out of line with the 'it is about job security' bits, sorry about that. Actually, scratch 'probably' from that apology, but having passed through the ops side, infra eng, and security roles at FB I ended up seeing all of the problems and a lot of the solutions developed there seemed in my view to be rather expensive ways to fix problems that either didn't manifest in Postgres or would be easier to fix in a different db.

I still maintain that while MySQL was the right choice in 2006, if you were starting FB today it would be hard to convince someone that MySQL is a better choice than Postgres.

That makes sense, though MySQL certainly had a lot more warts before version 5.6. Respectfully, you may be inadvertently comparing MySQL of 5+ years ago to Postgres of today, which isn't a fair fight :)

> if you were starting FB today it would be hard to convince someone that MySQL is a better choice than Postgres

Not sure I agree. Seems like more of a toss-up at the very least. How would you find or build the right talent pool? With MySQL, Facebook was able to hire the top folks from both Google (which cancelled its MySQL efforts around the exact right time for FB to poach) and MySQL AB / Sun.

I also strongly suspect it would be substantially harder to build something like Wormhole on top of Postgres replication.

That said, I will include my usual disclaimer that Postgres has a lot of compelling features that MySQL lacks, and I can definitely see the other side of the argument.

> I also strongly suspect it would be substantially harder to build something like Wormhole on top of Postgres replication.

Why's that? It seems like there's all the infrastructure needed for that?

Wormhole relied on smart propagation of MySQL binary logs iirc. Postgres lacks this and the WAL would probably not work in the same way.

https://www.postgresql.org/docs/current/static/logicaldecodi...

MySQL uses logical replication, and traditionally this is statement-based logical replication -- log of raw SQL write queries, including arbitrary SQL comments embedded, which can be used for any purpose. I'm not sure if Postgres has something similar? (entirely possible it does, if so I retract my earlier comment!)

This doesn't mean anything.

At some point, it's easier to throw resources at your old stack to make it scale instead of migrating to something else.

Example: Facebook and PHP. They decided it was easier to write an entirely new runtime instead of migrating to something else.

Often, this results in significant improvements (MySQL and PHP are great examples).