throwdbaaway

I almost fell for the hype of pcie gen4 after reading https://news.ycombinator.com/item?id=25956670, and it is quite interesting that pcie gen3 nvme drives can still do the job here. What would be the worst case disk I/O throughput while serving 400Gb/s?

https://github.com/GoogleCloudPlatform/compute-image-package... - it used to be just one project with Python, C++, and shell scripts. The Python bits got rewritten into Go, while the others got split into their own repositories.

https://github.com/Azure/WALinuxAgent - I think this is the equivalence of GCP guest-agent, serving similar functionalities, and is pre-installed on all official images, otherwise basic things like authentication and image baking will break.

By setting the provisionVMAgent property to false when creating a virtual machine, WALinuxAgent should run with all extensions disabled, and I think that's as minimal as a Linux VM can go on Azure.

This property, however, can't be set via https://github.com/ansible-collections/azure, which is of course another lovely OSS project by Microsoft. I didn't bother to send a PR.

The OMI agent seems to be a different beast that is way more obnoxious. The closest thing on GCP is probably the collectd agent and the fluentd agent installed for Stackdriver Monitoring and Stackdriver Logging? Plus whatever OS config to enable unattended upgrades.

I just learnt from this HN thread about the SSM agent on AWS. That one does seem equally obnoxious as the OMI agent.

EDIT: Looks like collectd and fluentd have been replaced by https://github.com/GoogleCloudPlatform/ops-agent

Ryzen 3100 can do the same, at the same price point, and was released 5 months earlier than 10100F.

Good for Intel for finally catching up?

It actually does more than fixing the mess now. With pipewire >=0.3.34, there is now support for a2dp duplex with both faststream and aptx-ll codecs (if the headset also supports them, of course). No more hsp/hfp crap.

I think linux + bluez + pipewire is the first ever software-only stack that can achieve this, as a USB dongle would be required on other platforms. We shall declare 2021 as the year of linux on the desktop!

Not a C/C++ programmer either, but I got an example. InnoDB was first released in 2001, and was ported from C to C++ in 2011: https://github.com/mysql/mysql-server/compare/78f4351..3a455...

Similarly, if the best use case of your software is to help with trip planning, it is not going to end well either:

> At LinuxCon last year, Carl Symon showed me a screen shot of an Activity created by Aaron Seigo to plan his holiday, which included a To Do list, a calendar, links to online travel guides, a weather report for his destination, and other features connected to the trip.

https://www.datamation.com/open-source/the-mystery-of-kde-ac...

Nah, I love SQL, but if I have to choose between SQL and ORM, I will go with ... something in between, like the core layer of SQLAlchemy.

Just take a look at that monstrosity of an UPDATE statement in the article. In my first job, I had to do something similar in SQL, but with a SELECT statement to allow users to search dynamically using any combination of the columns. I picked up SQLAlchemy in my second job, and have never looked back.

Looking at https://news.ycombinator.com/item?id=4004551, their AWS bill was about $100,000 a month back in 2012.

So it was $100,000 a month for 18 millions visitors, and it is now $20,000,000 a month for 454 millions MAU.

Corey Quinn has to take a look at this.

EDIT: Corey provided some insight on contract negotiation at https://twitter.com/QuinnyPig/status/1430327936519872514

> Maybe the situation has changed since 2016. In my experience with several employers since then, it seems like every enterprise architect fell in love with Cassandra around 2014-2015 and then had a long, painful, protracted breakup.

I think 2012-2014 was peak marketing from DataStax. There would be some new major feature with every new blog post, and it would mostly never work as expected. Between 2017 and now, things have settled down.

Of course, cassandra/mongodb/etc can perform their own batching when writing to the commit log, and can also benefit from write combining by not flushing out the dirty data immediately. That's besides the point.

Your use case allows you to perform batching for writes at the *application layer*, while discord's use case doesn't.

I thought you are doing millions qps with a 3 nodes mongodb cluster, from the top level comment. That would be impressive.

By batching 1-10 thousands records at a time, your use case is very different from discord, which needs to deliver individual messages as fast as possible.

I don't quite understand how is it possible to rollup percentile_agg from 15 minutes buckets to 1 day buckets. Won't the resulting values be off by a lot?

> First, we could not upgrade servers in place and needed to use logical dump and restore to build a new server. However, for very large mysqld instances, this can take many days on a live production server and this fragile process will likely be interrupted before it can complete.

This part is quite interesting. I'd think it will take maybe a day to dump/restore a database with a single 10TB table with no parallelism, at a speed of about 120MB/s. How big is these mysqld instances such that the process will take many days?

There is certainly no hard and fast rule. In the JDK case, in terms of language design, my observation is that with Brian Goetz in charge, it doesn't matter too much whether it was the slow release cycle back in the pre-9 days, or the shorter one from JDK 9 onward -- whenever a new feature is introduced, all kinds of compatibilities are being maintained at all costs.

(Except Project Jigsaw of course, but I'd like to think that he got overruled by his boss)

As for bugs, I will just quote Gil Tene from https://www.theserverside.com/opinion/Dont-ever-put-a-non-Ja...

> Play with them on your laptop, but don't use a single feature, and wait for the LTS.

Personally I think the LTS model is great. Features get rolled out as soon as they are ready, and then get stabilized after being used by early adopters. Meanwhile, production can stay on LTS and continue to get just the necessary fixes.

I don't think you need a FB account to read the posts there, you can just treat it like a blog with a crappy interface (but very good content). Probably some mandate to "eat your own dog food".

It is pretty scary because most of these new features or enhancements are buggy. By not adopting SemVer, users of MySQL who prefer stability (oxymoron?) have no obvious upgrade path. They could be happily using version 8.0.123, and then upgrade to version 8.0.126 to get some security fix, and suddenly encounter a bunch of functionality and/or performance regressions.

Some examples...

WL#10310 Redo log optimization:

Feb 2018, initial commit 6be2fa0bdbba, landed in 8.0.11 (Apr 2018)

Jun 2018, crash regression fix commit 270d18368650, landed in 8.0.13 (Oct 2018)

Apr 2019, performance regression fix commit 75c4f7161a56, landed in 8.0.18 (Oct 2019)

WL#5655 - InnoDB: Separate doublewrite file to ensure atomic writes:

Dec 2019, initial commit ce14ef911, landed in 8.0.20 (Apr 2020)

Feb 2020, data loss regression fix commit c1bc61dc7, landed in 8.0.20 (Apr 2020)

May 2020, stall regression fix commit 00b284707, landed in 8.0.21 (Jul 2020)

In comparison, Cassandra during the DataStax days was extremely buggy, but at least it tried to follow SemVer. So the operators can follow some simple guideline, e.g. >=2.0.14 is fine, >=2.1.17 is fine, >=2.2.9 is fine. Of course regression can still happen, but that would be an exceptional case.

> I recall having to explain to certain developers how they have to first figure out more than 5 different services for AWS, then use packer to build an AMI, which is provisioned using Chef, then they have to terraform the relevant services, then use cloud-init to pull configuration values. All in all I had to do most of the work, and the code was scattered in several places.

That sounds horrible. You can actually do all that from a single repository, using just ansible alone, with one cli invocation:

- use packer to build an AMI => ansible

- provisioned using Chef => ansible

- terraform the relevant services => ansible

- use cloud-init to pull configuration values => baked into the image

Ever since Microsoft acquired the company behind Helm and https://news.ycombinator.com/item?id=11922299 (try clicking the article link), it has been used as a showcase when onboarding azure customers, to somehow prove that "yeah azure is hip and we love open source".

So, yes, we need to know.

Interesting. Perhaps k8s is succeeding in the real world because it is the only one that tries to support all the weird and dumb use cases?

> Think of the history of data access strategies to come out of Microsoft. ODBC, RDO, DAO, ADO, OLEDB, now ADO.NET – All New! Are these technological imperatives? The result of an incompetent design group that needs to reinvent data access every goddamn year? (That’s probably it, actually.) But the end result is just cover fire. The competition has no choice but to spend all their time porting and keeping up, time that they can’t spend writing new features.

> Look closely at the software landscape. The companies that do well are the ones who rely least on big companies and don’t have to spend all their cycles catching up and reimplementing and fixing bugs that crop up only on Windows XP.

Instead of seeing k8s as the equivalent of "cover fire" or Windows XP, a more apt comparison is probably Microsoft Office, with all kinds of features to support all the weird and dumb use cases.

My observations after reading https://bugs.ruby-lang.org/issues/14718:

- Shyouhei Urabe did a marvelous job of resisting pressure from several well-known developers representing multiple high profile ruby companies/projects.

- Everyone eventually came to their senses in https://bugs.ruby-lang.org/issues/14718#note-89, and concluded that the best way forward is to use glibc with MALLOC_ARENA_MAX=2.

My personal feeling is that while the "Jemalloc FTW" section in http://oldblog.antirez.com/post/everything-about-redis-24.ht... might have been correct back in the jemalloc 3.x days, a lot has changed on both jemalloc and glibc since then. Furthermore, there is also musl in the equation these days.

You can build a binary distro based on Gentoo, e.g. https://github.com/sakaki-/gentoo-on-rpi-64bit (sadly EOL'ed several months ago).

Anyway, from my own experience, the video decoding performance of gentoo-on-rpi-64bit was somehow nowhere close to LibreELEC, on the same hardware. Perhaps there is a lesson to be learnt there...

> Like Gentoo it's one of those OSs that is cool for linux nerds and a headache for people who actually need to practice proper systems engineering.

Fear not. As GKE is running on Container-Optimized OS which is based on Chromium OS which itself is based on Gentoo, you can safely practice proper systems engineering within the container environment :)

On laptop, supposedly the simplest thing to do is to use the dedicated volume keys?

Again, this was back in kde 3 days, where desktop usage was more prominent than laptop, and even on laptop, a mouse with a scroll wheel was still a common accessory since touchpad gesture wasn't really a thing yet.

> Knobs are great in the physical world, never met one on a screen that wasn’t hard to use.

Well, I just gave you a perfect example where a knob on the screen was immensely useful. Of course it will be silly to try copying the physical world as-is, but with a little bit of creativity, applied in the right context, a knob on the screen can absolutely be useful.

Back in the kde 3 days, there was this applet called "knob" for volume control. For me personally, its circular knob UI is actually the best.

Firstly, you can place the applet at a corner of the panel, e.g. at the bottom right, such that you now have an infinitely large target [1] to hit.

Then, once you have the mouse pointer hovering above it, you can:

- scroll up/down to increase/decrease the volume

- middle click to toggle on/off the sound

For those who possess precise control of the mouse pointer, of course they can still click on an arbitrary spot on the knob to set the volume.

For bonus point, because it is a knob, you always know what is the current volume level. All that for maybe 40x40 pixels of screen real estate only.

[1] https://blog.codinghorror.com/fitts-law-and-infinite-width/

On the cloud, it is likely simpler and faster to just spin up a new cassandra datacenter, and then do a rebuild from the old datacenter to the new datacenter, either all nodes at once in parallel or in smaller batches. This procedure works fine regardless of using static tokens allocation or vnodes, and adds very little load to the old datacenter which is still serving traffic.

Somehow there is no mention here yet about this infamous rant: https://dgraph.io/blog/post/why-google-needed-graph-serving-...

> I started a project to unite all Google OneBoxes under this graph indexing system, which involved weather, flights, events, and so on.

Now we know who's to blame :)

> This piece of data is highly dynamic and processes every 30 seconds or so based on hundreds of thousands of metrics.

Perhaps you guys need a ... database? Relevant HN discussion few months ago when tailscale migrated from json to etcd: https://news.ycombinator.com/item?id=25767128

While we are piling on Airbnb, I just discovered like an hour ago that browsing image gallery of houses/apartments on Facebook Group/Marketplace is actually a lot pleasant and smoother than on Airbnb. This can't be possible.

Good thing bchesky has stopped taking photographs of the listings, otherwise his skill would just be wasted on his platform.

> However, the slow down only affects Pythons built with --enabled-shared anyway.

On Gentoo, the ebuild for Python and Ruby both have --enable-shared hardcoded.

Also, if you use prebuilt Ruby binaries from RVM, those were also built with --enable-shared. And if you build your own Ruby binaries with ruby-build, it also does --enable-shared.

So, there are still plenty of opportunities around for instant speedup.