Hacker News Reader

isotoppon You might want to stop running atop

Eh? Calling system() for a binary without a path? And why system() using execl() in the first place, when you could do something using execve() without a sh inbetween instead?

Even w/o an exploit this can be prettier and more secure.

isotoppon 50 Years in Filesystems: 1984

It is Sun ZFS.

And there are many interesting stops between FFS/UFS and ZFS. And ZFS, even today, is not the best system to use in all scenarios, by some margin.

isotoppon 50 Years in Filesystems: 1984

https://blog.koehntopp.info/2023/05/05/50-years-in-filesyste...

https://blog.koehntopp.info/2023/05/06/50-years-in-filesyste...

https://blog.koehntopp.info/2023/05/12/50-years-in-filesyste...

https://blog.koehntopp.info/2023/05/15/50-years-in-filesyste...

https://blog.koehntopp.info/2023/05/17/50-years-in-filesyste...

isotoppon Booking.com hackers increase attacks on customers

"We" don't. "We" need to create individual accounts for each eligible staff member and provide each with a 2FA system, for example a Fido key or a TOTP auth app, or an app that uses push notifications and asks for a "yes".

isotoppon Booking.com hackers increase attacks on customers

If you booked with Booking, and the hotel called you back, and told you they don't use Booking any more, how do you imagine did they get the information needed to call you back and knew of the booking?

What you have here is commission fraud on behalf of the hotel against Booking.com, and if you call back Booking.com and tell them about it, they will set the hotel straight.

Or you get back to the hotel, tell them you understand how commission fraud works and if they are willing to split that commission (18%) half and half with you.

isotoppon Tracing Python

Originally the question was "What is Honeycomb.io and Opentracing doing and how can I get this into my existing application?" on IRC. On explaining, the discussion went into various directions (Fred Fish C Debugging Macros and tracing Python).

isotoppon Tracing Python

I just pushed an addendum, and it is being rendered right now.

isotoppon Why I still have an old-school cert on my HTTPS site

Rachel's Site shows an Apache 2.4 banner.

Apache has a little known module as part of the build, at least in Ubuntu: mod_md.

Use this, it is trivial to set up, and completely automated the Let's Encrypt workflow and renewal, no external stuff necessary. It is as simple as adding a few lines to the global config, remove almost all SSL config and restarting the server (twice).

isotoppon In MySQL, use “utf8mb4” instead of “utf8” (2016)

Anything that is shipped on Android, on iOS, or as a macOS app, apparently. Sqlite is incredibly useful and much more prevalent that you imagine.

isotoppon In MySQL, use “utf8mb4” instead of “utf8” (2016)

If, in a customer facing database, you have to table scan, you have already lost.

For data warehouses this is a different story, but god help you if you are table scanning in OLTP context.

isotoppon In MySQL, use “utf8mb4” instead of “utf8” (2016)

There are collation changes between utf8 and utf8mb4. Specifically, utf8mb4 uses the (much better) UCA 9.0 algorithm.

But this implies that all indexes will have to be regenerated, if they contain a column that is utf8 and will be converted. As they sort differently, this requires rebuilding the index from scratch. Depending on data size this can take measurable time and effort.

isotoppon In MySQL, use “utf8mb4” instead of “utf8” (2016)

ONLY_FULL_GROUP_BY is the default in MySQL since 5.7.5.

isotoppon In MySQL, use “utf8mb4” instead of “utf8” (2016)

Back then MySQL used the memory engine for implicit temporary tables, and memory engine has no variable width data types. So a "varchar" becomes a "char", and a "varchar(255) charset utf8" becomes an allocation of six times 255 = 1530 bytes (filled with a lot of padding).

Reducing this to 3 bytes is still bad, but only half as bad as before.

Only MySQL 8 introduced innodb typed temporary tables (unlogged, even), so with MySQL 8 this is no longer a problem.

isotoppon NVMe is not a hard disk

The point of TCP is to provide protection against packet loss and reordering, and to provide flow control.

isotoppon NVMe is not a hard disk

You should be running flash with self-encryption (and make sure you have a drive that implements that correctly).

To zap a drive you ask it to securely drop the self-encryption key. The data will still be there, but without the key it is indistinguishable from random noise.

isotoppon NVMe is not a hard disk

> Backups that are easily restored trump replication for so many cases and much more cheaply

At 400 MB/s, you restore a terabyte in 45 minutes, and then you need to replay the changes that happened since the backup (in MySQL: replay the binlog), which will take aproximately another 15 minutes.

"One hour of MTTR or provisioning time per Terabyte of data at 400 MB/s sustained linear I/O speed" is a useful rule of thumb.

Having a replica that you can promote reduces that to seconds. Having a time delayed replica where you just roll forward the binlog reduces that to minutes.

You can work with modern databases without using replication, but that in most cases just shows you suck at operations.

isotoppon NVMe is not a hard disk

The GC in the device cleans up.

TRIM is useful, it gives the GC important information.

TRIM is not that important as long as the device is not full (less than 80%, generally speaking, but it is very easy to produce pathological cases that are way off in either direction). Once the device fills up above that it is crucial.

isotoppon NVMe is not a hard disk

You may want to play with a TiDB setup from Pingcap.

isotoppon NVMe is not a hard disk

If you are not running a database at home, you have the database in a replication setup.

That provides capacity, but also redundancy. Better redundancy than at the disk level - fewer resources are shared.

https://blog.koehntopp.info/2021/03/24/a-lot-of-mysql.html Here is how we run our datatbases.

isotoppon NVMe is not a hard disk

> I think the article is saying that a web server (customers) should be stateless, because everything important should be in a database (data track) on another host. And that database probably has application level handling for duplicating writes to another disk or another host.

Correct.

> Hardware level redundancy like raid1 seems useful because it simplifies handling a common failure case when a single HDD or NVME fails on a database server

Nobody needs that if your database does replicate. Cassandra replicates data. MySQL in a replication setup replicates data. And so on. Individual nodes in such a setup are as expendable as individual disks in a RAID. More so, because you get not only protection against a disk failure, but depending on deployment strategy also against loss of a node, a Rack or a Rack Row. Or even loss of DC or AZ.

isotoppon NVMe is not a hard disk

"I don't know what features the network of the future will have, but it will be called Ethernet."

Ethernet is absorbing all the features of other networking technology, and adding them to it's own perfection.

isotoppon NVMe is not a hard disk

The image is from 2012 and describes a topology, but not a technology.

Today you'd rather use Arista 7060, 7050cx orJuniper 5200 as ToR. You'd not build 1:1, but plan for it in terms of ports and cable space, then add capacity as needed.

Almost nobody in an Enterprise environment actually needs 1:1, unlike hosters or hyperscalers renting nodes to others. Even then you'd probably being able to get away with a certain amount of oversubscription.

isotoppon NVMe is not a hard disk

For distributed storage, we use this: https://www.slideshare.net/Storage-Forum/operation-unthinkab...

We then install SDS software, Cloudian for S3, Quobyte for File, and we used to use Datera for iSCSI. Lightbits maybe in the future, I don't know.

These boxen get purchased with 4 NVME devices, but can grow to 24 NVME devices. Currently 11 TB Microns, going for 16 or more in the future.

For local storage, multiple NVME hardly ever make sense.

isotoppon NVMe is not a hard disk

My environment is not a home environment.

It looks like this: https://blog.koehntopp.info/2021/03/24/a-lot-of-mysql.html

isotoppon NVMe is not a hard disk

In NVME you can get around 800.000 IOPS from a single device, but the latency gives you around 20.000 IOPS sequentially. You need to talk with deep queues or with multiple concurrent threads to the device in order to eat the entire IOPS buffet.

Traditional OLTP workloads do not tend to have the concurrency to actually saturate the NVME. You would need to be 40-way parallel, but most OLTP workloads give you 4-way.

Multiple instances per device are almost a must.

isotoppon NVMe is not a hard disk

Flash has a flash translation layer (FTL). It translates linear block addresses (LBA) into physical addresses ("PHY").

Flash can write blocks at a granularity similar to a memory page (cells, around 4-16 KB). It can erase only sets of blocks, at a much larger granularity (around 512-ish cell sized blocks).

The FTL will try to find free pages to write your data to. In the background, it will also try to move data around to generate unused erase blocks and then erase them.

In flash, seeks are essentially free. That means that it does no longer matter if blocks are adjacent. Also, because of the FTL, adjacent FTL are not necessarily adjacent on the physical layer. And even if you do not rewrite a block, it may be that the garbage collection moves data around at the PHY layer in order to generate completely empty erase blocks.

The net effect is that positioning as seen from the OS no longer matters at all from the OS layer, and that the OS layer has zero control over adjacency and erase at the PHY layer. Rewriting, defragging, or other OS level operations cannot control what happens physically at the flash layer.

TRIM is a "blatant layering violation" in the Linus sense: It tells the disk "hardware" what the OS thinks it no longer needs. TRIM'ed blocks can be given up and will not be kept when the garbage collector tries to free up an erase page.

isotoppon NVMe is not a hard disk

Correct.

For that you have time delayed replicas and of course backups. But time delayed replicas are usually much faster to restore than a backup.

isotoppon Facebook at Work spamblocks some SQL queries

1. De-mail it in the preferences.

2. STFU notifications, then carefully tune back in.

3. The feed is crap. Do not read the feed. Make a bookmark bar:

3.1 https://companyname.facebook.com/chat/t/<..> Chat link

3.2 https://companyname.facebook.com/notifications/hub/all/ Notification hub link

3.3 https://companyname.facebook.com/?sk=h_chr Feed, force-chronological order, for hard cases only

3.4 https://companyname.facebook.com/profile.php?id=<your id here> Link to your profile, to write notes et al

3.5 https://companyname.facebook.com/groups/?category=membership Group Overview. This is your cockpit, basically

3.6 https://companyname.facebook.com/saved Saved Posts. Because their search is crap.

isotoppon Facebook at Work spamblocks some SQL queries

Upwards of 20.000 employees in my case here, with some serious variance and fluctuation because of the work being seasonal.

isotoppon Facebook at Work spamblocks some SQL queries

I could, but to actually understand you have to try out, because it's an experienced quality and not some a thing that is amenable to description.

Before the corp FB instance we had Wikis, Email, Mailing Lists, Jabber and an internal "Youtube" like Video Store. It all sucked.

With the FB instance, the awareness of what goes on in other parts of the company is much higher, the noise level is lower, communication is more timely and generally you feel less stressed out. Email numbers are down, mailing lists practically do not exist any more (except for machine communication such as commit messages), things organize neatly in FB groups, and you kind of automatically find the groups that are relevant to work. The Wiki hardly sees discussion any more, and is mostly documentation.

There are a lot of problems in Workplace by Facebook, mostly in the messenger product (no spatial metaphor, i.e. no rooms, only groups, being the largest). But in general it has been an amazing improvement on what we had before.