HNHacker News
TopNewBestAskShowJobs

treffer

944 karma · joined December 5, 2010

submissionscomments
treffer··on AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage
This is not how such storage systems work. If the whole storage system is for you and you only store compressed video files on it then perhaps.

But we are talking about multi-tenant systems here. You will get better performance and/or better prices if the system finds compressible data for other customers.

All the benefits hold, even for you with non compressible data, if there is an overall benefit.

Hitting a bottleneck less often then this will reduce your tail latencies, too.

It is just that _your files_ won't contribute to these improvements. But you get all the benefits as well.

treffer··on AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage
Having a big CPU (which you need to get many PCIe lanes) and then not using it is waste.

It would be waste to _not_ use the CPU time.

Granted, the CPU powet consumption would drop with enabled power management, but that's only marginal gains as the CPU is likely busy anyway and the total duration of busy time might drop (race to sleep)

I have used this inverted logic to great extends. E.g. when expanding data center capacity I was usually swapping old harware for new hardware once it arrived. It is way better to have the older generations as spares and use the new capacity at the most utilized places.

Do not waste the things you pay for!

treffer··on AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage
I think it is a clear cut, mostly because I do not think that compression compromises any of those features, all while making the user experience better.

For any storage system like this you usually have a few bottlenecks. IO and Network are the obvious ones, followed by tiering (cache, fast io, slow io, ...) and at the very end CPU.

Now let's say network is your bottleneck. If you can send the data to the client in a compressed for then you get the compression ratio as additional bandwidth. And the user would get the data quicker! So compression to the network is a clear win.

But the common bottleneck is often IO, a high end SSDs with 1M IOPS at 4KB would _theoretically_ serve 4GB/s, a 40GBit link. That's without any redundancy over other overhead.

Again compression to the storage layer would decrease the total amount of IOs, thus making sure a customer gets data quicker.

Ok, let's say both are not the issue. The fastest compression algorithms compete with memcopy. So if you need just one copy of your data you might have been faster by compressing it.

Especially fast compression algorithms (zstd, lz4, snappy, lzo, ...) are worth the CPU cost with virtually no downsides. The problem is finding the right sweet spot that reduces the current bottleneck without creating a CPU bottleneck, but zstd offers the greatest flexibility there, too.

Oh for range requests.... Those large objects are likely split anyway, for easier error recovery (imagine 100MB into a 1GB transfer you notice that the file data was corrupted - not good). Once you work on blocks it's easy to do somewhat efficient range requests again.

treffer··on Shouting in the Datacenter (2008) [video]
And the other comment made me google noise vs. pressure....

Looks like Siemens did an investigation and tested various parameters (pressure, noise, vibration) independently.

They point towards noise, not pressure. Really interesting as I am _sure_ I was told it is pressure. But that might be 100% wrong!

https://virz.de/wp-content/uploads/2020/05/2015-White-Paper-...

treffer··on Shouting in the Datacenter (2008) [video]
I was once told by a datacenter operator that this is due to the pressure change when they pump the gas in and that we should plan on replacing all HDDs should the fire alarm go off.

...and I think it was Argon...

treffer··on Cast iron leet
Well, I was a huge fan of ceramic ones but found that they also wear off. Still better than non-stick pans, but not perfect. Might be that newer ones are better but I already switched to cast iron by now.

Right now my main pan is from a Kickstarter campaign (Stur). Plus a sticky metal one where that's enought.

treffer··on Ask HN: Which Linux terminal emulator do you prefer and why?
At some point I got hooked to kde konsole because it allows you to hide everything: hide the scroll bar, hide the menu, hide the tabs. Nothing but window decoration & a terminal.

Oh and my terminal window is slightly transparent. There is a sweet spot where I can still read e.g. this text through the terminal (if need be) while not getting annoyed by the transparency.

That plus I still have all the abilities of a "fancy" terminal emulator: tabs, large scroll window, hyperlink klicking, ....

treffer··on Booting Linux in five seconds (2008)
This has been one of my favorite talks, especially "Don't settle for 'make boot faster.' It's the wrong question. The question is 'make boot fast'"

The opposite of this was the massive performance improvement that sqlite had a few years back. Many small incremental improvements that add up.

These 2 approaches are IMHO what you have to consider if you want to go fast.

treffer··on Body integrity identity disorder patient satisfaction after surgery (2014) [pdf]
I skipped right to the conclusion, as I wanted to see what the document says, and it kicks of with "where no other treatment exists", and to prevent self-inflicted amputations. Plus it acknowledges that these people potentially need help _now_, before we find better solutions in the other fields.

I can highly recommend to give the conclusion section a read. I think the authors are well aware of these issues, to the point that this scoping (no other treatment available right now / other options exhausted + person needs help right now) makes me slightly supportive.

treffer··on Germany Ends Ban on Abortion Advertisement
Hrm, looked it up out of curiosity...

§218a lists that it is not a crime under certain circumstances ("Tatbestand des § 218 ist nicht verwirklicht, wenn"), but the heading of that one is "Straflosigkeit", so "no punishment".

IANAL but this reads like it is not a crime under the listed conditions. Which is different from no punishment.

treffer··on Germany Ends Ban on Abortion Advertisement
To clarify... This is at least partially the aftermath of a lawsuite if I recall correctly.

There was a case where a doctor listed abortions as one of their offerings on their website, and that was seen as an advertisement. [Citation needed, my memory might fail me here]

So "no advertisement" pretty much means "don't tell anyone anywhere beforehand".

And that was seen as non-helpful, especially as abortions are legal in Germany.

treffer··on Ask HN: Is secondary DNS setup worth it?
Terraform and OctoDNS can make this easy.

Never let the UI of your vendor be the source of truth for your DNS setup.

If you have the zones in code then updates should be easy to distribute to multiple vendors.

Also: given the nature of DNS caching you will have to roll changes in a way that is compatible with temporary inconsistency anyway.

I have seen such a setup once and it was a life saver. People recommending that are right. People fearing the complexity are also right. Be careful to have good tooling.

treffer··on pass – the standard Unix password manager
I am using pass on my Android phone all the time (e.g. to log in to ycombibator right now).

I use OpenKeychain (from f-droid) to access my yubikey + Password Store (also in f-droid) for git clone (readonly ssh key) and password filling.

It integrates with any input field on the whole system and my phone can't decrypt any passwords alone and it uses standard encryption methods.

treffer··on The best engineering interview question I've ever gotten
100% this. I have "The Art of Computer Programming" on my book shelf, and I think I had to pull it out 2-3 times over the course of _years_.

I am sure one was a reference in the golang source code. And I once wanted to check multi-way merge sorts (tournament sort).

treffer··on Liquorix Kernel
> Maybe the extra responsiveness under heavy loads is slowing it down?

Yes! Extra responsiveness is done by switching more often between tasks. This is not free, so throughput suffers. The linux kernel lists that as the downside of e.g. 1000Hz ticks (something the Liquorix kernel enables).

treffer··on Liquorix Kernel
And it's all about tail latency for processes.

A big latency spike in latency and your desktop seems to hang, the audio shutters or you see framerate drop (or even freezes) in games.

How do you go after these? You optimize for fairness (which guarantees your workload a fair fraction of the resources) and frequent scheduling (higher frequency / smaller slices).

The downside is that you will trade in maximum throughput.

Looking at the settings I can see that this is what they adjust, but they should clarify what they regard as crucial for such a setup.

treffer··on Grafana Incident: Smart incident management for your teams
Well, been down the Prometheus/Alertmanager path and it is pretty workable there.

You would get all the templating and grouping by labels.

Dashboards can showed Prometheus alerts through annotations on the graphs. So you get a visual feedback what was broken when.

Firing alerts are also a metric on Prometheus, so you can list those (or do other stuff with it).

It's not a UI thing though. More the lower layers to get stuff for Grafana.

treffer··on Some ways DNS can break
Bonus error: you can't have DNS packets >64k. Even over TCP. (Been there)

Someone has configured a delegation loop. (Yep, been there more than once)

Records are currently unavailable, but someone has busted the cache. (Enjoy)

All services wait for the same DNS record to be fetched and will all ask the same service at exactly the time when DNS is done. (Again, clear yes, happens. That's why you should use prefetch)

No one knows how to handle DNS SRV priority correctly (please correct me if I am wrong).

treffer··on The Container Throttling Problem
I have been running k8s clusters at utilizations far beyond 50% (up to 90% during incidents). For web services/microservices, so tail latencies were important.

The way we solved this? 1. Kernel settings. Check e.g. the settings of the Ubuntu low latency kernel for example. 2. CFS tuning. Short timeslices. There are good documentations on how to do that 3. CPU pressure. We cordoned and load shedded overloaded nodes (k8s-pressurecooker).

By limiting the maximum CPU pressure to 20% you can say "every service will get all the CPU it needs at least 80% of the time on most nodes". This is what you want. A low chance of seeing CPU exhaustion. This is needed for predictable and stable tail latencies.

There are a few more knobs. E.g. scale services such that you use at least one core as requests are effectively limits under congestions and you can't get half a core continuously.

Very nice to see that people go public about this. We need to drop the footprint of services. It is straight up wasted money and CO2.

treffer··on Moderna booster increased neutralizing antibody levels against Omicron ~37-fold
Ah, and...

KW49, 2021-12-06 until 2021-12-12 had originally 3M BionTech shots available, with KW50 blocked for kids. At the same time 10M Moderna shots were available, or 20M booster shots.

2 weeks earlier, when those order volumes became clear, Spahn advertises for Moderna.

A few weeks later and we got an extra 3M BionTech shots.

Official medical sources have been clear. But StiKo & politicians an thus media communication is skewed.

treffer··on Moderna booster increased neutralizing antibody levels against Omicron ~37-fold
E.g. "Das Vakzin von BioNTech sei der Mercedes und der von Moderna der Rolls-Royce, sagte Spahn."

> https://www.tagesschau.de/inland/corona-booster-spahn-modern...

This came out the moment that the deliveries of BionTech were low. Weekly estimates are available at

> https://www.bundesgesundheitsministerium.de/fileadmin/Dateie...

The recommendations usually follow that curve. And given the current availability: good luck getting a BionTech shot.

Related: I am pretty happy how open the underlying data is. Data quality is sometimes shit (hey RKI/Gesundheitsämter, looking at you) but overall the stuff is open.

treffer··on Moderna booster increased neutralizing antibody levels against Omicron ~37-fold
Cons: you might be sick for a day. (Had very bad reactions to the shots and this one lasted 1 day - but I caught something else before/after...)

Pros: seems to help against Omikron. And given the speed at which it spreads you don't have time to wait for updated shots or alike.

JJ was particularly bad in not catching it, and even worse after 6 months. Get a booster or have a close to 100% chance to catch Omikron.

Official communication is often skewed by availability. E.g. Germany didn't order enough BionTech but got plenty of Moderna so they announced that Moderna is way better. Just check the numbers and forget about anything else in the news. Numbers are like a weather forecast. Read it, prepare, put it aside.

treffer··on Dell Xps 13 Developer Edition availability?
In the past I've pinged Barton George about this on Twitter. Try pinging @barton808 instead. Usually sales and customer support should be able to give you details on when/how to buy.

It usually takes a few extra months to get the Developer Edition in Europe, though

treffer··on GitHub Copilot is not infringing copyright
It is probably a stretch, but I think less of a stretch than saying "it just a machine that learned to code and randomly reproduced these 10+ lines of code". That has IMHO a probability of 0.

So if I rule that out, where does it end up? What if we put this as the grown up ML brother of the chain of LZW, PPM, dictionary assisted compression (e.g. zstd) and various attempts at using neural networks for compression?

I would not want to judge this - that's why I put up the AGPL idea. Or even unlicensed code. It would be a very interesting case to watch.

treffer··on GitHub Copilot is not infringing copyright
Well, I have a hard time drawing a line between GitHub Copilot and a compression algorithm.

If you can reproduce a verbatim copy of Quake source code after taking that source code as input before then that's compression. A really fancy, but still.

And given that it reproduces the source code: it has to hold that somewhere.

It would be very interesting if someone could reproduce the Quake example with AGPL code, then request the whole model + code because it clearly contains the AGPL code in some encoded form.

treffer··on Did I break you? Reverse dependency verification
Yes, and the whole solution allows you to plan / decide on (2) and (3) - both need that awareness.

And (1) is common in the open source and cross organization world. But it is less desirable if both the consumers and producers of the library work in the same place. Version spread causes maintenance cost. And at one point it was a common theme that "an upgrade of JVMKit would have avoided this incident".

treffer··on Did I break you? Reverse dependency verification
The former approach was release + upgrade all projects. Now if one of those fails post-update you have to build a new release and upgrade all projects _again_. Multiply that by hundreds of services and it can become pretty annoying.

By pulling all the reverse dependencies and testing those _before_ releasing & upgrading you reduce the chance that you need another bugfix release.

And with hundreds of projects the average quality of tests does not matter that much. Some projects will happily pass, while some may fail. Those that fail will provide valuable insights.

treffer··on Why do security keys store secrets unencrypted or poorly encrypted?
to clarify: the biggest concern for these devices is that the prove of ownership is leaking. And an easy way would be to copy the device.

Anything that copies the identity (private key) to a physical key is broken because the identity could be copied before it is on the key.

That's why you create a cryptographic keypair on the device, and the device does not offer to extract the private key.

It must be impossible to clone the key. When you read articles that claim 100% phishing failure due to these YubiKeys then that's because phishing is copying secrets. And you simply can't for the key. You have to physically steal this device.

And then there is also a human aspect. If you have something that is convenient and 100% successful then don't make it less convenient. You risk that people try to circumvent it.

treffer··on Why do security keys store secrets unencrypted or poorly encrypted?
I think you have the wrong perception on how YubiKeys add to your security.

2fa means you use different proves. A password is proof that you know a secret. A YubiKeys outputs proof that you are in possession of this device. The tapping is there to avoid rootkit based exploits (e.g. streaming the data to another device on demand).

Adding a password to you YubiKeys does not add anything to this. It does not strengthen this.

treffer··on On UML
And most tools were tailored towards that use case!

The (open source) exception that I remember from back in the day was Green UML.

Their sourceforge is still live and saying "green is a LIVE round-tripping editor, meaning that it supports both software engineering and reverse engineering."

Renaming a method in a reverse engineered UML diagram and seeing that reflected in code was pretty neat.

I never got why this isn't a standard feature until now. Hearing about the relationship between UML and broken project management I get it (I wasn't into that at that time, except that I never bought into the waterfall promise).

Let's hope someone will revive UML as a code navigation and round trip refactoring tool. That would help a lot and nicely fit an agile approach.

← PreviousPage 6 of 8Next →