Hacker News Reader

Comment by nwmcsween | Hacker News Reader

What safer options are there with equal size, overhead (read: none) and not full of layers upon layers of abstractions?

That sounds like you're implying that safety has no value, the only "overhead" worth considering is CPU cycles (what about the programmer's time/effort/sanity?), and that abstractions have negative value.

I don't think that's how you're supposed to software.

Yeah. Especially considering that CPU overhead has very little value for time syncing program.

but latency, and jitter, are important in this application. Two things that discount most of the "safer" languages. After ignoring inmature ones, the field is pretty barren.

Yes, but if you're building an app for the future, then what excuse is there for not using Rust, which is both very low latency and very close to a 1.0 release?

I get that sometimes you just want to hack in a language that's familiar to you, but then call your project a fun side project and not the "NTPD replacement" (and yes, I know who phk is).

Please start by showing us the PLL in Rust (details are available at phk's blog) and reason about how those changes affects the workings of the program. Is it more readable? Less? More exact? That I would be interested in.

It's a little bit sad that at -3 votes, this is the lowest voted comment I've ever had in my ~4 years on HN, lower than a number of controversial political comments.

To clarify, I'm not claiming that everyone should start using Rust. Now was I claiming that PHK is somehow unqualified for the task, far from it. I'm not even saying C is a bad language (I like C), or that people should stop using C altogether. But if you're building an security-critical application designed to be used in the future across millions of servers, and one that requires low-latency, I simply don't understand why you wouldn't strongly consider Rust, which offers strong guarantees about memory and type safety along with low latency and a very modern and complete standard library. Or Ada. Or OCaml.

It's always a good idea to do a quick back-of-the-envelope calculation before generalizing like that:

About 2 million servers were sold world-wide in 2014Q2

Assume 25% runs UNIX and NTPD -> half a million servers

Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers.

Assume 100W/server -> 50 kW

50kW for half a year -> 220,000 kWh

Assume 500g CO2/kWh -> 110 tons of CO2

QED: I think CPU overhead is a very relevant concern for time synching programs.

> Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers.

Bold assumption. It uses 0.0006% on my slow 6 years old box (over 13 days uptime). Let's be generous and say it's 0.001%, then you get 1.1 ton of CO2, which is the equivalent of the production of about 32.8 Kg of beef. I don't think such dramatic conclusions can be made from those facts.

First, I doubt your kernels statistics actually account correctly at that level of precision. (If it does, that alone would waste a lot of CPU cycles!)

Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles.

And don't forget: That number was one quarters purchase of servers, running for half a year. All the servers bought in 2010, 2011, 2012, 2013 and the other half of the year were not included.

Things add up.

> Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles.

284K, which is a whopping 0.04% of available memory. I don't see how this increases CO2 output though.

> Things add up

To a still very insignificant number, dwarfed by the beef consumption of the people reading this.

I'd say there are better arguments in favour of using dated languages than this.

I can't change how much beef people eat.

I can change how their computers synchronize time.

I think you missed something in your calculations.

Removing 0.1% load from the machine will not result in 0.1% of energy savings, very very far from it if in any at all. And that's not even what we want to calculate. We want the difference with hypothetical alternative in a safer language, which is hard to guess, but is unlikely to be in the same order or two with your calculations.

It's a core operating system component not some web app.

Overhead is important since Linux is designed to run on a whole spectrum of devices. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. And frankly nobody could care less about programmer's time/effort/sanity in this case. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.

Overhead is important since Linux is designed to run on a whole spectrum of devices.

Like the difference between a ntpd.c vs ntpd.go is going to matter for anyone...

If you really find yourself stuck with an embedded device small enough that a few megabytes of RAM matter then you can still use the old C implementations, it's not like they are going away.

Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead.

In this case abstractions mean you can probably write the whole thing in under 2 kloc, get memory safety for free, and base it on a network stack that other people actually use and debug independently.

It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.

Umm. Yea. Right. Tell that to the ntpd guys with their 100kloc codebase...

The jitter introduced by garbage collection in a go implementation could well be significant, even with the new "never pause the mutator for more than 1ms" guarantee.

How much of the code is actually sensitive to interruptions?

Why not write that part in C and leave the rest to the more suitable language?

I would guess that's a rather small portion of the codebase, essentially the callouts to adjtime().

I don't buy into the low-latency FUD. C isn't a magical realtime language either; what happens when the system is under high load? What about delays in the kernel/network stack?

As a layman I would think incoming frames will have to be stamped with a hardware high resolution timestamp at arrival anyway, regardless of the language. And yes, this internal timestamp will have to be compared immediately before setting the hwclock. I would argue this is the only part that really needs to be written in a GC free language.

I might be wrong on this, but I'd like to see a stronger argument than "could well be significant" for writing yet another generation of a baseline system daemon in an unsafe language...

And speaking of latency: If this really is such an issue, why hasn't NTP long been moved into the kernel?

Overhead matters, but languages like Rust, D, and Nim add little overhead if any and should be perfectly possible to use as systems languages.

Safety has value it just doesn't imply overhead. Abstractions are a negative, they make reasoning burdensome.

Abstractions are what make it possible to reason about complex systems. Being able to ignore parts of your codebase is the only way to make changes to it once it gets beyond a trivial size.

The whole of the Unix philosophy is enforcing this policy of strong abstractions by making each part of a complex overall system its own binary, such that they can only communicate using the file I/O abstraction, possibly via pipes. C was therefore designed for a system which promotes abstractions.

Lifting up constants into defines/consts and DRYing up C code makes for shorter, less error-prone and easier-to-understand codebases. Eg there's no reason to continue writing code like it's 1960's because we can steal good software eng and maintainability lessons from everything that has come since. (If one must maintain absolute portability.)

PS: I would like to see a toolchain for Go that emits portable C instead of statically linking in a giant runtime... Lightweight camping vs. YAGNI hoarding. Seriously, the weight of runtimes would be the biggest barrier to serious systems deployment, outshining the slowdown, productivity and safety tradeoffs. (Eg Go would need to play nice with embedded constraints including memory usage as well.).

It's not "writing like 1960's programming' gratuitous abstraction just creates a burden for the programmer to dig through, it makes reasoning a pain.

That's taking things literally too far. Common sense says take some giant unsigned long long constant used in multiple places that cannot be vertically aligned and give it the semantic meaning it deserves. The goal of reducing LoC is to not obscure meaning but to write less code that states its behavior more clearly. Abstractions are garbage if they are confusing / hide meaning, but abstractions are immensely valuable if they perform complex operations in a clear manner (say an well-designed http client lib in C.)

Can you explain why abstraction is bad? Abstraction does _not_ mean your program will be slower. In fact, the opposite can happen, because the compiler has more context to work with, and can get rid of stuff.

Haskell + stream fusion gives better results than well tuned C in some very typical situations http://research.microsoft.com/en-us/um/people/simonpj/papers...

Even just comparing C++/C, C++'s templating mechanisms mean that generic containers can be optimised on a per-type basis.

Abstraction and overhead are not as tightly coupled as they used to be, and not introducing Heartbleed-style bugs is worth a lot in my opinion

Abstraction in my definition is hiding of complexity and it does just that.

The problem with abstraction is that in order to deterministically control performance, you have to break down all the walls of the abstraction anyway. If I'm not 100% confident that my compiler will optimize a vector op with SIMD, and I absolutely require SIMD performance, then I end up having to know the entire layer cake to be able to verify that it is doing exactly that.

In most cases it isn't a bad thing at all. But a use case like a timekeeping daemon, where a single unexpected garbage collection pause could cause a world of problems, it most definitely is more pain than gain.

Abstractions are certainly not "bad", there are many abstractions in Ntimed, it's also pretty object-oriented if you care to take a look.

This is more a matter of "the right language for the job" and for timing-critical systems programming running as root, that language is C.

And there are plenty of common abstractions I would love see added to the C language: basic linked lists, byte-endianess and packing for struct members, validity intervals for integers and FP variables (like Ada!)

Unfortunately my taste seems to be the direct opposite of ISO-C which have instead wasted time giving us another thread-API.

Rust. Some of it's design goals (as stated on their webpage) are among the exact things you mentioned: zero-cost abstractions, minimal runtime, guaranteed memory safety, threads without data races.

Rust isn't even at version 1.0 yet and is probably a year away from being in a "suitable for production" state.

System time is far, far too critical to take chances with.

On the other hand, this program is going to take a year or so before anybody seriously considers using it, simply due to the fact that few people are going to do something other than their distro default for such a small part of their system.

waiting for rust to hit production readiness will not avoid that.

The point is simply that if one started a project now with the idea that it would probably only be a serious competitor in a year, Rust will most likely be ready by then - it doesn't really matter if Rust's ready right now if your project's not going to be used by anybody right now.

Zero-cost to whom? The developer? no as now the implementation details have to be known, the machine? no as time will be spent in the compiler churning away removing junk or if the compiler isn't smart enough including it. Also the C++ / rust definition of zero-cost is only if you don't use the abstraction. Minimal runtime is relative.

Reply on news.ycombinator.com