Hacker News Reader

Comment by luckydude | Hacker News Reader

Pretty sure that he's just rediscovering what I did in lmbench back in the early 1990's. All written up and open source (and got best paper in the 1995 Usenix):

http://mcvoy.com/lm/bitmover/lmbench/lmbench-usenix.pdf

Edit: which now that I think about it, was over 20 years ago. Reminds me of something a friend (Ron Minnich) said long ago: people rediscover stuff in computer science over and over, nobody wants to read old papers. Or something like that. He said it better, I think it was something like "want to look smart? Just take any 5 year old paper and redo whatever it did, our attention span seems to be very short."

View on HN

Your link is about 1990's hardware, the article is about 2010 hardware.

The article was revisiting the same topic exactly because the hardware changed (the author said so multiple times), and cheap context switches were a big marketing topic at the time.

It would even be interesting to get some newer study of that. You can bet the relative costs of a context switch changed a lot since 2010, as did the waiting latency.

Did you read the posted article? It spends a lot of time trying to figure out how to measure context switches. Which I carefully wrote up around 1995 and still works to this day. That's independent of the hardware and was sort of my point. This is a solved problem.

And if you look at Table 10, the time for a context switch is remarkably similar to what it was 20 years ago (which means the OS is fatter, not surprising, it's scales up on SMPs now).

I got the impression you were talking about the numbers.

In that I agree. You've got basically the same procedure written there for anybody to learn. And a nice tool that runs it.

The numbers are just a snapshot into history but what I measured and how it was measured is far more interesting. For example your comment "you've got basically the same procedure" isn'tquite right. If you read the paper, much effort was spent explaining to people that the baseline context switch is just a floor, what you will see in real life will be bigger since in real life a context switch is more that just a register file save/restore.

Very few benchmark writers get that point but that's sort of the main point. Not getting it means you are doomed to be mislead.

Replying to my own reply, I reread what they were doing and yeah, you are right, they do have basically the same procedure. My bad. Good on them for getting it (though I'll grumble that this was a solved problem, you can apt-get install it).

But you were right, looks the same. Sorry about that.

No problems. Overall, I'm glad you shared your paper, lmbench seems very nice.

Cool, glad you aren't pissed. And this whole thread has made me want to do a lmbench follow on paper, just get a new set of results for 2017 for *BSD, Linux, MacOS and maybe Solaris (does anyone care about anything other than Linux, MacOS, Windows these days?).

You're like the auto mechanic I talked to that was bemoaning this generation and how automotive tech has basically "stayed the same" since the 50s... "MacPherson struts, you ever heard of those boy!? Yeah - invented in the 50s!" Nothing wrong with a fresh take on an old topic.

Author somewhat addresses that in a comment:

@Adrian Cockcroft: I didn't use lmbench because I was wondering how to write such a benchmark in the first place, and you always learn more by building things yourself than by using something already existing. My little benchmarks aren't supposed to be alternatives to lmbench, I simply provided the source code so others could unambiguously see how I did the benchmarks and reproduce them.

Everything old is new again it seems.

I guess this is a variant of CADT, based on ignorance of the past rather than a chase of shinies.

People have always obsessed over "news", since humans are information-hungry by nature. But the modern tech crowd seems to have forgotten that there are two kinds of news: the kind that is objectively new (i.e. recent data), and the kind that is subjectively new (i.e. I didn't know that before). I think the situation is a bit similar to the distinction between objectively established frequencies and subjective Bayesian priors.

Full results from mcvoy.com i7@3.5ghz:

                     L M B E N C H  2 . 0   S U M M A R Y
                     ------------------------------------
    
    
    Basic system parameters
    ----------------------------------------------------
    Host                 OS Description              Mhz
    --------- ------------- ----------------------- ----
    slovax.mc Linux 4.4.0-5   x86_64-glibc223-linux 3497
    
    Processor, Processes - times in microseconds - smaller is better
    ----------------------------------------------------------------
    Host                 OS  Mhz null null      open selct sig  sig  fork exec sh  
                                 call  I/O stat clos TCP   inst hndl proc proc proc
    --------- ------------- ---- ---- ---- ---- ---- ----- ---- ---- ---- ---- ----
    slovax.mc Linux 4.4.0-5 3497 0.09 0.12 0.28 1.76 2.170 0.12 0.58 86.4 249. 644.
    
    Context switching - times in microseconds - smaller is better
    -------------------------------------------------------------
    Host                 OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
                            ctxsw  ctxsw  ctxsw ctxsw  ctxsw   ctxsw   ctxsw
    --------- ------------- ----- ------ ------ ------ ------ ------- -------
    slovax.mc Linux 4.4.0-5 7.630 8.5200   10.5   10.9   15.5    10.8    19.0
    
    *Local* Communication latencies in microseconds - smaller is better
    -------------------------------------------------------------------
    Host                 OS 2p/0K  Pipe AF     UDP  RPC/   TCP  RPC/ TCP
                            ctxsw       UNIX         UDP         TCP conn
    --------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
    slovax.mc Linux 4.4.0-5 7.630  13.6 14.7  20.8  19.4  25.8  24.5 25.8
    
    File & VM system latencies in microseconds - smaller is better
    --------------------------------------------------------------
    Host                 OS   0K File      10K File      Mmap    Prot    Page   
                            Create Delete Create Delete  Latency Fault   Fault 
    --------- ------------- ------ ------ ------ ------  ------- -----   ----- 
    slovax.mc Linux 4.4.0-5 2.7920 2.0480 8.2600 4.0840    17.6K 0.158 5.00000
    
    *Local* Communication bandwidths in MB/s - bigger is better
    -----------------------------------------------------------
    Host                OS  Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                                 UNIX      reread reread (libc) (hand) read write
    --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
    slovax.mc Linux 4.4.0-5 1928 11.K 5674 43009M  11.0K  11.2K 7686.9 14.K 10.3K
    
    Memory latencies in nanoseconds - smaller is better
        (WARNING - may not be correct, check graphs)
    ---------------------------------------------------
    Host                 OS   Mhz  L1 $   L2 $    Main mem    Guesses
    --------- -------------  ---- ----- ------    --------    -------
    slovax.mc Linux 4.4.0-5  3497 0.889 2.6670   57.4

You should probably share the fixes that made it compile (let alone work) for you. I'd like them, anyways.

Kind of like hacker news posts.

One reason people are reluctant to trust old sources (regardless of how well received they were at the time) is that software, operating systems and hardware all move at lightning pace. I've not read your paper, but I'm willing to bet enough has changed to warrant a new writeup talking about modern systems.

I very much doubt it. If you read the paper, it took each of the things it could measure (memory, disk, network, etc) and measured bandwidth and latency. Those are pretty basic measurements and I suspect that they will still be interesting 100 years from now and measured in the same way.

I'm running the benchmark right now on a modern system just to see what it says. Yep, still works fine:

    "size=0k ovr=0.90
    2 7.63
    4 6.51
    8 7.78
    16 7.07
    24 7.78
    32 9.02
    64 10.35
    96 10.49

That says that a hot potato (I send a byte to you, you send it back to me, the time is the round trip for one pair) context switch is ~7 usecs. Do it in a ring with 96 processes and it goes up to ~10 usecs.

I think almost 100% of that benchmark is still valid (I say almost because there must be something that has bit rotted in 20+ years but I don't know of anything). It's worth a read.

Edit: formatting.

> operating systems and hardware all move at lightning pace

not really, last big leap was the ability to unlock gpu for stream processing, and that's it.

it's quite faster today as ten or one year ago, but at it's core it's still a von neumann machine and not much has been discovered ever since, mostly we're repackaging under a new name things that have been discovered in the 70s', like the thick vs thin client debate, which played more or less the same over and over again as the bottleneck switched from bandwidth to latency to client performances, the last iteration of which was ember and angular precompilation.

Some of the stuff goes back even further than that.

The thread on the Intel Optane stuff the other day had an interesting conversation about how you would program for such systems and someone said we already did, mainframes 50 years ago.

It's nice to see old ideas in a fresh context.

> I've not read your paper, but I'm willing to bet [...]

Offloading all burden of proof onto the partner in a conversation strikes me as.. odd - esp. if said partner already provided a paper on the very same topic that you did not even read.

Reply on news.ycombinator.com