Hacker News Reader

Hacker News

Comment by dmitrygr | Hacker News Reader

Full thread

dmitrygr

That last rust example is less readable than modern template-metaprogramming variants of C++.

There is something elegant about being able to beat it out on speed with about 30 lines of C

View on HN

The main function is egregiously overcomplicated.

This is how I'd write it instead:

  fn main() {
        match env::args_os().nth(1) {
                None => write(b"y\n"),
                Some(arg) => {
                        let mut arg = to_bytes(arg);
                        arg.push(b'\n');
                        write(&arg);
                }
        }
        process::exit(1);
  }

It's just as efficient and far more readable. It avoids Cow, a lambda, and the chain of combinators.

Doesn't this only print once, and not over and over, like `yes` should?

The write function takes care of repeating it. See the complete source code in the blog post.

> There is something elegant about being able to beat it out on speed with about 30 lines of C

What is this referring to? If it's referring to GNU C, the linked C implementation is about 100 non-comment lines, and is three times slower on the author's machine than the Rust version.

see sister comment

Ah, I'm sorry, I thought that program was meant to be humorous. :P

As other have pointed out elsewhere what matters here is not much the performance of the language but how much you can avoid calling IO code by making the path as straightforward as possible. You can get comparable performance ( 6.5 GB/s vs 8. GB/s on my machine) with 10 lines of python code, which probably took me the same time to write as the original rust version.

I'm sure a lot of people would like to see the code for comparison. I know I would.

The code is below. I have to admit that after a few more test I realized that the performance greatly depends on the length of the input, in some cases going as low 2 GB/s and some others reaching the performance of the GNU yes installed on my system. This is probably due to the fact that I build the buffer in a quite naive way. With a bit more work maybe that can be fixed.

    #!/usr/bin/python
    import sys

    yes = "yes\n"
    print(sys.argv)
    if len(sys.argv) == 2:
        yes = sys.argv[1] + "\n"

    size = 4096
    buf = bytes(yes * size, 'utf-8')

    while True:
	sys.stdout.buffer.write(buf)

Simply putting a comment above each function would make it bearable.

not as readable as this faster variant

  #include <sys/uio.h>

  //2048 "y\n" s
  static char __attribute__((aligned(4096))) str[] = "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n";
  static struct iovec v[] = {
  	{str, 4096},			//adjust number of these to taste
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  };
  
  
  int main(int argc, char** argv) {
  
  	while(1) {
  		//scatter-gather write (on linux there is basically no way for write to write an odd number of characters, except EOF, so we need not check for partial writes
  		(void)writev(1, v, sizeof(v) / sizeof(*v));
  	}
  	return 0;
  }

Bah, that'll never pass code review. How about this:

  #include <sys/uio.h>
  
  #define str(x) #x
  #define expand(x) str(x)
  #define y1 y\n
  #define y2 expand(y1) expand(y1)
  #define y4 y2 y2
  #define y8 y4 y4
  #define y16 y8 y8
  #define y32 y16 y16
  #define y64 y32 y32
  #define y128 y64 y64
  #define y256 y128 y128
  #define y512 y256 y256
  #define y1024 y512 y512
  #define y2048 y1024 y1024

  //2048 "y\n" s
  static char __attribute__((aligned(4096))) str[] = y2048;
  static struct iovec v[] = {
  	{str, 4096},			//adjust number of these to taste
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  	{str, 4096},
  };
  
  
  int main(int argc, char** argv) {
  
  	while(1) {
  		//scatter-gather write (on linux there is basically no way for write to write an odd number of characters, except EOF, so we need not check for partial writes
  		(void)writev(1, v, sizeof(v) / sizeof(*v));
  	}
  	return 0;
  }

Both suffer from way to much binary bloat, for small performance hit we can shave off a whole 4Kb.

Is anyone going to take a stab at an enterprise edition?

"Enterprise edition" would be the same thing but written in Java and require an Oracle backend.

Don't forget LDAP, SAML and SNMP support. What's the OID for 'y' again?

https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...

Could reuse type from 2.16.840.1.113883.18.347. It's an extended yes/no, to account for "flavours of null". (Not even joking...)

I just died a little inside.

Here you go.

   <?xml version="1.0" encoding="UTF-8"?>
   <xsl:stylesheet
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      version="3.0">

      <xsl:output method="text"/>
  
      <xsl:template match="/">
         <xsl:call-template name="y"/>
      </xsl:template>

      <xsl:template name="y">
         <xsl:text>y&#10;</xsl:text>
         <xsl:call-template name="y"/>
      </xsl:template>
   </xsl:stylesheet>

That doesn't have the same functionality - it doesn't accept words from the command line.

And the naive "readable" version is half the speed of python. Makes you wonder how typical that would be in real code?

Fun fact:

  $ python yes.py | pv -r > /dev/null
  [11.2MiB/s]

  $ python3 yes.py | pv -r > /dev/null
  [4.95MiB/s]

As for why the naive rust version is slower, it's because without adding a BufWriter in rust, stdout is line-buffered, so each line emits a write system call, while with python, stdout is buffered. Python 2 emits writes of 4096 bytes, and python 3... 8193 bytes (edit: not a typo, this is 8KiB + 1). That's the likely cause for it being slower.

Edit: A minimal version of the naive rust version would be:

  fn main() {
    loop {
      println!("y");
    }
  }

On the same machine as with the python tests above, I get:

  $ ./yes | pv -r > /dev/null
  [4.81MiB/s]

which is actually as slow as python 3, despite doing 4 thousand times more system calls.

A version with buffering would look like:

  use std::io::{stdout,BufWriter,Write};

  fn main() {
    let stdout = stdout();
    let mut out = BufWriter::new(stdout.lock());
    loop {
      writeln!(out, "y").unwrap();
    }
  }

And produces 129MiB/s on the same machine. And that's strictly doing the same as what the python version does (with a default buffer size of 8KiB, apparently).

And fwiw, on the same machine, both GNU yes and the full rust solution from OP do 10.5GiB/s.

> As for why the naive rust version is slower, it's because without adding a BufWriter in rust, stdout is line-buffered, so each line emits a write system call, while with python, stdout is buffered. Python 2 emits writes of 4096 bytes, and python 3... 8193 bytes. That's the likely cause for it being slower.

Does it have nothing to do with the fact that string-of-bytes is the default in Python 2, whereas string-of-characters is the default in Python 3? Or is that perhaps related to the explanation you gave? Forcing the byte interpretation, Python 3 is slightly faster than Python 2 for me. Forcing the character interpretation, Python 2 wins, but not by as much as before.

Bytes:

  while True:
      print(b'y')

Characters:

  while True:
      print(u'y')

Your bytes version outputs lines of, literally, `b'y'`.

The characters versions is still a clear win for python2 on my machine (8.9MiB/s vs. 5.6MiB/s)

It's also worth noting that the buffering behavior of python is only happening because the output is a pipe to pv. If it were the terminal, it would be line buffered, like the naive rust version.

python3 seems to do much better than either of those for me when using an unbuffered write(1, ...) syscall (plus it prints the correct thing)

    $ cat yes3.py
    stdout = open(1, 'wb')
    while True:
        stdout.write(b'y\n')
    $ python3 yes3.py | pv -r > /dev/null
    [13.7MiB/s]

    $ cat yes2.py
    import os
    stdout = os.fdopen(1, 'wb')
    while True:
        stdout.write('y\n')
    $ python2 yes2.py | pv -r > /dev/null
    [7.77MiB/s]

For better comparison with my numbers, I ran your scripts on my machine:

  $ python3 yes3.py | pv -r > /dev/null
  [18.4MiB/s]
  $ python2 yes2.py | pv -r > /dev/null
  [10.2MiB/s]

In both cases, a 4KiB buffer is used by python. That's still way slower than the equivalent rust code with a 4KiB buffer (use BufWriter::with_capacity(4096, stdout.lock()) instead of BufWriter::new(stdout.lock())).

Out of curiosity:

yes2.py:

  import os
  stdout = os.fdopen(1, 'wb')
  while True:
      stdout.write('y\n')


  $ python yes2.py | pv -r > /dev/null
  [9.12MiB/s]

  $ pypy yes2.py | pv -r > /dev/null
  [45.5MiB/s]

So pypy does a good job of speeding it up.

Off a quick 9 second run, python2 with profiling:

     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
          1    4.272    4.272    9.301    9.301 yes2.py:1(<module>)
   30856080    5.029    0.000    5.029    0.000 {method 'write' of 'file' objects}
          1    0.000    0.000    0.000    0.000 {posix.fdopen}
          1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

I got 1.26Gb/s in Python2 on MacOS (i7 mac air)

    import os
    ys = 'y\n' * 2048
    while True:
        os.write(1, ys)

update: seems to peek at around 3gb/s with ys = 'y\n' * 2 * * 16

(had to separate the asters to stop HN swallowing them)

Plain old `yes` gets a measly 33Mb/s!!

On my Macbook with way too much running, I get about 800MiB/s in Python2 with your script, but 1.12GiB/s with this Python3 script:

    import sys
    
    s = b'y\n' * 1024
    
    write = sys.stdout.buffer.write
    while True:
        write(s)
    
    # 1.12GiB/s

Plain old yes comes in at 22Mib/s.

The Python3 docs say to "use the underlying binary buffer object" when reading or writing binary data.

https://docs.python.org/3/library/sys.html

tested on my C1 on scaleway:

  perl -e 'print "y\n" while 1' | pv -r > /dev/null
  [3.37MB/s]
  perl -e 'print "y\n" x 2048 while 1' | pv -r > /dev/null
  [ 425MB/s]
  yes | pv -r > /dev/null
  [11.1MB/s]

>because without adding a BufWriter in rust, stdout is line-buffered, so each line emits a write system call,

Why the hell is it line-buffered when writing to a pipe? Yet another common sense “enhancement”?

It's always line buffered. Unlike python, where its behaviour depends on whether stdout is a tty or not.

That doesn’t answer my question, let me reformulate it: what’s the rationale behind this behavior?

For a program like this, execution is I/O bound rather than CPU bound, so even if Python is generally less CPU efficient than Rust the effect is overwhelmed by a different I/O strategy. It's just like how Node has greater max throughput for trivial server workloads than a naive C implementation.

That doesn't make a whole lot of sense when the guy above you posted that two versions of python (2 vs 3) have significantly different output throughput rates.

If it's I/O bound why are they changing at all?

It is I/O bound, but Python 2 and 3 do I/O differently. From the same comment that you're referring to:

> Python 2 emits writes of 4096 bytes, and python 3... 8193 bytes (edit: not a typo, this is 8KiB + 1).

>If it's I/O bound why are they changing at all?

Who said I/O functions didn't change between Python2 and Python3?

According to empirical evidence in my sibling post, that's not true in this case.

Your post seems to demonstrate _exactly_ what the post you're replying to is saying: most of the speed has to do with I/O strategy (buffering and syscall usage), not the actual speed of the involved language. Maybe I'm missing a distinction you're making, but I'm not following how your previous comment leads to what you're saying here.

The rust version that does exactly what python does (buffered output) is an order of magnitude faster (even if I force the rust buffer size to be 4KiB like with python2).

To head off this comment chain, I've softened the language in my original comment to "overwhelmed by" rather than implying that non-I/O factors are wholly irrelevant. :)

Reply on news.ycombinator.com