Python Generator Tricks for Systems Programmers | Hacker News Reader

Python Generator Tricks for Systems Programmers

dabeaz.com

65 pointsdood13 comments

The author of this, David Beazley, was a professor of mine for two courses as an undergrad at UChicago. He taught Networking and Operating Systems and was one of the top 3 professors in the CS department. He left teaching for a while, but it looks like he's back now, which is extremely encouraging for all CS students!

If anyone has the chance to listen to him speak or teach, I highly recommend it. (Also, if anyone has the chance to see him perform jazz music, I highly recommend that, too!)

This tutorial is awesome. It is not just the content. The sequence and the presentation is just plain awesome. You are a lucky basketball.

I was a little surprised that the genlog.py program is said in the slides to be slightly faster than the nongenlog.py... That is, several chained generator expressions actually ran faster than an explicitly written "for" loop. I suppose it may have something to do with using the C-implemented sum() function rather than doing a running total in Python; nevertheless, I expected more overhead from the generators. Very cool.

Stackless Python makes a lot of these possible without fancy generator tricks. Not that I don't love fancy generator tricks, but I think the stackless model is more intuitive.

Can you provide a more explicit example, perhaps a Stackless alternative to one of the ones from Beazley's page?

The most obvious is coroutines:

  def ping():
      while ping_channel.receive(): #blocks here
          print "PING"
          pong_channel.send("from ping")

  def pong():
      while pong_channel.receive():
          print "PONG"
          ping_channel.send("from pong")

Most of the things that he has in there don't need to be done with generators, though generators make it more efficient and arguably cleaner. However, sometimes generators are just a twisted workaround for controlling program flow. Channels and tasklets in Stackless express program flow very well, in my opinion.

You can see more of what stackless has to offer here: http://members.verizon.net/olsongt/stackless/why_stackless.h...

Very cool, just going through it now. It has the feel of my course slides in university, but much more to the point and with better examples.

nice examples. just a note, pickle shouldn't be used for serialization if the source is untrusted.

Source of what? Do you care to elaborate...?

http://www.python.org/doc/2.2.3/lib/pickle-sec.html

So add another step: sign the pickled data and check that upon arrival.

sorry i didn't elaborate, it isn't a question of signing requests, kirubakaran posted a link below that explains the basics (unpickling can create objects and run potentially harmful methods)

depending on what you are sending there are several alternatives for serializing data. there are a handful of json libraries to take a look at and some good benchmarks if you google for them.

> sorry i didn't elaborate, it isn't a question of signing requests, kirubakaran posted a link below that explains the basics (unpickling can create objects and run potentially harmful methods)

Parent poster meant signing the pickled data as a text stream, not as a collection of unserialized python objects. If the signatures don't match, you just discard the data, and don't unpickle it.

This guarantees that (as long as your keys aren't compromised, and your signing algorithm is unbroken) you will never unpickle tampered-with data.

Reply on news.ycombinator.com