'with' gets better with python 3.1 | Hacker News Reader

'with' gets better with python 3.1

h3manth.com

22 pointsinit029 comments

Nevermind that this feature isn't particularly exciting... Python 3.1 is "old." Python 3.2 has been out for a while, 3.3 will be coming out soon... there are plenty of interesting features in 3.x. This is a small part of the overall picture. Rock on, Py3k devs.

Compound with statements are nice, put this is not a 3.1 feature. This was added in 2.7: http://docs.python.org/reference/compound_stmts.html#the-wit...

And you could hack this with contextlib.nested before.

Despite the numbering, 2.7 was released a year after 3.1. It added a bunch of features that helped compatibility with 3.x.

Thanks, I forgot about that.

It was a feature to both. Check the same document for the 3.1 release (and the What's New).

This is cool.. But of the things that 3.1 has got me excited about, this doesn't really move the needle. It's pretty straight forward to throw together a context manager that deals with multiple files and access using `files.in` and `files.out` if you want to.

It's not that straightforward to create your own context manager, and creating a correct generic wrapper for the use case of composing multiple context managers is even harder (as demonstrated by the woes of contextlib.nested).

And if you're going to create a custom "composed" context manager every single time, you're probably better off just nesting `with` statements.

I don't get it. Can someone explain to me how this is different or better than:

  infile = open('site.log'); outfile = open('err.log', 'w')
  for line in infile:
       if 'ERROR' in line:
           outfile.write(line)
  infile.close(); outfile.close()

Sure, you're saving a line with the files being automatically closed at the end of your scope, but hardly "MIND == BLOWN".

What am I missing?

elegance?

"MIND == BLOWN" is a hyperbole in almost every context, but the `with` statement is just cleaner code with fewer chances to screw up (e.g. by forgetting to close).

Error handling.

You're missing that if an exception gets thrown, the files are still guaranteed to be closed as soon as you exit that 'with' block.

Imagine you open 'infile', but then opening 'outfile' fails. To prevent a resource leak, now you need to close 'infile'. The linked code guarantees this. Likewise if there is any error which aborts the copying, both files will get closed and you don't have to worry so much about corner cases.

See this for more:

http://effbot.org/zone/python-with-statement.htm

In a more complex application, an exception might be thrown before the last line that closes the file (and caught god knows where or not at all), so the files may not be closed... then again, if you actually try to catch the exceptions you get an extra level of indentation by using `with`.

More... "philosophy" here: http://stackoverflow.com/questions/11373502/zen-of-python-vs...

Ah, so 'with' is a clearer syntax for exception handling, that's what I was missing. Thanks to you, masklinn and graue for clearing that up.

Glad it helped you, but I wouldn't say "a clearer syntax for exception handling" really... more like a "way to ensure your code always works without having to actually think about exceptions". (Wrong as it may seem, I try to imagine I'm using "let" in a Lisp and then I instantly start to love the "with" syntax, although it's not a necessary part of Python :) )

> Sure, you're saving a line with the files being automatically closed at the end of your scope

That's actually pretty significant if you're, say, using Pypy or IronPython, otherwise you're leaking handlers if you've got an error raised.

But he was actually comparing the code to:

    with open('site.log') as infile:
        with open('err.log', 'w') as outfile:
            # code

. MIND=BLOWN is tongue-in-cheek. Hopefully. Then again, he's commenting on a 2 or 3 years old version of python so...

I think "with" is a great example of where Python and Ruby differ philosophically. Python uses "with" to very explicitly implement their state manager protocol.

Ruby doesn't need "with" because "with" is really just a limited special case of what you can do with blocks.

The Python fans will counter that their less general solution is harder to abuse.

> The Python fans will counter that their less general solution is harder to abuse.

Not sure why anybody would say that, you can plenty abuse `with` if you want to have fun, the only thing you can't do is "dual-use" the block really (grab a context and invoke the block multiple times). And that you could always do with a decorator, if you're going there. The only thing `with` is is less flexible/general.

`unwind-protect` or `BlockClosure#ensure:` are probably much better examples of the philosophy opposite python though, I've always found ruby rather half-assed.

Not being mean, but this article kinda made me laugh. Python 3.1 has been out over 3 years[1]. And with the over the top "Mind == BLOWN"... Somebody should have told him about that release 3 years ago :-)

http://www.python.org/doc/versions/

Since nothing got better with python 3.0 I'll stay with 2.x as long as I can.

Nothing got better with Python 3? What about the print function and true division? Sure, those are both there as from __future__ import's, but I'd definitely call those improvements.

I haven't switched to Py3k yet either, but I'm just waiting for a few more packages I use heavily to get the bugs worked out.

Also, dict comprehensions. You can fake them in 2.x, but the syntax is ugly and cumbersome.

dict comprehensions were backported to 2.7

    $ python2.7
    Python 2.7.3 (default, Apr 13 2012, 13:51:30) 
    >>> {i: i+1 for i in range(5)}
    {0: 1, 1: 2, 2: 3, 3: 4, 4: 5}

> What about the print function and true division?

Those are actually fairly minor in the grand scheme of things (and compound `with` statements have been backported to 2.7).

More interesting are:

3.0

* unicode strings by default and much better overall support of unicode

* chained exceptions (I don't believe they've been backported)

3.2

* argparse merged to the stdlib

* ElementTree 1.3 finalized in stdlib (3.2)

* functools.lru_cache, itertools.accumulate (nothing game-changing, but nice to have in the stdlib)

* threading.Barrier

* concrete tzinfo implementation (for fixed UTC offsets)

* contextlib.ContextDecorator (creates a type which works both as a contextmanager and a function decorator), and improved contextmanager support across the board (ftplib, popen, TarFile, ...)

3.3 (soon to be released)

* Virtual environments shipped by default

* Namespace packages

* Kind-of obscure, but flexible string representations (strings can dynamically and transparently switch their implementation between latin-1, ucs-2 and ucs-4, this kills the narrow/wide compile-flag shit and generally makes strings more space-efficient)

* sub-generator delegation, instead of:

    for item in sub_generator_function():
        yield item

  you can just write:

    yield from sub_generator_function().

  it also correctly forwards sent and thrown values (generator.send() and generator.throw()) to the sub-generator, so it actually translates into an unholy mess which now just works and allows more easily splitting a large function into sub-iterators.

* lzma/xz support (7z)

I'm actually excited for 3.3, there's plenty of cool stuff in there.

The syntax seems very similar to Lisp's with-open-file macro.

Since you can write your own "context managers," the with statement analogue from Lisp world would rather be unwind-protect.

Yes, basically the only difference is that Python's with operates over a context management protocol involving some `__magic__` methods so it can work with other things in addition to files.

You can also do that in common-lisp by using the special form `with-open-file` is built upon: unwind-protect[0]. `with-open-file` is just a macro over it, specialized for files handling.

[0] http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...

Reply on news.ycombinator.com