Hacker News Reader

Comment by orf | Hacker News Reader

That implementation looks pretty inefficient, creating lots of strings that are thrown away immediately. Could it not be refactored into a generator combined with a string.join?

View on HN

I think Python's string type has an optimization that concatenation on a string where the LHS only has a single reference to it will reuse the existing string when possible. You still have to grow the buffer periodically, of course, but it's not creating as much garbage as it looks like.

> I think Python's string type has an optimization that concatenation on a string where the LHS only has a single reference to it will reuse the existing string when possible.

It's specifically a CPython optimisation, it may (and usually will) not hold up on alternate implementations like pypy or jyton.

Sure, but the function in question lives in CPython's own implementation of its core libraries, so it seems reasonable for it to rely on that implementation.

If PyPy or Jython want to reuse that core library code, it's on them to make whatever changes are needed to make it fast on those implementations.

Usual use would be "translate pattern once, compile it and then match 1000 file names against it".

Ahh yes, it's cached in fnmatchcase. It's pretty fast as well, <10µs for a basic match.

A cache without any eviction! Probably should not expose `fnmatch` to the network where the pattern varies.

That's an old version of Python source from a mirror. It uses functools.lru_cache with a maxsize of 256 now: https://github.com/python/cpython/blob/fcfe80ec2592fed8b3941...

Gah, thanks for the proper link. I should've double checked (and realized it since no updates in 8 years is unlikely).

The RE engine here is only used as a fast VM; so the pathological cases from the article should still apply. So apart from the memory use issues there's also that.

It should be updated with a functools.lru_cache decorator I think.

Reply on news.ycombinator.com