Hacker News Reader

Comment by forrestthewoods | Hacker News Reader

I thought the answer was going to be "tricking the world into thinking regex was a good idea". I've always considered regex to have the rare and elusive "write only" flag. Write only. As opposed to read only. Because once you write a regex that's it. You will never know what it does ever again.

View on HN

I have taken to thoroughly explaining what a regex does in the code comments. Not just "match that and not that" but going through every character in the expression. It is a practice i took up when playing around with brainfuck.

My magnum opus was a program that factored numbers in an arbitrary amount of time. That is _NOT_ something you hammer out in an afternoon, and without commenting everything thoroughly (like what I was doing, where I was in code execution, what state the whole program was in) I would never have made it.

Non-related story time: At first when playing with brainfuck, I wrote my own BF-interpreter that compiled to an IL which was easier to debug (with my own interpreter that could give me stack traces with comments about where I was in memory and what the hell was going on). After a month, I really didn't need to. My brain had gotten used to reasoning about BF-code and I could actually understand what code was doing as long as I knew which kind of compiler/interpreter it was written for.

Just use tests and comments. For instance, in many language it's trivial to declare regexp in multiple lines like this:

    regex = "(^[0-9])" //catch the initial digit (group 1)
          + "/"        //skip the following slash
          + "([A-Z]+)  //capture the identifier (group 2)
          ...

Regexps are too powerful a tool to ignore. I'd much rather write a simple regex than a whole screenfull of code, especially when the former does the work faster (because regex engines are pretty efficient).

Another way of doing it is to use variables and helper functions, particularly if you're going to be creating lots of regexps from similar components. This is what I usually do in Python. For example, while making no promises about the value of this code specifically:

    def any(x): return x+"*"
    def many(x): return x+"+"
    def capture(x): return "("+x+")"
    def lit(x): return re.escape(x) # shorthand
    bol="^"; eol="$"
    ident="[A-Za-z_][A-Za-z0-9_]*";
    
    find_function=(bol+any(" ")+lit("def")+many(" ")+
                   capture(ident)+any(" ")+lit(":")+
                   any(" ")+eol)

You can handle more or less stuff this way according to how much you and/or your readers like regexp syntaxp. The above is probably further than I'd take it in practice; I'm familiar with the regular expression syntax, but I'd still probably at least use something like the `ident' variable just to keep clutter out of the regexp.

(A nice demonstration of this sort of thing is emacs's rx module (see, e.g., http://emacswiki.org/emacs/rx). I couldn't find any good non-emacs documentation about this, nor much that would make sense to people unfamiliar with lisp, but when you're in emacs you can get help on it using C-h f rx RET.)

I agree. In terms of variables, I think a good strategy is assigning names to meanings, e.g.

    PRODUCT_ID = "[0-9]{2}[A-Z]{1,5}"
    ...
    CUSTOMER_ID = "[0-9]{2}[A-Z]{1,5}"
    ...
    ...
    regexp = "(" + PRODUCT_ID + ")" // capture product ID in group 1
           + "something something"  // something something
           + "(" + CUSTOMER_ID + ")"; // capture customer ID in group 2

In this example, even though the two constants contain the same regular expresison, they refer to two different concepts. Part of the problem with understanding regexp-based code is connecting parts of the expression with what they mean. Above strategy addresses this.

I also like assigning names to capture groups I depend on. So instead of, later in code, asking for e.g. matcher.group(2), I ask for matcher.group(GROUP_CUSTOMER_ID). Makes for a much more readable code.

Speaking of Emacs's rx, it's absolutely amazing and I'm sad that I only discovered it just few days ago :(. Another similar concept, from the other side of "code = data" equality is Common Lisp's (of course it's Lisp again) CL-PPCRE and its internal representation of regular expressions:

    * (parse-string "(ab)*")
    (:GREEDY-REPETITION 0 NIL (:REGISTER "ab"))
    
    * (parse-string "(a(b))")
    (:REGISTER (:SEQUENCE #\a (:REGISTER #\b)))
    
    * (parse-string "(?:abc){3,5}")
    (:GREEDY-REPETITION 3 5 (:GROUP "abc"))

You can encode any regexp you want as an S-expression, trading off conciseness for legibility. See http://weitz.de/cl-ppcre/#create-scanner2 for more.

Simple regexes are still fairly readable (e.g. a three number version string match), but if you find yourself having to worry about stuff like matching within escapable delimiters, it's time to put the regexes down and pick a different solution.

Matching within escapable delimiters is not too hard. For instance this will match C-style escaped strings:

    "(\.|[^"])*"

You just won't be able to nest them. You're either in a string or you aren't. You're never in a string inside another string.

I'm glad I'm not the only one who feels this way.

I feel like I gain more than I lose by simply excluding regexps from my toolbelt. (It's a personal choice, I'm not saying it'd be the same for others, since their values may differ from mine.)

I've found regexes very useful when writing quick parsers. For these cases it's often the simplest solution. If the alternative would be a huge chunk of hand written parsing code I would claim the regex is more maintainable.

This is a bit true, but when you want to get things done, one line regex is often better than a whole file of code.

Write only regexes are trivial once you have tests: when you don't understand one, throw it off and replace with something you do. Tests pass? Here you go.

Whereas having a whole file of many lines of code is much harder to replace when you don't understand what's it all about.

In general, yes. That's what I always document my regexes well and include write unit tests with all of the positive/negative test cases that I used whilst building the regex.

That makes them pretty maintainable.

Why go to this effort? Because for they're beautiful and a very powerful tool when used for the right problem. Unfortunately they're widely misused (i.e. email validation).

Unless it is not overly complicated, it is possible to write regular expression in such a way you can later on understand what you were thinking at the time you wrote them (using the extended syntax available in Perl and elsewhere - it allows you to break up the regex into multiple lines and embed comments).

It takes a little more effort, and one does not get the "look how dense it is"-rush, but I guess the same can be said for all code. There is a point where it breaks down, but there is a range of text-matching/extracting problems where regexes are both useful and - if one makes the effort - maintainable.

Reply on news.ycombinator.com