Comment by arthurofbabylon | Hacker News Reader

Comment by arthurofbabylon | Hacker News Reader

arthurofbabylon

Cognition works in two ways: 1) highly deliberate, slow, patient and 2) spontaneous, quick, reactive. I believe software languages do a great job of leveraging both of these capacities of the minds of programmers.

The first type of thinking is generally not composed in code. It happens staring out the window, or looking at data tables or lists and reading documentation, drawing out the architecture. The second type happens in code composition: programmers know what snippet will accomplish which outcome, and it just comes out when we know it's needed. Boom, we have the thing we imagined during the architecture phase. Maybe we look something up or slow down while writing code, but the composition is fairly reactive/spontaneous. It flows.

So coding goes in this rhythm... thinking openly and patiently and deliberately, followed by a burst of code composition, and then more open/deliberate thought while staring out a window. Again and again. Maybe we reverse some of the quick/spontaneous thinking to rearchitect according to our slow/deliberate thought.

Enter regular expressions. I have never, ever spontaneously composed a regex. I have to be very slow and deliberate. I have to recall the precise definition of every character in the regex as I read or write it. Sometimes I stare at a regex for 2 minutes, look up some operator, stare again for 2 minutes, refactor it. Finally I understand it and can now apply the regex spontaneously to whatever environment I'm working in. It always feels like first principles architecture, never like flow. This is odd, as regex is like a giant shorthand engine.

Maybe I have not yet formed the neurological connections required for it to flow – but more likely this is a function of the language. There must be a syntax that better supports flow-state composition. (I don't think Pomsky accomplishes it.)

The regular formalism is, fundamentally, about composition, and the current syntax doesn't do it justice.

That syntax was devised as a math notation, where sub-patterns were abstracted as one letter variables. You didn't have to deal with complex patterns at all, and it was very readable for that use case.

It was then re-purposed as a write-only language for searching at the CLI (in `ed` and its descendants, then grep).

They gradually graduated into what they are today in general programming languages because they were familiar and worse is better... However, composition got lost in the process.

Swift takes a radically different approach, and gives RegExps a parser combinator syntax. So every sub-expression can be assigned to a variable and reused/tested independently.

Here's an example (sorry I don't have it at hand in text form): https://pbs.twimg.com/media/FipgjUdUcAAl1xS?format=jpg&name=...

I wrote a lib that does the same in JS (the lib predates the Swift syntax by years). Here's the same example ported to JS: https://flems.io/#0=N4IgtglgJlA2CmIBcBWAzAOgAwCYA0IAzgMYBOA9...

Here's the lib: https://github.com/compose-regexp/compose-regexp.js

When I first encountered the Haskell Parsec library, the beauty of parser combinators hit me and I felt I'd never have to write a regular expression again.

This hasn't been the case because I had to work in other languages where usage of regexes was more idiomatic/commonly understood. Parser combinators are great but most of the time, the problem can easily be solved using a regex and we can't really justify adding yet another library in the dependencies for the rare case it would be better suited.

In addition to all the benefits they can provide in terms or readability, re-usability and abstraction, parser combinators can even compete with regexes in terms of performance. https://pl-rants.net/posts/regexes-and-combinators/

>Enter regular expressions. I have never, ever spontaneously composed a regex. I have to be very slow and deliberate. I have to recall the precise definition of every character in the regex as I read or write it. Sometimes I stare at a regex for 2 minutes, look up some operator, stare again for 2 minutes, refactor it. Finally I understand it and can now apply the regex spontaneously to whatever environment I'm working in. It always feels like first principles architecture, never like flow. This is odd, as regex is like a giant shorthand engine.

I can do simple regexes at pretty close to the speed of thought. e.g. here's one to recognize the "arthurofbabylon 20 minutes ago" bit at the top of HN comments:

    (-?\d+ points? by )?[a-zA-Z0-9_-]* ((just now)|(\d+ (minutes|hours|days|months) ago)|(on (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) \d{1,2}, \d{4}))

I typed that out pretty much all in one go, just had to look up different ways the HN date thing gets formatted. I put it in an online checker and it seemed to work on some examples, just had to fix a mismatched parentheses pair.

Not saying it's completely correct (probably someone will reply with a counterexample or problem with it) but I think it's a demonstration that regex fluency is possible. I do this kind of thing on the command line a lot, interactively to figure stuff out from logs and so on. For production code I'm more circumspect.

RegExp is optimized for typing, not for reading.

Trouble comes when one need to update or debug a complex RegExp. The one you posted is not that complex.

But isn't that true of most programs once they reach a certain level of complexity? Especially after they've been optimized? Eventually one relies on tests to ensure your changes don't introduce bugs and unwanted side effects. Sadly, most folks don't bother to write tests for their regexes.

Just to chime on among the naysayers, I just read your regex at "the speed of flow". Made perfect sense to me. (I'd have used non-matching groups, but that's a very minor nit.)

The equivalent code to parse this is much longer and arguably to me not really easier to grok. (Unless of course you regularly avoid regex, in which case it will always remain hard.)

I'd say the hardest thing for beginners is the character class and how it matches EXACTLY ONE LETTER without a modifier. Everything matches exactly one letter without a modifier. It seems intuitive to me, but obviously a lot of folks have trouble with it, so I accept that we are outliers.

Reminds me a bit of folks who steadfastly avoid SQL. You have to change the flow in your brain; there's more than one flavor of flow. For regex, it's the watching for single character match with optional modifier. For SQL, it's thinking in terms of sets, not instructions.

If regexes make you feel stupid, lean into it instead of recoiling from it. That's where the learning happens: immersion into the uncertainty until it becomes familiar.

>Reminds me a bit of folks who steadfastly avoid SQL. You have to change the flow in your brain; there's more than one flavor of flow. For regex, it's the watching for single character match with optional modifier. For SQL, it's thinking in terms of sets, not instructions.

I'm one of those people, SQL just doesn't fit in my brain right :(

Nothing logical ever does. The human brain is a pattern recognition engine. You just haven't had enough time with these particular patterns. It's like spoken language; it just takes time, exposure, and practice.

Just find the right strategy for you to internalize the patterns. Maybe a game?

https://mystery.knightlab.com/walkthrough.html

Reply on news.ycombinator.com