I believe the point here is exactly that nothing will change, because the regex in question isn't for a service, it is just for that specific file, right now, today. I've regexed specific HTML files myself too, because even though I am very comfortable with XPath and Beautiful Soup and tree representations in general, regexes are even easier on a static file like that.
Three or four years later, I was having lunch with the guy I'd done that for when he got a phone call. Turns out they were using that program of mine to print labels monthly now, and one of the completely-safe assumptions I'd made years earlier had bitten them. Fortunately, he knew how to resolve it easily, but I learned a very important lesson that day.
From my perspective, this other guy was using a program outside of its design spec. That can be fine, but such a thing is prone to the problem of safe assumptions suddenly not being safe. You wrote the program for the problem you needed it for, and for which, it worked fine. Unless part of the problem at the time was "we expect to use this program for a couple years", there is no reason to make it overly robust.
Code designed to be used once has a habit of sticking around.
The only thing regex has going for it is you might already know it. But, this is one of those time when learning something is worth it.
PS: XP is doing a depth first search, which can be EXtreamly wasteful.
For re-usable tooling though, after some time you tend to avoid design patterns that you've personal witnessed break down repeatedly and cause issues.
Well, we are, because to quote coldtea, the person you directly replied to, "'Don't have a continuously running service parse arbitrary HTML with regular expressions' would be a bad thing. Parsing specific, given, HTML files, with known structure to the dev, with regular expressions (e.g. as part of a one-off scrapping script) is totally, absolutely, fine." In context the first sentence clearly means that running the service like that would be a bad thing (hooray English and it's deep ambiguities in double negatives).
I still wouldn't necessarily be too upset about telling a junior dev to be suspicious of REs or avoid them (for instance, parsing HTML with REs typically requires using non-greedy matches, and there are some subtleties around that), but if you know what you're doing on a quick job it's fine. If you're a senior dev and you still haven't figured out what's likely to entrench itself and what really is a one-off job, well, that's your real problem. I haven't been surprised about what gets entrenched in quite a while.