Regular Expressions Can Parse HTML (and More)
The true power of regular expressions (2012)
A 2012 deep dive argues that modern regex engines like PCRE are far more powerful than formal language theory suggests. By using recursive subpatterns and named groups, they can match context-free grammars—including HTML—and even transform BNF rules into working regexes. The author demonstrates this with an RFC 5322 email regex built directly from the grammar, proving that the common refrain "you can't parse HTML with regex" is misleading.
This statement - in the context of the question - is somewhere between very misleading and outright wrong.
- jakobnissen
This article misleads you by conflating regular expressions with specific implementations like PCRE, which also does non-regex string matches.
Annoyingly, the article does a good job of explaining what a regex is and what the limitations of regex are relative to PCRE, so the author should understand that what they are talking about when they talk about NP-complete string matching is not regex, but PCRE-specific features.
The distinction matters because regex absolutely can't match HTML, and because regex, unlike PCRE expressions, have guaranteed O(1) space and O(n) time complexity when matching a string of length n. When you use PCRE features for string matching, that may degrade to exponential time which makes it useless. For example, you can do denial of service PCRE attacks, but not denial of service regex attacks (unless you can query with some megabyte-large regex).
- AussieWog93
It might just be a me problem, but I've always been wary of regexes. They're not too bad to write, but reading them back and understanding what's actually going on can get a bit hairy. Plus, all of the subtle differences between regex libraries seems like a bit of a footgun.
Obviously they have their place, but I know a lot of the older guys seemed to love them way more than the young.
- cadamsdotcom
Some people, when confronted with a problem, think "I know, I'll get my agent to solve it with regular expressions."
Now they have three problems.
- HelloUsername
"Doom Using Regular Expressions" https://news.ycombinator.com/item?id=49094081
- lbriner
Something that seems obvious but not always implied by people's comments is that people are rarely trying to match an entire document with a regular expression so it doesn't really matter that "HTML is not a regular language".
If I am trying to e.g. count div tags with a regex like "<div" or whatever, then clearly this would work in 99.9% of cases and probably achieve what the poster is looking for.
As soon as you also add character classes to ignore various parts of the document that you are not interested in like "<div[^>]*>" or whatever it is, then it is eminently useful even if the bit we are ignoring is not fully regular.
One lovely thing about regex is how fast it is. I was asked to parse a massive CAN Bus log file for how many times some event had logged. This was the early 2000s and the file was 6GB, which was pretty big. I tried .Net's string.StartsWith or something and that took ages to run through the file. I did the same thing with a regex and it finished in like 5 seconds (HDD, not SSD!). I don't know how the magic works but it is very impressive.