Saltar al contenido ForoHackers

Parsing HTML with regex, a short tragedy in one act

Autómatas y lenguajes formalesPythonEstándares web
#958 · regex_regret · · Denunciar

Every few months someone asks: "I just need to grab the links from this page, can I use a regex?"

You can, and it will work on the page in front of you. Then the site adds a newline inside a tag, or uses single quotes, or puts href after class, or the string appears inside a comment. Your regex now returns confidently wrong results, which is the worst kind.

HTML is not a regular language, so a regular expression cannot match it in general. Use a parser. The standard library has one:

from html.parser import HTMLParser

class Links(HTMLParser):
    def __init__(self):
        super().__init__()
        self.hrefs = []

    def handle_starttag(self, tag, attrs):
        if tag == "a":
            for name, value in attrs:
                if name == "href" and value:
                    self.hrefs.append(value)

p = Links()
p.feed('<a class="x" href="/a">one</a> <a href=\'/b\'>two</a>')
print(p.hrefs)   # ['/a', '/b']

Handles both quote styles, attribute order, and ignores comments. A dozen lines. No regret.

#1259 · tabs_over_spaces_42 · · Denunciar

But my HTML is very regular! I wrote it myself, it always looks exactly like this.

(I know. I know. I will go and sit in the corner.)

#1260 · regex_regret · · Denunciar

"I wrote it myself" is the first line of every post-mortem. Sit in the corner, but bring a parser.

Inicia sesión o únete para responder.