CSS Selectors vs XPath

  flowchart LR
  subgraph "both do this"
    A["by tag, class, id"]
    B["descendant and child"]
    C["attribute equals / contains"]
    D["nth child"]
  end
  subgraph "only XPath"
    E["match on text content"]
    F["walk UP to a parent"]
    G["preceding sibling"]
    H["arbitrary boolean logic"]
  end
  A --> I["prefer CSS -- shorter, and the same syntax as your stylesheets"]
  E --> J["prefer XPath -- 'the row whose label says Total'"]
  F --> J

CSS selectors

Great for:

• BeautifulSoup (select, select_one)
• Playwright
• browser devtools

Examples:

• .price
• div.card a
• #main h2.title

XPath

Great for:

• lxml
• Selenium (via XPath)
• complex DOM relationships

Examples:

• //div[@class='card']//a
• //table//tr[1]/td[2]

Quick comparison

• CSS is usually simpler
• XPath is more powerful for “go to parent/sibling” cases

Same page, both languages

Given this HTML, here is how each language picks the same elements.

<div class="card"><h2>Pen</h2><a href="/pen">details</a></div>
<div class="card"><h2>Book</h2><a href="/book">details</a></div>
<table><tr><td>Total</td><td>41</td></tr></table>

// means "anywhere below". [@class="card"] filters by attribute. A CSS class selector such as .card matches an element that has card among several classes, but the XPath @class="card" matches only when the whole attribute equals card. For several classes use contains(concat(" ", normalize-space(@class), " "), " card "), or just switch to CSS.

Using each in Python

BeautifulSoup uses CSS. lxml uses XPath, and so does Selenium's By.XPATH.

from bs4 import BeautifulSoup
from lxml import html
 
page = """
<div class="card"><h2>Pen</h2><a href="/pen">details</a></div>
<div class="card"><h2>Book</h2><a href="/book">details</a></div>
<table><tr><td>Total</td><td>41</td></tr></table>
"""
 
soup = BeautifulSoup(page, "html.parser")
print([a["href"] for a in soup.select("div.card a")])
 
tree = html.fromstring(page)
print(tree.xpath('//div[@class="card"]//a/@href'))
print(tree.xpath('//td[text()="Total"]/following-sibling::td/text()'))

Both first lines print the two links. The last line finds the value next to the label Total, which is the classic case where XPath is the only choice.

How to choose

• Start with CSS. It is shorter, and you already know the syntax from stylesheets and the browser's devtools.
• Move to XPath when you must match on text, walk up to a parent, or use a preceding sibling.
• Prefer stable hooks: an id, a data- attribute, or a meaningful class. Avoid selectors that depend on position, like div > div > div:nth-child(3), because they break when a banner is added.
• In Chrome devtools, right-click an element and choose Copy, then Copy selector or Copy XPath. Treat the result as a starting point, since it is usually far too specific.

Common mistakes

• Using a long absolute XPath like /html/body/div[2]/.... It breaks at the next redesign.
• Forgetting quotes inside quotes. Wrap the XPath in single quotes when it contains double quotes.
• Expecting exact class equality in CSS. .card is a "contains this class" test.
• XPath indexes start at 1, not 0.

Check your understanding