Markdown-First Web Scraping: Cut LLM Token Costs by Up to 80%

Token prices keep falling, but token waste is a design bug, not a budget line. If you're feeding raw HTML to an LLM, you're paying for megabytes of <div>, class=, and inline styles that carry zero meaning.

The token math

A typical article page is around 40–80 KB of HTML. The same article as semantic markdown is roughly 5–10 KB. That is a 4–8× reduction in bytes, and because most of the removed content is markup symbols — not words — the token reduction is even more favorable.

# A real example from an average blog post
$ curl -s https://example.com/post | wc -c
      46120   # raw HTML bytes

$ curl -s -X POST https://api.apicall.co/v1/scrape \
    -H "Authorization: Bearer sk_live_YOUR_KEY" \
    -d '{"url":"https://example.com/post"}' \
  | jq -r '.data.markdown' | wc -c
       7421   # clean markdown bytes

Roughly 6× fewer bytes — and the token count drops even more because HTML tags tokenize inefficiently. In practice, markdown-first extraction cuts LLM token costs by 60–80% on typical article content.

Why tokens aren't the only win

The cost saving is the headline, but quality is the compounding return:

• Fewer hallucinations from noise. Navigation menus and cookie banners are not context; they're distraction. Clean markdown removes them.
• Better retrieval. Embeddings of clean text are more semantically coherent than embeddings of tag soup, so RAG recall improves.
• Lower latency. A 7 KB document embeds and indexes far faster than a 46 KB one — critical when an agent loop calls retrieval repeatedly.
• Cheaper to store. Vector indexes and caches hold the same knowledge in a fraction of the memory.

What markdown-first extraction should return

A production extraction layer is more than a converter. The right contract looks like this:

1. 1.Semantic markdown — headings, lists, links, and code blocks preserved; boilerplate and scripts stripped.
2. 2.Metadata — title, author, and publication date parsed from meta tags and JSON-LD, not guessed.
3. 3.Links — deduplicated absolute URLs, with an option to filter social/share links.
4. 4.Content mode — a main-content flag that drops sidebars, footers, and related-article noise.

The cheapest token is the one you never send. Markdown-first ingestion is the single highest-ROI optimization available to every LLM app in 2026.