Built-in Scrapers vs LLM Schema ExtractionWhat 50 Product Pages, Run Three Times, Say About Structured Data at Scale

An empty page capture should produce an extraction failure. In our test it produced a Mainstays 5-Piece Dining Set priced at $149, with 4.5 stars and 120 reviews. The Walmart URL actually sold a $59 electric multi-cooker. The model's fabricated record passed our JSON Schema, while the built-in Walmart scraper returned the multi-cooker in all three runs.

That example captures a problem with structured data extraction: well-formed JSON can still describe the wrong product. To measure the difference, we tested three paths on 25 Amazon and 25 Walmart product pages: a named Crawlbase scraper, autoparse=true, and Markdown followed by a strict JSON Schema call to GPT-4o-mini. Each path ran three times. We measured field coverage, repeatability, values absent from the captured text, latency, and request and model-token counts.

The companion repository contains the implementation, nine walkthrough scripts, offline fixtures, and tests. Its URL list defines the candidates and reserves, and the recorded results retain the measurements from the November 2, 2026 runs.

Choose how the page becomes JSON

The Crawling API supports both retrieval and maintained extraction. For an Amazon scraper API workflow, sending scraper=amazon-product-details returns product JSON. Walmart uses scraper=walmart-product-details. With autoparse=true, the API detects a supported page type and applies the corresponding scraper.

The initial probes returned the same product object through autoparse and the named scraper. When no supported scraper matches, Crawlbase falls back to the generic-extractor, which returns general page data. These are still separate requests: the later benchmark includes differences in returned values and timeouts. Deterministic extraction describes the parsing approach; it does not make a live retailer page or every response immutable.

The LLM route retrieves the page with format=md, then sends the Markdown, product URL, instructions, and an eight-field schema to OpenAI. On an uncached successful attempt, that adds one model call to the crawl. The AI data extraction overview explains this fetch-then-extract pattern; here, we compare it with the maintained product parsers using the same data extraction API for acquisition.

A probe also tested scraper= together with format=md on Amazon. The response contained scraper JSON only. The benchmark therefore fetches parsed data and Markdown separately. That matters when interpreting its evidence checks: the model is checked against its own captured input, while each parser is checked against another crawl of the same URL.

Define one record before comparing extractors

The three paths need a common representation before their outputs can be compared. The shared schema and mappings define eight fields: title, id, brand, price, currency, availability, rating, and reviews_count.

The live scraper responses used different keys:

These observed keys differed from the documented names used during development, so the mappings follow the recorded payloads. Walmart availability remains null when no stock field is supplied. The mapper also handles outOfStock or availability if either appears in a response.

LLM schema extraction uses a separate output contract. Every key must be present, but every value can be JSON null. The prompt says to use only values printed on the page, and each field description tells the model to return null when the value is missing:

MAX_MARKDOWN_CHARS = 60_000

SYSTEM_PROMPT = (
    "You extract one product record from a retail product page converted to Markdown. "
    "Use only values printed on the page. If a value is not on the page, return null. "
    "Do not guess, convert, or fill in values from general knowledge."
)

The LLM client requests strict structured output at temperature 0 and sends the first 60,000 Markdown characters:

response = self.client.chat.completions.create(
    model=self.model,
    temperature=0,
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"URL: {url}\n\n{page}"},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "product_v1", "strict": True, "schema": LLM_SCHEMA},
    },
)

After extraction, the application normalises the result and validates it against product.v1. That acceptance schema requires a title and ID of at least three characters, nonnegative prices and counts, ratings from 0 to 5, and an allowed stock value or null. A numeric price requires a currency string matching three uppercase letters:

PRODUCT_SCHEMA = {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": SCHEMA_VERSION,
    "type": "object",
    "required": FIELDS + ["_meta"],
    "additionalProperties": False,
    "properties": {
        "title": {"type": "string", "minLength": 3},
        "id": {"type": "string", "minLength": 3},
        "brand": {"type": ["string", "null"]},
        "price": {"type": ["number", "null"], "minimum": 0},
        "currency": {"type": ["string", "null"], "pattern": "^[A-Z]{3}$"},
        "availability": {"enum": AVAILABILITY + [None]},
        "rating": {"type": ["number", "null"], "minimum": 0, "maximum": 5},
        "reviews_count": {"type": ["integer", "null"], "minimum": 0},
        "_meta": {
            "type": "object",
            "required": ["url", "path", "extractor", "schema_version"],
            "properties": {
                "url": {"type": "string"},
                "path": {"enum": ["scraper", "autoparse", "markdown_llm"]},
                "extractor": {"type": "string"},
                "schema_version": {"const": SCHEMA_VERSION},
            },
        },
    },
    "allOf": [
        {"if": {"properties": {"price": {"type": "number"}}, "required": ["price"]},
         "then": {"properties": {"currency": {"type": "string"}}}},
    ],
}

The code normalises familiar currency symbols, but this pattern alone does not verify membership in the ISO 4217 currency list. The distinction between these two schemas explains the results: the model can return a complete, structurally valid nullable object that the application later rejects because its title is missing.

Compare 50 pages across three runs

The benchmark harness starts with 25 Amazon /dp/ URLs, 25 Walmart /ip/ URLs, and reserves. Each selected URL's first-run Markdown must exceed 1,500 characters and contain its product ID. This applies the same entry rule before measuring the three paths, although it also means the dataset begins with pages that yielded usable Markdown.

Selection replaced 28 candidates. Amazon contributed seven: six 404 pages and one unavailable item whose page did not name its ASIN. Walmart contributed 21: six bot checks, ten consent walls, two empty bodies, two not-found pages, and one page that failed the entry rule. Five Walmart candidates and all Walmart reserves came from one organic walmart-serp listing for "kitchen essentials".

The three runs took place back to back on November 2, 2026. The published matrix contains 450 path results: 50 URLs multiplied by three paths and three runs. Of those retained results, 412 used the Normal token and 38 used the JavaScript token after a retry. These totals describe recorded outcomes; URL-selection requests and additional retry attempts add to the underlying network work.

The Python requirements pin Crawlbase SDK 1.1.0. The recorded LLM calls requested gpt-4o-mini and returned gpt-4o-mini-2024-07-18.

The crawl client sets a 120-second client timeout. When a JavaScript token is configured, it permits one switch after cb_status 207 or 503, or a successful response containing a recognised challenge, an almost empty Markdown body, or a parser result without a title. Original 404 and 410 responses are excluded from that retry. Transport exceptions are recorded as failures rather than retried by this client:

TIMEOUT_S = 120
RETRY_ON_JS = {"207", "503"}
# A 200 from the API can still carry an interstitial instead of the product page.
CHALLENGE_MARKERS = ("robot or human", "health information consent", "enter the characters you see below")
MIN_MARKDOWN_CHARS = 40


def unusable(result: "CrawlResult") -> bool:
    """Same rule for every path: no product content in an otherwise successful response."""
    if result.cb_status != "200" or result.error:
        return False
    if result.path == "markdown":
        text = (result.markdown or "").lower()
        return len(text) < MIN_MARKDOWN_CHARS or any(m in text for m in CHALLENGE_MARKERS)
    body = result.data.get("body") if isinstance((result.data or {}).get("body"), dict) else result.data
    return not (isinstance(body, dict) and (body.get("name") or body.get("title")))

The metrics module measures several different properties. Fill rate is the mean non-null coverage across the eight fields. Attempts that produce no record contribute zero filled fields; records rejected by the application schema still contribute any non-null fields. Consistency requires the normalised eight-field record to match across all three runs; metadata is excluded. Schema acceptance counts records that pass product.v1, and latency covers the crawl duration (both attempts when a JavaScript retry happened) plus the model duration for the LLM path.

For the page-text check, the LLM's values are compared with the Markdown prefix it received, with the URL also available as evidence for the product ID. Parser values are compared with that run's separate Markdown crawl, and the check is skipped when that capture fails the product-page rule. The benchmark deliberately disables the router's Markdown content gate so it can measure what the model does with bot checks and empty pages.

Read the measured results

The two parser paths passed the schema on 297 of 300 attempts without model tokens. All three named-scraper failures were Walmart calls that hit the client timeout. The LLM returned an object on every attempt, but normalisation and application validation accepted 130 of the 150 records.

Walmart showed the largest separation. Across its 75 Markdown captures, 21 contained a bot check, consent wall, or empty body. All 21 occurred among the 50 captures in runs two and three, after the first-run selection had admitted the URLs. The model still produced an object for each one. Fifteen titles were the string "null", and 16 responses supplied in_stock without supporting stock text. The fabricated dining-set record from the introduction was the one Walmart non-product capture that still passed the application schema after normalisation.

Amazon requires a different reading. The LLM had slightly higher field coverage, and its page-text check flagged fewer values than either parser. In 27 of the named scraper's 36 flagged values, the model returned null for that field when reading the same run's Markdown. The recorded captures included listing-style layouts without a buy box, so some discrepancies reflected different content returned by separate crawls of one URL.

Price, rating, and review count changed across runs on 11 or 12 Amazon URLs for each parser path, and on 7 to 11 URLs for the model path. Model titles changed on five Amazon URLs; parser titles changed on none. These observations show why consistency needs to be interpreted alongside captured inputs. No URL's full Markdown was byte-identical across all three runs, so the experiment did not isolate model randomness from page variation. Among the nine Walmart URLs with product-page Markdown in every run, the LLM record was identical for two and the named scraper record for seven.

The lower LLM latency on Walmart also includes short interstitial and empty inputs. A faster failed input is not evidence of faster successful product extraction. Likewise, the reported median is per attempt, rather than time per accepted record or total batch throughput.

Check what the returned values actually mean

The page-text checker looks for normalised evidence of each value. It catches both model inventions and parser discrepancies. On one Walmart page, the named scraper returned reviewsCount as "122" while the page showed 12,292; autoparse returned "12292". On another page, the two paths swapped outcomes. Field coverage alone counted the incorrect count as filled.

def is_supported(field: str, value, text: str) -> bool:
    if value is None:
        return True
    if field in ("title", "brand", "id"):
        return _has_text(text, str(value))
    if field == "price":
        return _has_number(text, float(value))
    if field == "reviews_count":
        return _has_number(text, float(value)) or _has_abbreviated(text, float(value))
    if field == "rating":
        return _has_rating(text, float(value))
    if field == "currency":
        return any(cue in text for cue in CURRENCY_CUES.get(value, (value.lower(),)))
    if field == "availability":
        cues = IN_STOCK_CUES if value == "in_stock" else OUT_OF_STOCK_CUES
        return any(cue in text for cue in cues)
    return True


def invented_fields(record: dict, markdown: str, fields: list[str], url: str = "") -> list[str]:
    """The URL counts as input: an id taken from it is not invented."""
    text = page_text(markdown) + " " + url.lower()
    return [f for f in fields if record.get(f) is not None and not is_supported(f, record[f], text)]

A hand-check of three Amazon and two Walmart pages led to four refinements. Abbreviated counts are accepted at their displayed precision: 27,406 matches "27.4K", and 14,084 matches "14K". Text matching ignores spacing, allowing "HeroCosmetics" to match "Hero Cosmetics". Walmart's "Pickup as soon as 3pm" counts as an in-stock cue. Walmart ratings are excluded from support checking across all paths because the captured Markdown showed other products' ratings rather than the item's own rating.

That exclusion does not remove ratings from the fill-rate calculation or the non-null totals used as the evidence-rate denominator. The reported counts and chart percentages should therefore be read as this harness's text-support measurements, rather than an accuracy score over an identically checked set of fields.

Two hand-checked disagreements illustrate the remaining limitation. The checker accepted out_of_stock when only an open-box offer was unavailable, and accepted a review count of 30 because an unrelated "30" occurred elsewhere on the page. Text presence establishes that a value appears somewhere in the input; associating it with the correct product, offer, and field requires context.

Strict output solved a different problem. The 150 model calls produced no JSON parse errors or partial objects, but strict structure did not prevent unsupported stock values or a fabricated product. Requiring every key still permits an honest JSON null. The failure occurred when the model filled a field without evidence, not because the schema required a non-null answer.

For related schema-driven workflows, Firecrawl Extract provides an extraction interface whose documentation now introduces /agent as its successor. Its JSON mode guidance recommends explicit null handling, focused prompts, smaller schemas when field counts reach roughly 30 or more, and removing array minItems and maxItems constraints, which do not control how much content is collected. The measured LLM path in this article is a direct OpenAI call over Crawlbase Markdown.

The issue reports also show why output failures need separate diagnosis. Firecrawl reports #1836 and #3300 concern Zod-to-JSON-Schema handling that produced empty or missing data. Report #2491 concerns a self-hosted model integration that generated no object. These examples describe schema and integration failures; they do not measure hallucination frequency or establish that strict mode would fix them.

Normalise and validate before falling back

Once the failure modes are visible, the routing rule becomes straightforward: try the supported parser, normalise its output, and accept it only if it meets the application's contract. The normaliser converts placeholder strings such as "null", "none", and "n/a" to null, parses prices and abbreviated counts, cleans brand labels, and maps stock text to in_stock, out_of_stock, or null.

Converting "null" titles to real null values turned 15 bogus titles into application-schema rejections. The normaliser also clears currency when price is absent. Its supported number formats cover the examples in this experiment, including $1,299.99 and €39,90; additional locales need explicit parsing rules.

The packaged router returns the first accepted result:

def route(url: str, crawl: CrawlClient, llm: LLMExtractor, run: int = 1) -> Extraction:
    """First accepted record wins. A rejected record falls through to the next path."""
    attempts = []
    if scraper_for(url):
        attempts.append(lambda: via_scraper(url, crawl, run))
    attempts.append(lambda: via_autoparse(url, crawl, run))
    attempts.append(lambda: via_markdown_llm(url, crawl, llm, run))
    result = None
    for attempt in attempts:
        result = attempt()
        if result.accepted:
            return result
    return result

Each via_* function returns an Extraction object. Successful extraction is followed by normalisation, metadata stamping, and JSON Schema validation. Here, accepted means that a record exists and has no validation errors. If all paths fail, the router returns the final rejected result, and write_record() writes nothing.

Before a model call, via_markdown_llm rejects Markdown shorter than 40 characters or containing one of three recognised challenge phrases. It returns not_a_product_page for those cases:

def via_markdown_llm(url: str, crawl: CrawlClient, llm: LLMExtractor, run: int = 1, gate: bool = True) -> Extraction:
    res = crawl.fetch(url, "markdown", run)
    extractor = f"openai:{llm.model}"
    if not res.ok:
        return Extraction(url, "markdown_llm", extractor, None, [res.error or f"cb_status {res.cb_status}"], res.to_dict())
    if gate and unusable(res):
        # A 200 that carries a challenge or an empty body never reaches the model.
        return Extraction(url, "markdown_llm", extractor, None, ["not_a_product_page"], res.to_dict())
    out = llm.extract(url, res.markdown, run)
    if not out.ok:
        return Extraction(url, "markdown_llm", extractor, None, [out.error], res.to_dict(), out.to_dict())
    return _finish(url, "markdown_llm", f"openai:{out.model or llm.model}", out.data, res.to_dict(), out.to_dict())

Applied to the recorded Walmart captures, this gate would have kept all 21 unusable pages away from the model. It is a targeted content check, rather than a complete detector for every interstitial or wrong-page response.

The router's acceptance decision does not run the benchmark's page-text checker. Add contextual evidence checks to your own acceptance policy when a schema-valid but incorrect price or stock value would affect downstream decisions. This keeps structural validation and factual checking as explicit implementation steps.

Accepted records include _meta.url, _meta.path, _meta.extractor, and _meta.schema_version. Examples include crawlbase:amazon-product-details, openai:gpt-4o-mini-2024-07-18, and product.v1. These identify the route and model or scraper name; the scraper name alone does not identify a particular parser revision.

The walkthrough scripts expose the individual built-in scraper call, autoparse call, LLM extraction, and three-run comparison.

Token usage and caching

The disk cache makes repeated analysis possible without repeating the same acquisition. Crawl entries are keyed by path, run number, and URL. Model entries include the requested model, run number, URL, and full Markdown hash. Neither cache expires automatically, and the model key does not include the prompt or schema. Clear the relevant entries before measuring a revised extraction contract or obtaining fresh pages.

The recorded results were recomputed from saved responses. Zero API calls in that invocation's metadata describe the replay, rather than the original experiment. Recomputing those measurements requires the saved response cache; the published metrics file alone does not contain the raw captures.

Input size remains a separate scaling cost. The 150 model calls consumed 1,793,923 prompt tokens and 9,415 completion tokens, or 1,803,338 combined. Amazon averaged 22,131 total tokens per call; Walmart averaged 1,914. Forty-nine of the 75 Amazon calls exceeded the 60,000-character Markdown cap and were truncated. Raising that cap sends more context; lowering it risks removing fields near the end of a page.

The diagram is illustrative. With a fixed average input size, both approaches grow linearly with page count; the model path adds per-page token processing. Its steeper line does not specify a measured price or speed ratio. The benchmark counts logical crawl attempts and model tokens, while actual acquisition costs also depend on retries, domain complexity, rendering, and storage.

Firecrawl's v2.11.0 release describes another way to reduce repeated model work: deterministicJson generates a reusable extractor and caches it per site. That mechanism fits the same engineering goal of amortising extraction work across repeated pages, though it was not a measured path in this experiment.

Queue collection and retrieve results by RID

For larger batches, the acquisition step can finish independently of the process submitting URLs. The async walkthrough sends async=true&store=true, receives a request ID immediately, and later retrieves the result from Cloud Storage:

def submit(api, url: str) -> str | None:
    from urllib.parse import urlparse

    options = {"async": "true", "store": "true"}
    name = SCRAPERS.get(urlparse(url).hostname or "")
    options.update({"scraper": name} if name else {"autoparse": "true"})
    res = api.get(url, options)
    payload = res.get("json")
    if not isinstance(payload, dict):
        try:
            payload = json.loads(res.get("body") or b"{}")
        except ValueError:
            payload = {}
    return payload.get("rid")  # the SDK puts the async rid in res["json"], not res["rid"]


def fetch_stored(storage, rid: str, attempts: int = 12, wait_s: float = 10.0) -> dict | None:
    for _ in range(attempts):
        try:
            res = storage.get(rid, {"format": "json"})
        except KeyError:  # crawlbase 1.1.0 raises this while the rid is not stored yet
            res = {}
        if res.get("status_code") == 200:
            stored = res.get("json_body")
            if isinstance(stored, dict) and stored:
                return stored
            body = res.get("body") or b""
            if body:
                return {"body": body.decode("utf-8", "replace") if isinstance(body, bytes) else body}
        time.sleep(wait_s)
    return None

With SDK 1.1.0, the asynchronous ID is available at res["json"]["rid"]. The walkthrough polls by RID, treating the SDK's observed KeyError for a not-yet-available item as a pending result. In the two-URL test, the Amazon result was readable within two minutes. Walmart was absent on the first two checks and present on a later check. These are recorded completion observations, rather than a guaranteed delivery interval.

The batch helpers also accept a callback URL. Crawlbase POSTs the completed result to that endpoint, while store=true retains a copy for retrieval. The Zapier callbacks walkthrough demonstrates that delivery pattern. The async example maps and normalises retrieved payloads; apply the same application validation policy before persisting them as accepted product records.

Storing a page adds 0.5 credits to the crawl that fetched it, as listed on the pricing page, and the page is kept for 30 days. Retrieval is free. Include storage in your budget alongside the crawl and model work.

Billing also depends on the response status. The Crawling API reference lists billable requests as cb_status 200 or 207 with original_status 200, 201, 301, 302, 303, 307, 308, 404, or 410. A recognised 503 block is not billed. The bot-check responses in this experiment arrived as successful 200s and were billed as such, which is why body inspection remained necessary even after checking the status.

The API documentation recommends asynchronous delivery for batches over a few hundred URLs and the Crawler for high-volume jobs such as millions of URLs. For custom throughput and service requirements, the enterprise offering provides the next operational step.

Use the model where interpretation adds value

A maintained parser is a practical starting point for supported product pages. It removes the per-page model call and exposes a known set of fields. The experiment also shows the tradeoff: Walmart stock was unavailable through the tested parser, and incorrect review counts still required checking. Choose the extraction route against the fields your application actually needs.

A model is useful for shops without a matching parser, irregular documents such as forum threads and press releases, or one-off research where building a dedicated extractor would cost more than the model calls. Those documents need their own schema and mappings; this router implements a product-specific contract. The ChatGPT, Gemini, and Perplexity tutorials provide related fetch-then-interpret implementations, while the Amazon price extraction tutorial applies the pattern to retail data.

Models can also work after deterministic extraction. A product record can support classification into a custom taxonomy or candidate matching between Walmart and Amazon. Complaint tagging needs review text, so supply that content from an appropriate scraper or capture as well. Each task can receive the fields it needs instead of the entire page.

For recurring price intelligence, collect and normalise the price first, then add model interpretation where needed. The scheduled Amazon scraping guide covers recurring acquisition. Keeping acquisition, field extraction, validation, and interpretation explicit makes web scraping to JSON easier to inspect when a value changes.

Conclusion

On these 50 product pages, the maintained scraper paths produced 297 schema-accepted records from 300 attempts without model tokens. Markdown plus GPT-4o-mini offered higher field coverage on Amazon, but Walmart interstitials exposed the risk of accepting plausible records from unusable inputs. The recorded results support starting with an available parser, checking the returned fields, and reserving model extraction for the gaps your application needs to fill.

The full implementation provides the mappings, router, benchmark, and offline examples. To try it against your own catalogue, create a Crawlbase account, configure the tokens, and inspect a small set of accepted records before expanding the run. The Amazon scraper, Walmart scraper, and broader e-commerce scraping overview help identify the acquisition route for your targets.

Frequently Asked Questions (FAQs)

Which sites have built-in Crawlbase scrapers?

The scrapers index lists supported extractors for Amazon, Walmart, eBay, AliExpress, Best Buy, Google, Bing, Facebook, Instagram, TikTok, LinkedIn, Reddit, Booking.com, Airbnb, and other sites, plus generic extractors. Choose the scraper for the page type and send its name through scraper=. The example domain router maps only Amazon and Walmart product pages; extend it for additional targets.

Does autoparse work on any page?

With autoparse=true, Crawlbase detects the page type and applies a matching supported scraper. When no supported scraper matches, it falls back to the generic-extractor, which returns general page data rather than a retailer-specific product record. In this repository, the application maps the returned JSON to its product schema and validates it. If that process does not produce an acceptable product record, the application makes a separate Markdown request and attempts LLM extraction. This additional fallback belongs to the application's routing logic.

How often do built-in scrapers break, and who maintains them?

Crawlbase maintains its built-in scrapers and updates them when website changes affect extraction. Breakage is reported automatically and is usually fixed within hours. If you notice missing fields, incorrect values, or an issue that persists, contact Crawlbase Support with the affected URL and an example response so the team can investigate.

Does JSON Schema validation guarantee accurate extracted data?

No. JSON Schema checks the record's structure, required fields, and value constraints. It does not verify that the values describe the correct product. Check that the captured page contains usable product content, then verify important values against evidence associated with the correct product and offer.

IK

Ian Kalvin

Technical Support Engineer · Crawlbase

Technical support engineer at Crawlbase, writing from the front line of what actually breaks in production scraping and proxy setups.

NZ

Neil Zamora

Senior Architect · Crawlbase

Senior architect at Crawlbase, focused on the systems behind large-scale crawling: proxy rotation, anti-bot resilience, and the APIs that hide that complexity.