Cookbook·Automotive·autotrader.com

Scrape product pages on autotrader.com

The call that returns product pages from autotrader.com today, with the parameters, patterns and failure modes read off the request log since 14 Aug 2026. Last tested 6 Sep 2026.

99.3% success · Aug 2026Browser neededStandard price · 1 credit

About autotrader.com

autotrader.com describes itself as “Explore new and used cars, trucks and SUVs with confidence. Autotrader is the one-stop shop for everything you need in your car buying experience including expert advice, instant cash offers, an affordability calculator, and quality car news and reviews”.

Its sitemaps list 3,645 URLs across 5 files (main, makes, srp-geo-main, sell-car).

Its robots.txt disallows 12 paths, among them /research/.

Measured on Crawlbase

Success rate over every request from every account in August 2026: 99.3%. Patterns, countries and failures come from the request log since 14 Aug 2026; 99.7% of successful calls used the JavaScript token and the median answer took 27.4s.

Among the 16 automotive sites in the Cookbook, autotrader.com ranks 7 of 16 by success rate; the category median is 99.3%. Its median answer of 27.4s is slower than the category median of 1.9s. 12 of the 16 accept the plain token; this one needs the JavaScript token.

The call

Product pages are the shape the accounts fetch most here, with the country and token that succeed most often. Replace the placeholder path with a real product URL from the site.

Run in Playground

curl "https://api.crawlbase.com/?token=YOUR_JS_TOKEN&country=US&url=https%3A%2F%2Fwww.autotrader.com%2Fexample-section%2F1234567%2Fexample-page"

from crawlbase import CrawlingAPI
api = CrawlingAPI({'token': 'YOUR_JS_TOKEN'})

r = api.get('https://www.autotrader.com/example-section/1234567/example-page', {'country': 'US'})
html = r['body']

const { CrawlingAPI } = require('crawlbase');
const api = new CrawlingAPI({ token: 'YOUR_JS_TOKEN' });

const r = await api.get('https://www.autotrader.com/example-section/1234567/example-page', { country: 'US' });

Parameters that matter

Parameter	Set it to	Why
`country`	`US`	99.1% of successful calls set it and they succeed at 99.6%; calls without a country succeed at 22.4%.
`javascript`	use the JavaScript token	JavaScript-token calls succeed at 99.6% against 24.1% for the plain token, and 99.1% of calls use it.

URL patterns accounts fetch

Pattern	Share of successes	JavaScript token	Country	Query parameters
`/{slug}/{id}/{slug}`	98.3%	100%	US	none

Fields you get

Structured data Crawlbase found on autotrader.com, by page type. These are the schema.org types in the page, the fields your parser can read straight from the JSON-LD block:

Home: Organization
Other: WebPage, BreadcrumbList, WebSite, Organization

What breaks, and the fix

Of the failures since 14 Aug 2026, the site answered:

403: 44.2% of failures A 403 is the site refusing the exit. Set the country the successful calls use and retry once; a second 403 means the URL itself is protected.

200: 11.0% of failures The site answered 200 with a block or consent page instead of the content, and Crawlbase counted it as a failure rather than bill you for it. Retry once with the JavaScript token and the country the successful calls use.

0: 5.9% of failures A status of 0 is a request that never got an answer inside the timeout. Retry once with the JavaScript token if the pattern needs a browser.

504: 5.1% of failures A 504 is a timeout: the page did not finish inside the limit. Retry once, and keep your own client timeout above the median answer time shown for this site.

591: 30.6% of failures A 5xx is the site failing, not the fetch. Retry after a minute.

Run it on a schedule

A catalogue refresh is a list of product URLs. Push it to the Crawler with a callback, let it pace the calls and retry what the site refuses, and read the results from Cloud Storage. The median answer on this site is 27.4s.

Questions

Do I need a browser to scrape autotrader.com?

Yes for most pages. JavaScript-token calls succeed at 99.6% while plain-token calls reach 24.1%.

How much does a autotrader.com page cost on Crawlbase?

1 credit without a browser. autotrader.com is in the standard tier.

Which country should I set for autotrader.com?

US: that is what 99.1% of successful calls use.

Other pages on autotrader.com

Pages autotrader.com lists in its own sitemaps, fetched by Crawlbase on 6 Sep 2026 and verified in 2 consecutive runs. Each row is the cheapest call that returned the page with its content.

Page	Example	Token	Country	Median
Home `/` Structured data: Organization	autotrader.com/	Plain	none	22.0s
Other `/{slug}/{slug}` Structured data: WebPage, BreadcrumbList, WebSite, Organization	none	Plain	none	2.6s

Measured on CrawlbaseAUGUST 2026

Success rate99.3%

Needs browserYes

Price tierStandard · 1 credit

Last retest6 Sep 2026

How we count: every request from every account in the calendar month; success means Crawlbase answered with cb_status 200 and the site answered with a normal status. No sampling, no exclusions. Method

Try it

Run this recipe in the Playground with your token. Up to 5,000 free requests, no card.

Open in Playground

Run it at scale

A page of autotrader.com costs 1 credit, 2 credits with the JavaScript token this recipe needs. Pay as you go needs no plan: 1,000,000 pages come to about $1,055, on a ladder that falls from $6 to $0.04 per 1,000 requests as volume grows, and you start with up to 5,000 free requests. Starter covers 100,000 pages a month for $99.

Starter · $99/mo All plans

Keep it fresh

Push your URL list to the Crawler with a cadence and a callback. It paces the requests, retries what the site refuses, and delivers every page to your endpoint or Cloud Storage.

See the Crawler

Report a break in this recipe·Suggest a site

Related recipes

Site names identify the pages a recipe fetches. Crawlbase is not affiliated with any site listed. Use recipes within the acceptable use policy.Measured numbers from August 2026, refreshed monthly