Scraping cookbook · Ecommerce
Scraping ecommerce sites
20 sites in this category, measured on Crawlbase traffic in August 2026. Each page carries the call that works and the numbers behind it.
General retailers with large catalogues, walmart.com, target.com, kohls.com and sears.com among them. This is where anti-bot tiers cluster: several sites price above the standard tier, and about half need the JavaScript token for search and category pages. 9 of the 20 sites answer the plain token. Prices run from 1 credit to 4.5 credits a page.
kohls.comEcommerce
Working call, parameters that matter, the fields you get and what breaks.
Success · Aug99.0%
BrowserNo
Price2 credits
walmart.comEcommerce
92.1% of calls to walmart.com succeed. product pages, JavaScript token, median answer 12.2s.
Success · Aug92.1%
Browser19% of calls
Price4.5 credits
target.comEcommerce
How to fetch product pages from target.com: no browser needed, 96.4% success, one call per page.
Success · Aug96.4%
BrowserNo
Price1 credit
costco.comEcommerce
Product pages on costco.com: JavaScript token, 90.8% success in Aug 2026.
Success · Aug90.8%
Browser71% of calls
Price1 credit
staples.comEcommerce
How to fetch product pages from staples.com: no browser needed, 92.9% success, one call per page.
Success · Aug92.9%
Browser21% of calls
Price1 credit
macys.comEcommerce
94.0% of calls to macys.com succeed. category pages, JavaScript token, median answer 15.2s.
Success · Aug94.0%
Browser23% of calls
Price1 credit
lego.comEcommerce
Category pages on lego.com: no browser needed, DE exit, 94.9% success in Aug 2026.
Success · Aug94.9%
BrowserNo
Price1 credit
woolworths.co.zaEcommerce
Product pages on woolworths.co.za: no browser needed, 99.9% success in Aug 2026.
Success · Aug99.9%
Browser90% of calls
Price1 credit
elcorteingles.esEcommerce
Product pages on elcorteingles.es: JavaScript token, ES exit, 93.0% success in Aug 2026.
Success · Aug93.0%
Browser82% of calls
Price1 credit
sears.comEcommerce
How to fetch product pages from sears.com: JavaScript token, US exit, 94.2% success, one call per page.
Success · Aug94.2%
BrowserYes
Price1 credit
jcpenney.comEcommerce
How to fetch product pages from jcpenney.com: no browser needed, US exit, 94.1% success, one call per page.
Success · Aug94.1%
Browser11% of calls
Price1 credit
action.comEcommerce
How to fetch product pages from action.com: no browser needed, 92.5% success, one call per page.
Success · Aug92.5%
BrowserNo
Price1 credit
officeworks.com.auEcommerce
96.3% of calls to officeworks.com.au succeed. category pages, JavaScript token, median answer 10.6s.
Success · Aug96.3%
Browser56% of calls
Price1 credit
dillards.comEcommerce
99.8% of calls to dillards.com succeed. pages pages, no browser needed, median answer 1.1s.
Success · Aug99.8%
BrowserNo
Price1 credit
jomashop.comEcommerce
97.6% of calls to jomashop.com succeed. search results, JavaScript token, median answer 16.0s.
Success · Aug97.6%
BrowserYes
Price1 credit
waltons.co.zaEcommerce
How to fetch buy pages from waltons.co.za: no browser needed, 98.2% success, one call per page.
Success · Aug98.2%
BrowserNo
Price1 credit
harrods.comEcommerce
How to fetch product pages from harrods.com: no browser needed, 95.6% success, one call per page.
Success · Aug95.6%
BrowserNo
Price1 credit
jacksonsart.comEcommerce
How to fetch product pages from jacksonsart.com: browser only where the pattern needs it, 97.6% success, one call per page.
Success · Aug97.6%
Browser44% of calls
Price1 credit
cassart.co.ukEcommerce
Product pages on cassart.co.uk: no browser needed, 99.8% success in Aug 2026.
Success · Aug99.8%
BrowserNo
Price1 credit
myer.com.auEcommerce
99.9% of calls to myer.com.au succeed. category pages, no browser needed, median answer 2.1s.
Success · Aug99.9%
BrowserNo
Price1 credit
Site names identify the pages a recipe fetches. Crawlbase is not affiliated with any site listed. Use recipes within the acceptable use policy.Measured numbers from August 2026, refreshed monthly