Scrape category pages on tesco.com
The call that returns category pages from tesco.com today, with the parameters, patterns and failure modes read off the request log since 14 Aug 2026. Last tested 6 Sep 2026.
99.9% success · Aug 2026Browser on 18% of callsStandard price · 1 credit
About tesco.com
tesco.com describes itself as “Welcome to Tesco online. Discover great value groceries, plus clothing, recipes, bank, and mobile services. Browse on Tesco.com now!”.
Its home page declares 6 languages: en-gb, pl-pl, cs-cz, sk-sk, hu-hu, en-ie.
Its sitemaps list 217,884 URLs across 15 files (taxonomy, products, products-1, products-2, products-3, products-5). 6,174 of the 6,513 dated entries changed in the last 30 days, the newest on 6 Sep 2026.
Its robots.txt names 7 sitemaps and disallows 12 paths.
Measured on Crawlbase
Success rate over every request from every account in August 2026: 99.9%. Patterns, countries and failures come from the request log since 14 Aug 2026; 17.8% of successful calls used the JavaScript token and the median answer took 9.3s.
Among the 9 grocery sites in the Cookbook, tesco.com ranks 2 of 9 by success rate; the category median is 98.8%. Its median answer of 9.3s is slower than the category median of 2.6s. 5 of the 9 accept the plain token; this one does on some pages only.
The call
Category and listing pages are the shape the accounts fetch most here, with the country and token that succeed most often. Replace the placeholder path with a real section URL from the site.
curl "https://api.crawlbase.com/?token=YOUR_JS_TOKEN&url=https%3A%2F%2Fwww.tesco.com%2Fshop%2Fen-GB%2Fbrowse%2Fthe-best-night-in%2Fbritish%2Fall"from crawlbase import CrawlingAPI api = CrawlingAPI({'token': 'YOUR_JS_TOKEN'}) r = api.get('https://www.tesco.com/shop/en-GB/browse/the-best-night-in/british/all') html = r['body']
const { CrawlingAPI } = require('crawlbase'); const api = new CrawlingAPI({ token: 'YOUR_JS_TOKEN' }); const r = await api.get('https://www.tesco.com/shop/en-GB/browse/the-best-night-in/british/all');
Parameters that matter
| Parameter | Set it to | Why |
|---|---|---|
country | leave unset | Most successful calls set no country (98.7%) and succeed at 99.9%. |
javascript | use the JavaScript token | 99.9% of calls use the JavaScript token and succeed at 98.6%. The plain token was tried too rarely (14 calls) to say anything about it. |
URL patterns accounts fetch
| Pattern | Share of successes | JavaScript token | Country | Query parameters |
|---|---|---|---|---|
/shop/{slug}/{slug} | 96.5% | 100% | none | sortBy, page, count |
/{slug}/{slug}/shop | 2.8% | 100% | none | department, viewAll, aisle |
Fields you get
Structured data Crawlbase found on tesco.com, by page type. These are the schema.org types in the page, the fields your parser can read straight from the JSON-LD block:
- Home:
Organization - Detail page:
Corporation, WebSite, Product, BreadcrumbList - Listing:
Corporation, WebSite, Product, BreadcrumbList, ItemList
What breaks, and the fix
Of the failures since 14 Aug 2026, the site answered:
403: 33.8% of failures A 403 is the site refusing the exit. Set the country the successful calls use and retry once; a second 403 means the URL itself is protected.
504: 9.0% of failures A 504 is a timeout: the page did not finish inside the limit. Retry once, and keep your own client timeout above the median answer time shown for this site.
591, 529: 46.2% of failures together A 5xx is the site failing, not the fetch. Retry after a minute.
Run it on a schedule
Walk the listings on a cadence to find new items, then push the item URLs to the Crawler with a callback; it paces the calls and retries what the site refuses. The median answer on this site is 9.3s.
Questions
Do I need a browser to scrape tesco.com?
Yes for most pages. JavaScript-token calls succeed at 98.6% while plain-token calls reach 78.6%.
How much does a tesco.com page cost on Crawlbase?
1 credit without a browser. tesco.com is in the standard tier.
Which country should I set for tesco.com?
None is needed: most successful calls leave it unset.
Other pages on tesco.com
Pages tesco.com lists in its own sitemaps, fetched by Crawlbase on 6 Sep 2026 and verified in 2 consecutive runs. Each row is the cheapest call that returned the page with its content.
| Page | Example | Token | Country | Median |
|---|---|---|---|---|
Home/Structured data: Organization | tesco.com/ | Plain | none | 3.7s |
Detail page/shop/{slug}/productsStructured data: Corporation, WebSite, Product, BreadcrumbList | tesco.com/shop/en-GB/products/326454286 | Plain | none | 2.8s |
Listing/shop/{slug}/{slug}Structured data: Corporation, WebSite, Product, BreadcrumbList, ItemList | tesco.com/shop/en-GB/browse/the-best-night-in/british/all | Plain | none | 2.8s |
Site names identify the pages a recipe fetches. Crawlbase is not affiliated with any site listed. Use recipes within the acceptable use policy.Measured numbers from August 2026, refreshed monthly