Scrape twitter.com

Measured on Crawlbase

Success rate over every request from every account in August 2026: 99.8%. Patterns, countries and failures come from the request log since 14 Aug 2026; 100% of successful calls used the JavaScript token and the median answer took 14.9s.

Among the 8 social sites in the Cookbook, twitter.com sits at 2 of 8 by success rate; the category median is 98.9%. Its median answer of 14.9s is slower than the category median of 9.2s. 6 of the 8 accept the plain token; this one needs the JavaScript token.

The call

The most common shape of URL the accounts fetch, with the country and token that succeed most often. Replace the placeholder path with a real one from the site.

curl "https://api.crawlbase.com/?token=YOUR_TOKEN&url=https%3A%2F%2Fwww.twitter.com%2Fexample-page"

from crawlbase import CrawlingAPI
api = CrawlingAPI({'token': 'YOUR_TOKEN'})

r = api.get('https://www.twitter.com/example-page')
html = r['body']

const { CrawlingAPI } = require('crawlbase');
const api = new CrawlingAPI({ token: 'YOUR_TOKEN' });

const r = await api.get('https://www.twitter.com/example-page');

Parameters that matter

URL patterns accounts fetch

Fields you get

Fields depend on the page type. Product pages usually carry an ld+json block with name, price, currency and availability, and article pages a headline and body; check the first response you get.

What breaks, and the fix

Of the failures since 14 Aug 2026, the site answered:

Run it on a schedule

The median answer on this site is 14.9s. Push a URL list to the Crawler with a callback for anything larger than a few thousand pages a day; it paces the calls and retries what the site refuses.

Questions