Scraping cookbook · News and media

Scraping news and media sites

74 sites in this category, measured on Crawlbase traffic in August 2026. Each page carries the call that works and the numbers behind it.

Publisher sites: newspapers, portals and blogs, from medium.com and yahoo.com to the Chinese portals 163.com and qq.com. Article pages carry their text in the HTML, so most recipes here run on the plain token; the rest need the JavaScript token. 58 of the 74 sites answer the plain token. Every site is priced at 1 credit a page.

yahoo.comNews and media

99.6% of calls to yahoo.com succeed. articles, no browser needed, median answer 1.0s.

Success · Aug99.6%

medium.comNews and media

How to fetch articles from medium.com: no browser needed, 93.9% success, one call per page.

Success · Aug93.9%

bbc.comNews and media

97.9% of calls to bbc.com succeed. sport pages, browser only where the pattern needs it, median answer 5.4s.

Success · Aug97.9%

Browser23% of calls

qq.comNews and media

99.9% of calls to qq.com succeed. tag pages, JavaScript token, median answer 6.1s.

Success · Aug99.9%

news.com.auNews and media

91.9% of calls to news.com.au succeed. finance pages, no browser needed, median answer 3.1s.

Success · Aug91.9%

indiatimes.comNews and media

99.3% of calls to indiatimes.com succeed. articles, no browser needed, median answer 4.7s.

Success · Aug99.3%

prnewswire.comNews and media

Articles on prnewswire.com: no browser needed, 96.1% success in Aug 2026.

Success · Aug96.1%

cnbc.comNews and media

99.4% of calls to cnbc.com succeed. articles, no browser needed, median answer 6.1s.

Success · Aug99.4%

Browser6% of calls

businesswire.comNews and media

98.8% of calls to businesswire.com succeed. articles, JavaScript token, median answer 5.7s.

Success · Aug98.8%

indianexpress.comNews and media

99.1% of calls to indianexpress.com succeed. section pages, no browser needed, median answer 3.0s.

Success · Aug99.1%

ft.comNews and media

How to fetch content pages from ft.com: JavaScript token, 90.9% success, one call per page.

Success · Aug90.9%

Browser77% of calls

theguardian.comNews and media

Business pages on theguardian.com: browser only where the pattern needs it, 99.8% success in Aug 2026.

Success · Aug99.8%

Browser28% of calls

163.comNews and media

93.8% of calls to 163.com succeed. articles, no browser needed, median answer 7.6s.

Success · Aug93.8%

abcnews.go.comNews and media

100% of calls to abcnews.go.com succeed. articles, no browser needed, median answer 2.9s.

Success · Aug100%

sina.com.cnNews and media

99.7% of calls to sina.com.cn succeed. articles, no browser needed, median answer 3.1s.

Success · Aug99.7%

chicagotribune.comNews and media

Sports pages on chicagotribune.com: no browser needed, 99.6% success in Aug 2026.

Success · Aug99.6%

news24.com.auNews and media

94.4% of calls to news24.com.au succeed. opinion pages, no browser needed, median answer 5.8s.

Success · Aug94.4%

kompas.tvNews and media

The homepage on kompas.tv: no browser needed, 97.4% success in Aug 2026.

Success · Aug97.4%

marthastewart.comNews and media

How to fetch articles from marthastewart.com: JavaScript token, 96.9% success, one call per page.

Success · Aug96.9%

heraldsun.com.auNews and media

Education pages on heraldsun.com.au: no browser needed, 93.2% success in Aug 2026.

Success · Aug93.2%

houstonchronicle.comNews and media

94.6% of calls to houstonchronicle.com succeed. articles, JavaScript token, median answer 24.9s.

Success · Aug94.6%

Browser80% of calls

techtimes.comNews and media

Articles on techtimes.com: no browser needed, 100% success in Aug 2026.

Success · Aug100%

digitaltoday.co.krNews and media

100% of calls to digitaltoday.co.kr succeed. articles, no browser needed, median answer 2.4s.

Success · Aug100%

sfchronicle.comNews and media

99.7% of calls to sfchronicle.com succeed. sports pages, no browser needed, median answer 2.1s.

Success · Aug99.7%

Browser18% of calls

openpr.comNews and media

Articles on openpr.com: no browser needed, 99.7% success in Aug 2026.

Success · Aug99.7%

lesechos.frNews and media

How to fetch articles from lesechos.fr: no browser needed, 92.2% success, one call per page.

Success · Aug92.2%

parade.comNews and media

Articles on parade.com: no browser needed, 96.6% success in Aug 2026.

Success · Aug96.6%

joe.ieNews and media

Entertainment pages on joe.ie: no browser needed, 98.3% success in Aug 2026.

Success · Aug98.3%

adelaidenow.com.auNews and media

Health pages on adelaidenow.com.au: no browser needed, 92.9% success in Aug 2026.

Success · Aug92.9%

rtve.esNews and media

How to fetch search results from rtve.es: no browser needed, 98.2% success, one call per page.

Success · Aug98.2%

couriermail.com.auNews and media

93.0% of calls to couriermail.com.au succeed. sport pages, no browser needed, median answer 5.3s.

Success · Aug93.0%

livemint.comNews and media

How to fetch rss pages from livemint.com: no browser needed, 99.9% success, one call per page.

Success · Aug99.9%

patch.comNews and media

Articles on patch.com: browser only where the pattern needs it, 99.7% success in Aug 2026.

Success · Aug99.7%

Browser93% of calls

nikkei.comNews and media

How to fetch articles from nikkei.com: no browser needed, 96.3% success, one call per page.

Success · Aug96.3%

nydailynews.comNews and media

99.2% of calls to nydailynews.com succeed. sports pages, no browser needed, median answer 1.3s.

Success · Aug99.2%

miamiherald.comNews and media

Articles on miamiherald.com: no browser needed, 93.3% success in Aug 2026.

Success · Aug93.3%

Browser7% of calls

itc.cnNews and media

How to fetch articles from itc.cn: no browser needed, 98.0% success, one call per page.

Success · Aug98.0%

efe.comNews and media

96.5% of calls to efe.com succeed. articles, no browser needed, median answer 1.4s.

Success · Aug96.5%

gq.comNews and media

Culture pages on gq.com: no browser needed, 92.3% success in Aug 2026.

Success · Aug92.3%

sabah.com.trNews and media

99.6% of calls to sabah.com.tr succeed. rss pages, no browser needed, median answer 1.1s.

Success · Aug99.6%

mothership.sgNews and media

How to fetch category pages from mothership.sg: no browser needed, 99.2% success, one call per page.

Success · Aug99.2%

Browser6% of calls

cw.noNews and media

How to fetch articles from cw.no: no browser needed, 94.6% success, one call per page.

Success · Aug94.6%

sohu.comNews and media

Articles on sohu.com: no browser needed, 98.3% success in Aug 2026.

Success · Aug98.3%

daryo.uzNews and media

Articles on daryo.uz: no browser needed, 99.9% success in Aug 2026.

Success · Aug99.9%

spectator.orgNews and media

Articles on spectator.org: no browser needed, 100% success in Aug 2026.

Success · Aug100%

suara.comNews and media

Articles on suara.com: no browser needed, 99.7% success in Aug 2026.

Success · Aug99.7%

themercury.com.auNews and media

92.6% of calls to themercury.com.au succeed. entertainment pages, no browser needed, median answer 5.4s.

Success · Aug92.6%

refinery29.comNews and media

Articles on refinery29.com: no browser needed, 99.9% success in Aug 2026.

Success · Aug99.9%

ledauphine.comNews and media

How to fetch rss pages from ledauphine.com: no browser needed, 99.9% success, one call per page.

Success · Aug99.9%

azcentral.comNews and media

94.3% of calls to azcentral.com succeed. articles, browser only where the pattern needs it, median answer 32.4s.

Success · Aug94.3%

Browser77% of calls

headtopics.comNews and media

95.8% of calls to headtopics.com succeed. the homepage, no browser needed, median answer 1.6s.

Success · Aug95.8%

yicai.comNews and media

How to fetch articles from yicai.com: no browser needed, 99.9% success, one call per page.

Success · Aug99.9%

gazzetta.itNews and media

99.5% of calls to gazzetta.it succeed. the homepage, no browser needed, median answer 1.5s.

Success · Aug99.5%

bravo.deNews and media

98.9% of calls to bravo.de succeed. articles, no browser needed, median answer 1.7s.

Success · Aug98.9%

ole.com.arNews and media

How to fetch esports pages from ole.com.ar: no browser needed, 98.8% success, one call per page.

Success · Aug98.8%

itmedia.co.jpNews and media

98.1% of calls to itmedia.co.jp succeed. articles, no browser needed, median answer 6.3s.

Success · Aug98.1%

kurier.atNews and media

98.2% of calls to kurier.at succeed. articles, no browser needed, median answer 1.0s.

Success · Aug98.2%

rawstory.comNews and media

99.2% of calls to rawstory.com succeed. articles, no browser needed, median answer 1.4s.

Success · Aug99.2%

milenio.comNews and media

How to fetch opinion pages from milenio.com: no browser needed, 98.7% success, one call per page.

Success · Aug98.7%

macdailynews.comNews and media

Articles on macdailynews.com: no browser needed, 98.9% success in Aug 2026.

Success · Aug98.9%

foreignaffairs.comNews and media

How to fetch articles from foreignaffairs.com: JavaScript token, 99.7% success, one call per page.

Success · Aug99.7%

concise.ngNews and media

How to fetch articles from concise.ng: no browser needed, 99.9% success, one call per page.

Success · Aug99.9%

tsf.ptNews and media

Articles on tsf.pt: no browser needed, 99.4% success in Aug 2026.

Success · Aug99.4%

expansion.mxNews and media

Articles on expansion.mx: no browser needed, 97.6% success in Aug 2026.

Success · Aug97.6%

israelhayom.co.ilNews and media

Articles on israelhayom.co.il: JavaScript token, 99.9% success in Aug 2026.

Success · Aug99.9%

gendai.mediaNews and media

99.9% of calls to gendai.media succeed. list pages, no browser needed, median answer 1.2s.

Success · Aug99.9%

jeuneafrique.comNews and media

Articles on jeuneafrique.com: JavaScript token, 97.4% success in Aug 2026.

Success · Aug97.4%

montrealgazette.comNews and media

How to fetch category pages from montrealgazette.com: no browser needed, 99.1% success, one call per page.

Success · Aug99.1%

lecturas.comNews and media

Articles on lecturas.com: no browser needed, 93.3% success in Aug 2026.

Success · Aug93.3%

thepaper.cnNews and media

How to fetch articles from thepaper.cn: no browser needed, 98.9% success, one call per page.

Success · Aug98.9%

toutiao.comNews and media

How to fetch articles from toutiao.com: no browser needed, 100% success, one call per page.

Success · Aug100%

cadenaser.comNews and media

99.4% of calls to cadenaser.com succeed. deportes pages, no browser needed, median answer 2.2s.

Success · Aug99.4%

diaspora.roNews and media

How to fetch articles from diaspora.ro: no browser needed, 99.7% success, one call per page.

Success · Aug99.7%

indianweb2.comNews and media

How to fetch articles from indianweb2.com: no browser needed, 100% success, one call per page.

Success · Aug100%

Site names identify the pages a recipe fetches. Crawlbase is not affiliated with any site listed. Use recipes within the acceptable use policy.Measured numbers from August 2026, refreshed monthly