Scrape opinion pages on news24.com.au

About news24.com.au

news24.com.au describes itself as “News24, the new name for Sky News Australia, delivers trusted breaking news, fearless commentary and sharp analysis on the stories shaping Australia and the world”.

Its sitemaps list 1,400 URLs across 19 files. 1,400 of the 1,400 dated entries changed in the last 30 days, the newest on 6 Sep 2026.

Its robots.txt names 4 sitemaps and disallows 12 paths, among them /printpage/, /akam/.

Measured on Crawlbase

Success rate over every request from every account in August 2026: 94.4%. Patterns, countries and failures come from the request log since 14 Aug 2026; 1.0% of successful calls used the JavaScript token and the median answer took 5.8s.

Among the 74 news and media sites in the Cookbook, news24.com.au ranks 61 of 74 by success rate; the category median is 98.9%. Its median answer of 5.8s is slower than the category median of 2.3s. 58 of the 74 accept the plain token, this one among them.

The call

Article pages are the shape the accounts fetch most here, with the country and token that succeed most often. Replace the placeholder path with a real article URL from the site.

curl "https://api.crawlbase.com/?token=YOUR_TOKEN&url=https%3A%2F%2Fwww.news24.com.au%2Fopinion"

from crawlbase import CrawlingAPI
api = CrawlingAPI({'token': 'YOUR_TOKEN'})

r = api.get('https://www.news24.com.au/opinion')
html = r['body']

const { CrawlingAPI } = require('crawlbase');
const api = new CrawlingAPI({ token: 'YOUR_TOKEN' });

const r = await api.get('https://www.news24.com.au/opinion');

Parameters that matter

URL patterns accounts fetch

Fields you get

Structured data Crawlbase found on news24.com.au, by page type. These are the schema.org types in the page, the fields your parser can read straight from the JSON-LD block:

• Home: WebPage, Organization, WebSite
• Article: NewsArticle, BreadcrumbList
• Other: NewsArticle, BreadcrumbList

What breaks, and the fix

Of the failures since 14 Aug 2026, the site answered:

Run it on a schedule

News moves by the hour. Poll the section or the sitemap for new links, then push the article URLs to the Crawler with a callback so the fetches are paced. The median answer on this site is 5.8s.

Questions

Other pages on news24.com.au

Pages news24.com.au lists in its own sitemaps, fetched by Crawlbase on 6 Sep 2026 and verified in 2 consecutive runs. Each row is the cheapest call that returned the page with its content.