Scraping cookbook · Public data

Scraping public data sites

11 sites in this category, measured on Crawlbase traffic in August 2026. Each page carries the call that works and the numbers behind it.

Government and open-data sources, sec.gov, nyc.gov, justice.gov and openstreetmap.org. Search and record pages are the common shape; about half the sites render results client-side and need the JavaScript token. 5 of the 11 sites answer the plain token. Every site is priced at 1 credit a page.

sec.govPublic data

Pages on sec.gov: no browser needed, 97.6% success in Aug 2026.

Success · Aug97.6%
BrowserNo
Price1 credit
nyc.govPublic data

Pages on nyc.gov: no browser needed, 92.0% success in Aug 2026.

Success · Aug92.0%
Browser12% of calls
Price1 credit
justice.govPublic data

The homepage on justice.gov: browser only where the pattern needs it, 97.2% success in Aug 2026.

Success · Aug97.2%
Browser58% of calls
Price1 credit
openstreetmap.orgPublic data

How to fetch search results from openstreetmap.org: no browser needed, 99.7% success, one call per page.

Success · Aug99.7%
BrowserNo
Price1 credit
bizapedia.comPublic data

98.6% of calls to bizapedia.com succeed. pages, no browser needed, median answer 7.6s.

Success · Aug98.6%
BrowserNo
Price1 credit
ademe.frPublic data

95.7% of calls to ademe.fr succeed. pages, JavaScript token, median answer 1.7s.

Success · Aug95.7%
BrowserYes
Price1 credit
nysenate.govPublic data

92.5% of calls to nysenate.gov succeed. pages, JavaScript token, median answer 33.4s.

Success · Aug92.5%
BrowserYes
Price1 credit
unep-wcmc.orgPublic data

How to fetch pages from unep-wcmc.org: no browser needed, 95.9% success, one call per page.

Success · Aug95.9%
BrowserNo
Price1 credit
fmi.fiPublic data

90.1% of calls to fmi.fi succeed. pages, no browser needed, median answer 1.8s.

Success · Aug90.1%
BrowserNo
Price1 credit
societe.comPublic data

98.6% of calls to societe.com succeed. pages, JavaScript token, median answer 7.2s.

Success · Aug98.6%
BrowserYes
Price1 credit
planninginspectorate.gov.ukPublic data

99.8% of calls to planninginspectorate.gov.uk succeed. projects pages, JavaScript token, median answer 3.4s.

Success · Aug99.8%
Browser55% of calls
Price1 credit
Site names identify the pages a recipe fetches. Crawlbase is not affiliated with any site listed. Use recipes within the acceptable use policy.Measured numbers from August 2026, refreshed monthly