Cookbook·Public data·nyc.gov
Scrape nyc.gov
The call that returns pages from nyc.gov today, with the parameters, patterns and failure modes read off the request log since 14 Aug 2026.
92.0% success · Aug 2026Browser on 12% of callsStandard price · 1 credit
Measured on Crawlbase
Success rate over every request from every account in August 2026: 92.0%. Patterns, countries and failures come from the request log since 14 Aug 2026; 11.8% of successful calls used the JavaScript token and the median answer took 1.8s.
Among the 11 public data sites in the Cookbook, nyc.gov sits at 10 of 11 by success rate; the category median is 97.2%. Its median answer of 1.8s is faster than the category median of 2.3s. 5 of the 11 accept the plain token; this one does on some pages only.
The call
The most common shape of URL the accounts fetch, with the country and token that succeed most often. Replace the placeholder path with a real one from the site.
curl "https://api.crawlbase.com/?token=YOUR_TOKEN&url=https%3A%2F%2Fwww.nyc.gov%2Fexample-page%2Fexample-page%2Fexample-page"from crawlbase import CrawlingAPI api = CrawlingAPI({'token': 'YOUR_TOKEN'}) r = api.get('https://www.nyc.gov/example-page/example-page/example-page') html = r['body']
const { CrawlingAPI } = require('crawlbase'); const api = new CrawlingAPI({ token: 'YOUR_TOKEN' }); const r = await api.get('https://www.nyc.gov/example-page/example-page/example-page');
Parameters that matter
| Parameter | Set it to | Why |
|---|---|---|
country | leave unset | Most successful calls set no country (99.8%) and succeed at 90.4%. |
javascript | leave unset | Plain-token calls succeed at 89.1% and make up 87.4% of the traffic. A browser costs more and, on this site, buys nothing. |
URL patterns accounts fetch
| Pattern | Share of successes | JavaScript token | Country | Query parameters |
|---|---|---|---|---|
/{slug}/{slug}/{slug} | 92.0% | 32.9% | none | hid_borough, hid_block, hid_lot |
Fields you get
Fields depend on the page type. Product pages usually carry an ld+json block with name, price, currency and availability, and article pages a headline and body; check the first response you get.
What breaks, and the fix
Of the failures since 14 Aug 2026, the site answered:
410: 46.2% of failures 410 is not a standard HTTP answer; the site sent it as a refusal. Treat it like a block: retry once with the JavaScript token and the country the successful calls use, then stop.
404: 30.4% of failures Note that a 404 counts as a success in these numbers: the site answered and the page is gone. Treat it as gone, not as a retry.
0: 16.1% of failures A status of 0 is a request that never got an answer inside the timeout. Retry once with the JavaScript token if the pattern needs a browser.
Run it on a schedule
The median answer on this site is 1.8s. Push a URL list to the Crawler with a callback for anything larger than a few thousand pages a day; it paces the calls and retries what the site refuses.
Questions
Do I need a browser to scrape nyc.gov?
No. 89.1% of plain-token calls succeed and accounts send 87.4% of their traffic without a browser.
How much does a nyc.gov page cost on Crawlbase?
1 credit without a browser. nyc.gov is in the standard tier.
Which country should I set for nyc.gov?
None is needed: most successful calls leave it unset.
Site names identify the pages a recipe fetches. Crawlbase is not affiliated with any site listed. Use recipes within the acceptable use policy.Measured numbers from August 2026, refreshed monthly