Docs
Log in

API Reference

Endpoint specs and parameter references for every Crawlbase API. One token authenticates all of them; pricing and concurrency budgets are shared across the products you subscribe to.

One platform, one token

All APIs below authenticate against the same token (Normal or JavaScript variants — see Authentication). The Crawling API is the engine; everything else is a different surface on top of it (proxy interface, persistent storage, queue management) or a small specialized helper.

Core APIs

Three endpoints cover 95% of crawl + scrape workloads. Pick one based on how you want to address the API:

  • Crawling API — REST endpoint. Pass URL + parameters as a query string, get the page back. Powers JS rendering, anti-bot bypass, geo-routing, and the scraper library. The default choice for new integrations.
  • Enterprise Crawler — high-throughput async queue: push millions of URLs, get results streamed back to your webhook. Manages retries, rate, and persistence so your client doesn't have to.
  • Smart AI Proxy — proxy interface. Same network, same feature surface as the Crawling API; configured once in your HTTP client instead of per-request. The right fit when you can't or don't want to change the request shape of an existing scraper.

Data & storage

  • Cloud Storage — durable storage for crawl results. S3-compatible, CDN-fronted; persists HTML or parsed JSON keyed by request ID so you can fetch later without re-crawling.

Account & metadata

  • Account API — monthly usage, credits, success rates, per-domain stats. Useful for in-app metering displays and proactive backoff.
  • User Agents API free — randomized User-Agent strings tuned for crawling, free with a 1 req/s rate limit. Drop-in for clients that want to rotate UAs without maintaining their own pool.

Legacy APIs

These predate the modern endpoints above and are still operational for existing customers — closed to new sign-ups, no shutdown scheduled. New integrations should use the modern equivalents named in the migration callout at the top of each page.

  • Scraper API legacy — standalone scraper endpoint. Migrate to: Crawling API + &scraper=.
  • Screenshots API legacy — standalone screenshot endpoint. Migrate to: Crawling API + screenshot params, or MCP's crawl_screenshot.
  • Proxy API legacy — Proxy Backconnect. Migrate to: Smart AI Proxy.
  • Leads API legacy — domain-scoped email extraction. No direct replacement; closest workflows live in the email-extractor scraper.

Full overview of legacy options at /docs/legacy.