Status Codes · Crawlbase Documentation

Two statuses, two questions

Most HTTP APIs give you a single status code. Crawlbase gives you two because crawling involves two layers - Crawlbase's infrastructure, and the target site behind it.

HTTP status

int

The status of your request to Crawlbase. 200 means the crawl succeeded; 520 means it failed and cb_status says why; other 4xx/5xx codes mean the request was refused before any crawl.

cb_status

intheader

Crawlbase's verdict on the crawl, after its own block, redirect and content checks. 200 means a usable page; other codes describe what went wrong.

original_status

intheader

The raw HTTP status the target site returned. Useful when the site itself returns a non-200 you need to handle (404, 403, etc.).

The mental model

Always check HTTP status first. If it's 200, then check cb_status. If that's 200, then check original_status for site-side errors.

HTTP status codes

What Crawlbase itself returned to your client.

Code	Meaning	Action
`200`	Request processed. Check `cb_status` for outcome.	Continue to `cb_status`
`400`	Invalid request: `url` missing, not URL-encoded or too long, or a parameter with an invalid value.	Fix the request; retrying it unchanged won't help
`401`	Token missing or invalid.	Verify token; check it hasn't been reset
`402`	This workspace reached its monthly spend cap.	Raise the cap in the console, or wait for the next period
`403`	The account is blocked (for example, free credits are used up and no billing is set up) or has reached its account spending limit.	Check the Billing page in the console
`429`	Rate limit reached for this token and target domain.	Back off and retry; see Rate Limits
`500`	Crawlbase internal error. Rare and transient.	Retry with backoff; check status page
`503`	Service temporarily unavailable.	Retry with backoff
`504`	The request timed out before the crawl finished (`cb_status` is also `504`). Not charged.	Retry with backoff
`520`	The crawl failed. `cb_status` says why. Not charged.	Read `cb_status` and follow the tables below

cb_status codes

What happened during the crawl. Returned as the cb_status response header (and in the body with format=json) on every crawl response, successful or not. Formerly named pc_status, which is still sent as an alias.

Success

Code	Meaning
`200`	Page crawled successfully. Body is the target page's HTML or JSON. `original_status` can also be `404`, `410` or a redirect: those are valid answers, count as successful crawls and are charged.
`207`	The site answered `200` with an empty body. Counts as a success and is charged; if you expected content, retry with the JavaScript token.

Target site responded with an error

Crawlbase reached the site, but the site refused the request. original_status holds the site's own status, and none of these are charged. A 404 or 410 is not an error here: it comes back as cb_status 200.

Code	Meaning
`704`	The site answered another non-success status. Read `original_status` and handle it as the site's answer, or list it in `custom_success_codes` to receive the body anyway.
`705`	The site answered `403`, usually a block. Lower your request rate and investigate before retrying.
`706`	The site answered `429` and is rate limiting you. Back off and lower your request rate.

Blocked or failed

Code	Meaning	What to try
`503`	Crawlbase recognized a block or CAPTCHA page.	Switch from the Normal to the JavaScript token; if already on JavaScript, back off and retry
`595`	The `css_click_selector` element was not found on the page.	Fix the selector or add a fallback such as `#start-button,body`
`599`	Internal Crawlbase error.	Retry with backoff; send the `rid` to support if it persists
`5xx-7xx`	Any other `5xx`, `6xx` or `7xx` code is an internal or site-specific failure. None of them are charged.	Retry with a cap; send the `rid` to support if it persists

Reading the response

curl -i 'https://api.crawlbase.com/?token=YOUR_TOKEN&url=https%3A%2F%2Fexample.com'


# HTTP/1.1 200 OK
# cb_status: 200
# original_status: 200
# url: https://example.com/
# content-type: text/html
from crawlbase import CrawlingAPI


api = CrawlingAPI({'token': 'YOUR_TOKEN'})
res = api.get('https://example.com')


# Layer 1: did the crawl succeed? 520 means it failed.
if res['status_code'] != 200:
    raise RuntimeError(f"Crawl failed: {res['status_code']}, cb_status {res['cb_status']}")


# Layer 2: did Crawlbase get content? 207 means an empty page.
if res['cb_status'] == 207:
    raise RuntimeError("Empty page; retry with the JavaScript token")


# Layer 3: did the target site return content?
if res['original_status'] != 200:
    print(f"Site returned {res['original_status']}")


print(res['body'])

Next steps

Error Handling

Patterns for retries, dead-letter queues, and observability.

Rate Limits

Specific guidance on 429s and concurrency.