# Crawler APIs

If you require to monitor the stats/history of your crawlers, purge the crawler, or delete a job from a specific crawler, through an API, please follow the below instructions:

Note: Please replace the TCP token with the JS one if you require to perform the below API calls on your JS crawlers.

# Stats API

To view a summary of your crawlers including concurrency, waiting and retry queues, and the crawling history with a breakdown of success and failure:

curl 'https://api.crawlbase.com/crawler/_USER_TOKEN_/stats'

To filter the History Stats by a date interval, please make the below GET request:

curl 'https://api.crawlbase.com/crawler/_USER_TOKEN_/stats?history_from=yyyy-mm-dd&history_to=yyyy-mm-dd'

# Purge API

To purge a crawler, please make the below POST request sending the correct Crawler name and the token (JS/TCP):

curl -X POST 'https://api.crawlbase.com/crawler/_USER_TOKEN_/YourCrawlerName/purge'

Note: All pages in the given crawler will be instantly removed.

# Delete Job API

To delete a job from a given crawler, please make the below POST request sending the correct request RID, token (JS/TCP) and the Crawler name:

curl -X POST 'https://api.crawlbase.com/crawler/_USER_TOKEN_/YourCrawlerName/delete_job?rid=RID'

# Find Job API

To find a request by RID in your crawler queue, please make the below GET request sending the correct request RID, token (JS/TCP), and the Crawler name:

curl 'https://api.crawlbase.com/crawler/_USER_TOKEN_/YourCrawlerName/find_by_rid/RID'

If in QUEUE: If the request is in the Queue, Waiting to be retried or in Retry set, then you get a response with status: QUEUED.

{
  "status": "QUEUED",
  "request_info": {
    "rid": "YOUR_RID",
    "url": "YOUR_URL",
    "retry": 3,
    "created_at": 1600494969.189415
  }
}

If not in QUEUE: If the request is Crawled Already or not in the Queue, you get status: NOT_QUEUED.

{
  "status": "NOT_QUEUED",
  "request_info": {
    "rid": "YOUR_RID"
  }
}