Crawlbase’s Frequently Asked Questions

At the moment the only way to use Crawlbase is by integrating it to your product or service, therefore you will need to know development or know/pay someone which can help you.

Please note that we have some very easy to integrate libraries.

You can the API Key either in the Normal Request Tokens or the Javascript Request Tokens.

https://crawlbase.com/dashboard/account/docs

All the leads you get from the Leads API are up to date. We don't work with outdated sources.

By default a request is retried 110 times for a period of 48 hours from the first time it starts to be processed.

For LinkedIn crawls, the limit is 30 push per second. But for other URLs, we can increase the limit on a case-to-case basis

We want to offer more than just a list or lists of proxies, therefore our API and Crawler products are built on top of thousands of residential and data center worldwide proxies combined with artificial intelligence plus our engineering team which helps delivering the best data results possible. If you are looking just for proxies and you don't want to get any of the other benefits, we recommend that you use the Proxy Backconnect

The Crawling API is a generic API that can be used to crawl any website and get the raw source page. It also gives the option to use our classified scrapers. The Scraper API is an API that focuses on auto scraping and parsing HTML pages into structured data. One of their differences as well, the Crawling API is tiered pricing and payments are made at the end of every month based on the request consumption while the Scraper API is subscription-based.

Yes, the smart proxy can be used for https websites, but the connection to the proxy must be done with http. So when connecting to the proxy use http://smartproxy.crawlbase.com and not https:// All https traffic that you send to any website using the smart proxy, will continue to be securely transfered and we won't be able to access it in any way.

No, you cannot change the account email address. If you need to do so, please create a new account with the new email and re-add your billing details to the new account. Don't forget to update your account tokens.

We prorate the charges, so only the the amount according to the remaining days of the month period will be charged.