Storage API
Easily store and search your crawled or scraped data.
Try it free. No credit card required. Instant set-up.
Contact us if you need more information.
Store. Search. Scale
Cloud storage on demand
Start saving results in Crawlbase Storage and store up to
10,000 documents per 14 days for free
Easy-to-use API
Use the Storage API to retrieve saved pages, screenshots and perform full-text searches on the pages you are crawling.
Scrape and crawl
We provide tools to scrape and crawl any website while remaining anonymous and bypassing any restrictions, blocks, or captchas.
Pricing
Storage comes as a free solution for developers and a very inexpensive solution for companies.
If you need a custom solution or expert advice on web scraping, our team of engineers is ready to handle every challenge.
Store. Search. Scale.
Are you using Amazon AWS S3 or SQS to temporarily or permanently store your crawled pages?
Do you maintain your own database? Searching your crawled data is becoming a problem the more you scale up your web scraping requests?
Crawlbase Storage is a cloud scalable storage solution where you can permanently or temporarily store your HTML pages, screenshots and scraped data.
Storage handles scaling, backing up and cleaning of your cloud space so you can focus on what really matters for your business.
How it Works
Store the API response in the cloud by adding &store=true parameter on the Crawling API call, or use the Storage API as a Crawler webhook.
Save the RID that will be generated in the API response
Use the RID to retrieve the crawled data from the cloud. Send a GET request to the Storage API or use the Storage dashboard
Example Storage API call using cURL: curl "https://api.crawlbase.com/storage?token=USER_TOKEN_&rid=RID"
Smart Storage
Simply add "&store=true" to your Crawling API or Crawler calls, and the data will be securely stored for you.
Storage is designed to solve the problem of having data to store when you have no reliable or cost-efficient solution to scale it.
99%
Success Rate
100%
Network Uptime
24/7
Support from a team of experts
Our pricing plans
Choose one of the pricing plans according to your number of requests
Do you need more than 1M requests?
Contact us for more requests
- X requests
- X days retention period
- Get API
- Delete API
- Bulk API
If you need more requests, please contact us here
OR
Happy customer stories
Stefan Wagner
Full stack DevThe Storage API is our strategic storage platform that allows our team to securely access data and collaborate from any location. It is a great help to maintain smooth business operation.
April, 2022
Terrence Huang
Product specialistA very convenient way to store scraped content. We can share documents quickly, easily, and maintenance-free.
Mar, 2022
Cho Noguchi
Data ScientistThanks to Crawlbase, I can now store and download tons of my data anytime I want without a huge dent in my wallet.
Feb, 2022Trusted by more than 70,000 paying customers
Create Free Account!Customer Success storiesUsed by the world’s most innovative businesses – big and small
Supporting all kinds of crawling projects
Move your crawled and scraped data to the cloud now
Try it free. No credit card required. Instant set-up.
