The Crawling API parameters can be passed to the Smart Proxy as a header prefixed with
This allows you to provide the specific configuration and instructions for crawling behavior to the Smart Proxy, enabling you to customize the request to suit your specific requirements.
The following example illustrates how to use Smart Proxy to issue a geo-located request, specifically targeting the United States with
country=US. Additionally, various API parameters can be utilized, including but not limited to:
curl -H "CrawlbaseAPI-Parameters: country=US&get_headers=true" \ -x "http://[email protected]:8012" \ -k "http://httpbin.org/ip"