Crawling API / DuckDuckGoDuckDuckGo Scraper.
DuckDuckGo Scraper.
Every result, fully rendered.
Send any DuckDuckGo results URL and get the fully rendered HTML back, through residential proxies with anti-bot handling built in.
Turn it into JSON with the generic extractor.
Live crawl feed · DuckDuckGo1.24M req/minStreaming
200duckduckgo.com/?q=residential+proxyNL163ms
200duckduckgo.com/?q=data+pipeline&t=h_BR119ms
200duckduckgo.com/?q=machine+learningDE151ms
200duckduckgo.com/?q=ecommerce+platform&t=h_CA132ms
200duckduckgo.com/?q=remote+jobs&ia=webDE215ms
301duckduckgo.com/?q=link+buildingSG49ms
200duckduckgo.com/?q=link+buildingDE121ms
200duckduckgo.com/html/?q=best+vpnFR153ms
404duckduckgo.com/?q=python+tutorial&t=h_GB146ms
200duckduckgo.com/html/?q=seo+toolsDE138ms
200duckduckgo.com/?q=ecommerce+platform&t=h_GB152ms
200duckduckgo.com/?q=data+pipeline&t=h_NL52ms
200duckduckgo.com/html/?q=seo+toolsGB64ms
200duckduckgo.com/?q=electric+cars&ia=webNL82ms
200duckduckgo.com/html/?q=seo+toolsIN120ms
200duckduckgo.com/?q=machine+learningBR213ms
200duckduckgo.com/?q=web+scraping+api&ia=webUS164ms
200duckduckgo.com/?q=web+scraping+api&ia=webUS182ms
200duckduckgo.com/?q=electric+cars&ia=webIN50ms
200duckduckgo.com/?q=web+scrapingDE115ms
200duckduckgo.com/?q=residential+proxyDE82ms
200duckduckgo.com/?q=cloud+hostingSG94ms
301duckduckgo.com/?q=ai+startups&ia=webJP200ms
200duckduckgo.com/html/?q=seo+toolsNL144ms
200duckduckgo.com/html/?q=best+vpnAU208ms
200duckduckgo.com/?q=cloud+hostingGB150ms
200duckduckgo.com/?q=residential+proxyNL163ms
200duckduckgo.com/?q=data+pipeline&t=h_BR119ms
200duckduckgo.com/?q=machine+learningDE151ms
200duckduckgo.com/?q=ecommerce+platform&t=h_CA132ms
200duckduckgo.com/?q=remote+jobs&ia=webDE215ms
301duckduckgo.com/?q=link+buildingSG49ms
200duckduckgo.com/?q=link+buildingDE121ms
200duckduckgo.com/html/?q=best+vpnFR153ms
404duckduckgo.com/?q=python+tutorial&t=h_GB146ms
200duckduckgo.com/html/?q=seo+toolsDE138ms
200duckduckgo.com/?q=ecommerce+platform&t=h_GB152ms
200duckduckgo.com/?q=data+pipeline&t=h_NL52ms
200duckduckgo.com/html/?q=seo+toolsGB64ms
200duckduckgo.com/?q=electric+cars&ia=webNL82ms
200duckduckgo.com/html/?q=seo+toolsIN120ms
200duckduckgo.com/?q=machine+learningBR213ms
200duckduckgo.com/?q=web+scraping+api&ia=webUS164ms
200duckduckgo.com/?q=web+scraping+api&ia=webUS182ms
200duckduckgo.com/?q=electric+cars&ia=webIN50ms
200duckduckgo.com/?q=web+scrapingDE115ms
200duckduckgo.com/?q=residential+proxyDE82ms
200duckduckgo.com/?q=cloud+hostingSG94ms
301duckduckgo.com/?q=ai+startups&ia=webJP200ms
200duckduckgo.com/html/?q=seo+toolsNL144ms
200duckduckgo.com/html/?q=best+vpnAU208ms
200duckduckgo.com/?q=cloud+hostingGB150ms
01 Live demo
Any DuckDuckGo URL in. HTML or JSON out.
The Crawling API, typed live. Get the rendered HTML, or switch to the generic extractor for JSON. Hover to pause and read.
Run your first request in minutes. Up to 10,000 free requests, no credit card.Start free
02 Capabilities
One API, everything DuckDuckGo throws at you.
DuckDuckGo renders its results with JavaScript and trips rate limits and bot checks at volume. The Crawling API renders it in a real browser, reaches it through residential IPs, and hands you clean HTML or JSON.
render
Full JavaScript rendering
A real browser executes the page, so the dynamically loaded organic results, titles, snippets and links are all captured, not just the initial shell.
proxies
140M residential IPs
Every request rotates a residential IP across 30 geographies, so you reach DuckDuckGo like a real local searcher and see local rankings.
anti-bot
Blocks handled for you
CAPTCHAs, bot walls and rate limits are cleared automatically. Nothing to solve, nothing to maintain.
format
HTML or JSON
Get the full rendered HTML, or add scraper=generic-extractor to return title, content, images and links as structured JSON.
extras
Screenshots and async
The same call can capture a full-page screenshot, or run asynchronously with webhooks and cloud storage.
one token
One API for every site
The Crawling API works on any URL, so the same token covers DuckDuckGo and everything else you crawl. See the live demo.
03 Output
Rendered HTML, or clean JSON.
By default you get the rendered HTML. Add the generic-extractor and the same page comes back as typed JSON.
{
"title": "web scraping at DuckDuckGo",
"favicon": "https://duckduckgo.com/favicon.ico",
"meta": { "description": "DuckDuckGo. Privacy, simplified.", "keywords": "..." },
"content": "Organic results, titles, snippets and result URLs for the query...",
"canonical": "https://duckduckgo.com/?q=web+scraping",
"images": [ "..." ],
"og_images": [ "..." ],
"links": [ "..." ]
}
Page
title · string canonical · string favicon · string
Meta
meta.description · string meta.keywords · string
Content
content · string
Media
images · array og_images · array
Links
links · array
04 How it works
From URL to data in one call.
Every DuckDuckGo request moves through the same path. You send a URL, we operate everything in between.
01
Send the URL
Pass any public DuckDuckGo results URL with your token: a query, the HTML endpoint, a vertical or a filtered search.
02
Rotate a proxy
A residential IP and geography that reach DuckDuckGo cleanly, drawn from 140M IPs across 30 regions.
03
Render the page
A real browser loads the page so the JavaScript-rendered organic results, titles and snippets render before capture.
04
Clear anti-bot
DuckDuckGo's rate limits and bot checks are handled automatically. Nothing to solve, nothing to maintain.
05
Return HTML or JSON
The fully rendered HTML comes back, or typed JSON when you add the generic extractor.
05 Use cases
What teams build on DuckDuckGo data.
USE / 01Rankings
Keyword rank tracking
Crawl a results URL per query and geography on a schedule to track positions and rank changes over time.
USE / 02SERP
SERP monitoring
Watch results pages for new entrants, feature changes and movement across the queries you care about.
USE / 03SEO research
SEO research
Pull ranked titles, snippets and result URLs to study intent, competitors and content gaps.
USE / 04Discovery
Lead & link discovery
Surface domains, profiles and pages ranking for your terms to feed outreach and link prospecting.
USE / 05Training
Training data & RAG
Feed clean results text and result URLs into models, RAG pipelines and agents through one API.
USE / 06Coverage
Any URL, one API
Crawl queries, the HTML endpoint, verticals and filters, plus any other site you need.
06 Notes
Good to know when scraping DuckDuckGo.
Rendered like a real browser
DuckDuckGo loads its results dynamically with JavaScript; the Crawling API runs a real browser so the organic results render before capture.
HTML by default, JSON on request
You get the full rendered HTML. Add scraper=generic-extractor for parsed title, content, images and links, or parse the HTML yourself for result titles, snippets and URLs.
Localize results by region
Geotargeting picks the proxy region, so you can read rankings as a searcher in a given country would see them.
Reach DuckDuckGo from anywhere
Rotating across 30 regions and 140M residential IPs means consistent access at volume without managing proxies or tripping rate limits.
07 Why Crawlbase
Built to crawl DuckDuckGo at scale.
The Crawling API runs on the same network that serves 46,000+ paying customers and 70,000+ developers. No proxies to buy, no browsers to run, nothing to patch when DuckDuckGo changes.
99%
Average request success rate
140M
Residential IPs, plus 98M datacenter
30
Geographies for accurate local results
20/s
Requests per second by default, more on request
One token, official SDKs for Python, Node and Ruby, and a 99.99% uptime network underneath.
08 FAQ
DuckDuckGo scraping questions.
Send the DuckDuckGo results URL to the Crawlbase Crawling API with your token. Crawlbase rotates a residential proxy, renders the page in a real browser, clears bot checks, and returns the fully rendered HTML with the organic results in place. Add scraper=generic-extractor to get structured JSON instead.
Yes. By default the Crawling API returns rendered HTML; add the generic extractor (scraper=generic-extractor) to receive title, meta, content, images and links as JSON, or parse the HTML yourself for result titles, snippets and URLs.
Yes. DuckDuckGo loads its results dynamically with JavaScript. A real browser executes the page, so the organic results render before capture, not just the initial shell.
Crawlbase routes each request through rotating residential IPs across 30 geographies and clears bot checks and rate limits automatically. You do not manage proxies or solve CAPTCHAs, and there is nothing to maintain when DuckDuckGo changes its setup.
Yes. Crawl a results URL per query and geography on a schedule, then read the ranked result titles and URLs from the HTML or JSON to track positions over time.
Any public results URL: standard queries, the HTML endpoint, web and news verticals, and region or time filtered searches. The same API works on any other site too.
Start free with up to 10,000 requests and no credit card. Paid plans scale with usage, and the same token works across the Crawling API and every Crawlbase scraper.
Start scraping DuckDuckGo.
Track rankings without the blocks.
Free to begin with up to 10,000 requests. One token for the Crawling API and every scraper.