Docs
Log in

Search Engines

Two scrapers that turn Google and Bing search-results pages into clean structured JSON, including organic listings, ads, and side-panels. Useful for SEO monitoring, rank tracking, and competitive research.

Overview

Search-engine scrapers turn a Google or Bing results page into structured JSON: organic listings, ads, knowledge panel, related searches, and "people also ask" — all surfaces, one parser, one bill. The browser dance (consent walls, JS rendering, anti-bot, geo-routing) happens server-side; you receive the parsed shape directly.

Common use cases:

  • Rank tracking: monitor a list of keywords daily, store the position of your domain (and your competitors') in organic[].url, alert on movement.
  • SERP-feature monitoring: detect when Google promotes a People-Also-Ask box, knowledge panel, or AI overview that pushes organic results below the fold.
  • SEO research: harvest related_searches and people_also_ask for content-gap analysis without subscribing to a third-party SaaS.
  • AI-training datasets: build a corpus of "what Google shows for query X" snapshots for retrieval-augmented generation and answer-engine evaluations.

Geo and language are explicit: pass country + language on the request and the scraper hits the right Google domain (e.g. google.de for country=DE) so the SERP you see matches what a user in that market sees. Combine with the Smart Proxy when you need sticky-session pagination across deep result pages.

Google

Full Google SERP — organic, ads, snippets, related searches, and people-also-ask. The most-used scraper in the catalog; pairs naturally with the e-commerce *-product-details scrapers when you need a discovery → enrichment pipeline.

  • Google SERP — Google search-results page — organic, ads, knowledge panel, related searches.

Bing

Bing search-results page. Useful when you need a second data source for SEO research, or when your audience skews enterprise/Windows where Bing share is meaningfully higher than its overall market share suggests.

Example call

Below: a single google-serp call. Pass country and language alongside the URL when you need the regional storefront — Google routes its SERP off both your IP location and your hl/gl URL params, so we set them explicitly.

curl 'https://api.crawlbase.com/?token=YOUR_TOKEN' \
  --data-urlencode 'url=https://www.google.com/search?q=samsung+social+accounts' \
  --data-urlencode 'scraper=google-serp' \
  --data-urlencode 'country=US' -G

Sample response

{
  "query": "samsung social accounts",
  "results": [
    {
      "title": "Samsung Mobile (@SamsungMobile) / X",
      "url": "https://x.com/SamsungMobile",
      "snippet": "The official Samsung Mobile X account…"
    }
  ],
  "related_searches": ["samsung instagram", "samsung official tiktok"],
  "people_also_ask": [
    {
      "question": "Does Samsung have a TikTok?",
      "answer": "Yes, Samsung is on TikTok at @samsung."
    }
  ]
}

Full reference (parameters, all 4 SDK languages, edge cases):Google SERP — full reference