Changelog

July 2026

2026-07-24 · New scrapers: OLX

• Two new scrapers for OLX classifieds, in the E-Commerce category, across the shared frontend (olx.pl, olx.ua, olx.pt, olx.ro, olx.bg, olx.kz, olx.uz) - OLX SERP (a search or category results page as a ranked array of listings with pricing and location, plus pagination) and OLX Item (a single ad with its full attributes, price, seller, and images).

2026-07-22 · New scrapers: Exercism

• Four new scrapers for Exercism, in the Developer & Tech category - Exercism Exercises (a track exercises page as a structured array with difficulty and unlock state), Exercism Exercise (a single exercise with its full instructions as text and HTML), Exercism Solutions (an exercise community-solutions page as a paginated array of authors, languages, and stars), and Exercism Solution (a single published solution with its iteration history and full source code).

2026-07-20 · New scrapers: Stack Exchange

• Two new scrapers for the Stack Exchange network (Stack Overflow, Super User, Ask Ubuntu, Server Fault, MathOverflow, and every *.stackexchange.com site), in the Reviews & Q&A category - Stack Exchange Questions (a questions, tagged, or search-results page as a structured array with pagination) and Stack Exchange Thread (a single question with its full answer and comment threads).

2026-07-16 · New scrapers: Booking.com

• Two new scrapers for Booking.com, in the Travel, Events & Real Estate category - Booking SERP (a search-results listing as a structured array of properties with pricing and review scores, plus pagination) and Booking Hotel (a single hotel page with pricing, review scores, and facilities).

2026-07-16 · New scrapers: Product Hunt

• Two new scrapers for Product Hunt, in the Reviews & Q&A category - Product Hunt Leaderboard (daily and weekly leaderboards as a ranked array of products) and Product Hunt Product (a single product page with upvotes, makers, topics, and reviews).

2026-07-15 · New scrapers: Reddit

• Three new scrapers for Reddit, in the Social Media category - Reddit Subreddit (a subreddit listing as a ranked array of posts with pagination), Reddit Search (a Reddit search-results page as a structured array with pagination), and Reddit Post (a single post with its full comment tree).

2026-07-14 · New scrapers: GitHub

• Three new scrapers for GitHub, grouped under the new Developer category - GitHub Repository (a single repository page: stars, forks, watchers, languages, topics, license, open issues and PRs, default branch, and latest release), GitHub SERP (a repository search-results page as a structured array with pagination), and GitHub Profile (a user or organization profile: bio, followers, following, public repos, pinned repos, and organizations).

June 2026

2026-06-30 · New scrapers: Google Trends

• Two new scrapers for Google Trends - Google Trends (the "Trending now" page, returning the top trending searches with search-volume metrics and trend breakdowns) and Google Trends Explore (returning interest over time, interest by sub-region, related topics, and related queries for a given topic).

2026-06-29 · Crawler API: delete a crawler

• New POST /crawler/<TOKEN>/<NAME>/delete endpoint deletes a crawler entirely - it clears the queue and unregisters the crawler, so it no longer shows up in your dashboard or stats. Unlike Purge, which only empties the queue, delete removes the crawler itself. This action is permanent.

2026-06-06 · New scrapers: Galaxus

• Three new scrapers for Galaxus - Product, SERP, and Product Reviews - returning structured product, search, and review data from galaxus.ch.

2026-06-04 · Crawling API: PDF output

• New pdf=true parameter on the Crawling API returns the fully rendered page as a PDF (Content-Type: application/pdf) instead of HTML.
• Combine it with the rendering parameters (country, device, page_wait); PDF requests are billed the same as a JavaScript-rendered request.

2026-06-02 · Crawler management API

• Create, retrieve, and update Crawlers programmatically over REST, instead of only through the dashboard.
• New endpoints: POST /crawler/<TOKEN> to create, GET /crawler/<TOKEN>/<NAME> for details, and PUT /crawler/<TOKEN>/<NAME> to update.

May 2026

2026-05-21 · Passwordless onboarding

• Sign in with a one-time code emailed to you. No password to set, remember, or reset; existing accounts continue to work as before.
• Redesigned onboarding for new accounts. We pre-select a workflow (scraping, MCP, or integrations) from your stated use case, show a live preview of your first Crawling API request, and route you into the matching dashboard section when you finish.

2026-05-08 · New docs site

• Relaunched /docs end-to-end - restructured navigation, dark mode, and instant in-page transitions across every section.
• Command palette search (⌘K / Ctrl K) jumps straight to pages, sections, and API parameters from anywhere.
• New Ask AI button opens a Crawlbase-trained assistant in-page so you can ask anything about the docs without leaving them.
• Interactive API Playground runs real Crawling API calls right in the browser, with response headers and a rendered page preview alongside the body.
• Append .md to any docs URL (e.g. /docs/crawling-api.md) to get a clean Markdown copy you can hand to an LLM.
• Published /llms.txt as an index for AI-assistant discovery.
• Refreshed German, French, Russian, and Simplified Chinese translations across every page for sharper, more idiomatic copy.

2026-05-03 · Go SDK v0.1.0

• First official Go SDK for the Crawlbase API. Single CrawlingAPI client, dependency-free, idiomatic Go.
• Source: github.com/crawlbase/crawlbase-go. Reference docs published on pkg.go.dev.

2026-05-02 · LangChain integration v0.1.0

• langchain-crawlbase is now on PyPI - a document loader, tool, and retriever backed by the Crawling API.
• Source: github.com/crawlbase/langchain-crawlbase.

April 2026

2026-04-24 · Crawling API: Markdown output

• New format=md parameter on the Crawling API returns clean Markdown instead of HTML.
• Pair it with md_readability=true to strip nav, ads, and chrome before conversion - same idea as Reader Mode.

2026-04-23 · Tablet device option

• Crawling API device parameter now accepts tablet alongside desktop and mobile.

2026-04-23 · MCP Server v1.3.0 - storage tools

• The Crawlbase MCP server gains six storage tools so agents can list, read, and clean up Cloud Storage items between crawls - not just trigger new ones.
• Published as @crawlbase/[email protected].

2026-04-23 · Codex plugin

• The Crawlbase Codex plugin brings Crawlbase MCP into OpenAI Codex.
• Manual install today via git clone into ~/.codex/plugins/; Codex Marketplace listing in review.
• Source: github.com/crawlbase/crawlbase-codex-plugin

March 2026

2026-03-25 · Enterprise Crawler: queue_timeout

• Enterprise Crawler push now accepts queue_timeout, so you can cap how long a request sits in queue before it's dropped instead of waiting forever.

February 2026

2026-02-10 · Crawler renamed to Enterprise Crawler

• The asynchronous push/pull product is now branded Enterprise Crawler across the dashboard and docs to distinguish it from the synchronous Crawling API.
• Endpoints, parameters, and tokens are unchanged.

November 2025

2025-11-25 · MCP Server v1.2.0 - auth & HTTP mode

• The MCP server now supports header-based authentication and an optional HTTP transport mode in addition to stdio - useful for shared/remote MCP setups.
• Published as @crawlbase/[email protected].

2025-11-04 · Crawlbase brand across all locales

• The Crawlbase rename is now reflected in every translated edition of the docs (previously English-only).

October 2025

2025-10-20 · Storage API renamed to Cloud Storage

• The retrieval-and-retention product is now Cloud Storage across docs, dashboard, and SDK method names. Existing /storage endpoints continue to work.

September 2025

2025-09-26 · custom_success_code parameter

• Crawling API gains custom_success_code so you can mark non-2xx responses as successful when scraping endpoints that legitimately return e.g. 404 or 451.

2025-09-16 · Smart Proxy → Smart AI Proxy

• Smart Proxy is now branded Smart AI Proxy, reflecting the AI-driven routing and retry logic that's been added underneath. No client-side changes.

July 2025

2025-07-10 · Crawlbase MCP Server v1.0

• First public release of the Crawlbase MCP server as @crawlbase/mcp on npm - three crawl tools (crawl, crawl_markdown, crawl_screenshot) usable from Claude Desktop & Claude Code, Cursor, and VS Code / Windsurf.

June 2025

2025-06-25 · Smart Proxy: header & cookie forwarding

• Smart Proxy now forwards custom request headers and cookies through to the target site - useful for authenticated crawls and session-pinned scraping.

2025-06-18 · scroll_interval billing clarified

• Crawling API docs now spell out exactly how scroll_interval counts toward billing on long-scroll pages, so you can predict the cost of an infinite-scroll crawl before issuing it.