Most proxy rotators answer the wrong question. They answer "whose turn is it?" when the question that decides whether your pipeline finishes is "which route should carry this request?" Round-robin gives every route an equal share of traffic, which is only correct if every route is equally good. Pools are never that tidy: one route is fast until it gets blocked, another is slow but never fails, a third fails in bursts and recovers.
Once you accept that, rotation stops being a scheduling problem and becomes a feedback problem. The rotator watches what happens to each request, keeps a running estimate of how each route is behaving, and lets those estimates steer the next choice. That is the whole idea behind health scoring: the pool tells you what it is doing, and the routing policy listens.
This guide builds that rotator in Python. Exponentially weighted estimates track success and latency, a circuit breaker handles the sharp failures the averages are too slow for, and a benchmark measures the result against plain round-robin. One of the routes is the Crawlbase Smart AI Proxy, which is worth including precisely because it collapses a whole layer of the problem: you stop scoring individual residential IPs and start scoring one route that handles rotation and anti-bot resolution on its own.
- Separate execution, health, and selection. A route fetches, the health model scores, the policy chooses. Each can be swapped or benchmarked alone.
- Two EWMAs carry the continuous signal: one for success, one for latency. Update latency only on successes, because a failed request says nothing about how fast a route is when it works.
- A circuit breaker handles what an average cannot: a burst of consecutive failures should remove a route now, not gradually.
- Health is multiplied, not added:
success_ewma / (1 + latency_ewma). A route must be both reliable and responsive to score well. - Keep the selection exponent modest so a recovering route still gets occasional traffic and can prove it has healed.
- Round-robin is the control, not a straw man. Without it you cannot show the feedback loop did anything.
Why rotation needs health scoring
A proxy pool is not homogeneous, and it does not hold still. Routes differ in latency, reliability, and how a given target treats them, and all three can change while a job is running. Treating them as interchangeable makes the policy blind to the only conditions that matter.
A health-aware rotator needs three signals, and they are not the same signal at different sensitivities:
- Liveness. Which routes are succeeding right now, and away from which should traffic move.
- Latency. Among the routes that work, prefer the responsive ones. A successful request that took nine seconds still cost you nine seconds.
- Failure stability. Repeated failure should pull a route out of normal traffic and let it back through a controlled probe, not an immediate return to the pool.
Round-robin supplies none of it. It spreads requests evenly whether a route is healthy, slow, or dead. That is exactly what makes it the right control: run both policies against the same workload and the difference is the value of the feedback.
The shape of the system
Three responsibilities, kept apart on purpose. A route performs a fetch and reports a normalized result: succeeded or not, HTTP status, how long it took. A health model turns that stream of results into a score. A policy reads the scores and picks the next route. Because they are separate, you can replace the policy without touching transport, or benchmark two policies against identical routes.
The Smart AI Proxy earns its place in that picture by absorbing a layer. It presents one endpoint and handles residential IP rotation and anti-bot resolution behind it, so your application scores a route rather than a fleet of addresses. Your policy still decides which retrieval path deserves the traffic.
Every snippet below is excerpted from the companion repository, ScraperHub/smart-ai-proxy-rotation-in-python-health-scoring-at-scale, which holds the runnable implementation under final/ and staged checkpoints under steps/.
Environment
Python 3.11 or newer, and a Crawlbase account for the proxied route. Your Normal token is enough here; the JavaScript token only matters when a target needs rendering to produce content, which is a separate concern from routing. Both live in the console settings.
git clone https://github.com/ScraperHub/smart-ai-proxy-rotation-in-python-health-scoring-at-scale.git cd smart-ai-proxy-rotation-in-python-health-scoring-at-scale/final python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate pip install -r requirements.txt cp .env.example .env # then set CRAWLBASE_TOKEN
The token belongs in the environment rather than in source, which is ordinary hygiene but also draws the configuration boundary the next section depends on.
Configuration as a contract
Configuration is the first runtime contract. The rotator should refuse to start when a required dependency is missing, rather than boot with an unusable route and discover it mid-workload.
def _required(name: str) -> str: value = os.environ.get(name) if not value: raise RuntimeError(f"Missing required environment variable: {name}") return value @dataclass(frozen=True) class Config: crawlbase_token: str = field(default_factory=lambda: _required("CRAWLBASE_TOKEN")) smart_proxy_host: str = field(default_factory=lambda: os.environ.get("SMART_PROXY_HOST", "smartproxy.crawlbase.com")) smart_proxy_port: int = field(default_factory=lambda: int(os.environ.get("SMART_PROXY_PORT", "8012"))) success_decay: float = field(default_factory=lambda: float(os.environ.get("SUCCESS_DECAY", "0.3"))) latency_decay: float = field(default_factory=lambda: float(os.environ.get("LATENCY_DECAY", "0.3"))) breaker_threshold: int = field(default_factory=lambda: int(os.environ.get("BREAKER_THRESHOLD", "3"))) breaker_cooldown_s: float = field(default_factory=lambda: float(os.environ.get("BREAKER_COOLDOWN_S", "15")))
Note what the split says. CRAWLBASE_TOKEN is a credential and is required. Everything else is policy: how fast the success estimate reacts, how fast the latency estimate reacts, how many consecutive failures open the breaker, and how long an opened route stays out. Freezing the dataclass means the routing path reads a fixed configuration instead of reaching for environment variables at request time.
The baseline worth beating
A route here is anything that can fetch a URL and report what happened. The implementation ships two: a direct connection, and the Smart AI Proxy.
class CrawlbaseRoute(Route): name = "crawlbase-smart-proxy" def __init__(self, config: Config) -> None: proxy_url = ( f"http://{config.crawlbase_token}:@" f"{config.smart_proxy_host}:{config.smart_proxy_port}" ) self._client = httpx.Client(proxy=proxy_url, verify=False, timeout=config.request_timeout_s, follow_redirects=True)
The token goes in as the proxy username with an empty password, which is how Smart AI Proxy authenticates. verify=False is not a shortcut and is worth understanding rather than copying: the proxy terminates TLS in order to add its own headers, so your client is presented with a Crawlbase certificate rather than the target's, and strict verification would reject it. This is the documented behaviour of the endpoint, not a workaround for a misconfiguration.
DirectRoute is the same interface with no proxy. It is often quicker against an unprotected target and degrades first when anti-bot controls appear, which makes it a useful source of the failure signal the health model is built to consume.
The baseline policy ignores every one of those signals, deliberately:
class NaiveRotator: def __init__(self, routes: list[Route]) -> None: self._routes = routes self._cycle = itertools.cycle(routes) def fetch(self, url: str) -> tuple[str, FetchResult]: route = next(self._cycle) return route.name, route.fetch(url)
That is the entire policy. It is deterministic and perfectly fair, and the fairness is the problem: it persists after the routes stop being equally good.
Scoring health: two averages and a breaker
An exponentially weighted moving average fits this job because it weights recent observations and forgets old ones gradually. A route that failed ten minutes ago should not be punished forever; a route that started failing thirty seconds ago should fall fast. The decay factor is the dial between those.
def observe(self, ok: bool, latency_s: float) -> None: self.samples += 1 outcome = 1.0 if ok else 0.0 self.success_ewma = self.success_decay * outcome + (1 - self.success_decay) * self.success_ewma if ok: self.latency_ewma_s = self.latency_decay * latency_s + (1 - self.latency_decay) * self.latency_ewma_s self.consecutive_failures = 0 if self.breaker_state is BreakerState.HALF_OPEN: self.breaker_state = BreakerState.CLOSED else: self.consecutive_failures += 1 if self.consecutive_failures >= self.breaker_threshold: self.breaker_state = BreakerState.OPEN self.opened_at = time.monotonic()
Two choices in there are deliberate. Latency updates only on success, because the time a failed request took describes the failure, not the route's responsiveness when it works; folding those in would make a fast-failing route look fast. And stability is handled by the breaker rather than the average, because consecutive failures are a different kind of evidence from a drifting mean and deserve a sharper response.
The two estimates then collapse into one number:
def value(self) -> float: if self.breaker_state is BreakerState.OPEN: return 0.0 latency_term = 1.0 / (1.0 + self.latency_ewma_s) return self.success_ewma * latency_term
Multiplication rather than addition is the design decision here, and it encodes a conjunction: a route has to be both reliable and responsive to score well. Sum the terms and a fast route that fails most of the time still accumulates a respectable score from its latency half. Multiply them and a near-zero success rate drives the whole value to near zero no matter how quick the failures are. An open breaker short-circuits to exactly 0.0, which is the explicit withdrawal mechanism rather than an emergent one.
The loop, one request at a time
Every request runs the same four steps: find the eligible routes, choose one by health, execute, feed the outcome back.
observe call is the only step that changes future behaviour.def fetch(self, url: str) -> tuple[str, FetchResult]: candidates = self._eligible() or self._scored chosen = self._select(candidates) result = chosen.route.fetch(url) chosen.health.observe(result.ok, result.latency_s) return chosen.route.name, result
Selection weights each eligible route by its health raised to an exponent, gamma. At zero the choice is uniform; as it grows, traffic concentrates on the best-scoring routes. Keep it modest. A large exponent produces a rotator that commits hard to whichever route happened to look good first and then has no way to discover that a demoted route recovered, because it never sends it anything.
The fallback in the first line matters more than it looks: when the breaker has opened on everything, _eligible() is empty and the rotator falls back to the full scored set rather than raising. A pool that is entirely unhealthy should still attempt the least-bad option.
What the benchmark actually shows
python src/main.py benchmark 20 Policy comparison: policy requests success rate mean latency round-robin 20 100.0% 1.589s health-weighted 20 100.0% 0.289s
Read that carefully, because the headline number is the least interesting part of it. Against an unprotected target such as example.com both routes succeed, so the success rates are identical and the entire difference lands in latency. Round-robin keeps sending half its traffic through the slower route because that is what fairness means. The health-weighted policy notices and stops.
The absolute latencies will vary between runs and targets and are not the claim. The claim is about behaviour: one policy responds to evidence and the other cannot. Point the same benchmark at a protected target and the same mechanism expresses itself through success rate instead, as the direct route's success EWMA falls, its consecutive failures trip the breaker, and traffic moves while round-robin keeps feeding requests to a route that is refusing them.
One endpoint in front of rotating residential IPs, with anti-bot resolution handled upstream, so your rotator scores a route instead of managing a fleet of addresses. Point an HTTP client at it and keep your routing policy where it belongs. Start free with up to 5,000 requests, no card.
Taking it to production
The benchmark is single-process and synchronous because that makes the control loop legible. Four things change when it is not.
Health state has to be shared. In memory, every worker keeps a private opinion of the pool, so one process can have a route broken while another cheerfully keeps using it, and the breaker never opens anywhere consistently. Moving success EWMA, latency EWMA, breaker state and failure counters into something like Redis under a consistent key per route is what makes the signal collective rather than per-process folklore.
Updates have to be concurrency-safe. Real workloads observe outcomes in parallel, and every field in observe is a read-modify-write. Without synchronization or atomic operations, two simultaneous failures can both read the same counter and write back the same increment, so a threshold of three quietly becomes a threshold of five.
The decay factors need tuning against your traffic. 0.3 is a defensible starting point, not an answer. Higher values chase recent observations and react faster, at the cost of overreacting to a transient blip. Lower values are steadier and slower to notice a genuine change. Which mistake you prefer depends on how noisy your targets are.
Health may not be your only objective. Routes differ in cost as well as quality, and the fastest route is not automatically the one you want carrying every request. Extending the scalar with a cost term lets the policy trade reliability and latency against spend instead of optimizing one dimension and being surprised by the invoice.
One boundary sits outside the model: point this at targets you are permitted to access, and respect the terms, rate expectations, and constraints that come with them. The benchmark uses example.com precisely because it is a controlled target that makes none of those demands.
Key takeaways
A rotator becomes useful the moment selection is driven by observed behaviour instead of position in a list. Three mechanisms do the work: an EWMA over success for recent reliability, an EWMA over latency for responsiveness, and a circuit breaker for the sharp failures an average smooths over. Multiplying the first two keeps a route honest on both axes; the breaker handles the case where gradual is the wrong speed.
Round-robin stays in the picture as the control that makes the improvement measurable. And the Smart AI Proxy fits as a route rather than a responsibility, absorbing IP rotation and anti-bot resolution so the policy above it can stay a routing decision.
Select, execute, observe, update, select again. Everything else in this post is a detail of how well each step is done.
Frequently Asked Questions (FAQs)
Why score routes myself if the Smart AI Proxy already rotates?
They operate at different levels. Smart AI Proxy rotates IPs inside its own pool, so your application does not need to model individual addresses. Your rotator scores routes: a direct connection against a proxied one, one regional endpoint against another, a cheap path against an expensive path. That is a decision only your application can make, because it is the one that knows what the traffic is for. The model in this post is deliberately general so it applies to whatever set of routes you actually operate.
What decay factor should I use?
Start at 0.3 for both. Each new observation contributes 30% of the updated estimate and the prior estimate carries the remaining 70%. Raise it when your targets change behaviour quickly and you need the policy to notice sooner; lower it when measurements are noisy and you would rather not have a single slow response move traffic around.
When does the circuit breaker help more than the success EWMA alone?
When failure arrives suddenly. The EWMA is a smoother by construction, so a route that dies outright still takes several observations to score low enough to matter, and every one of those observations is a wasted request. The breaker keys off consecutive failures instead and pulls the route immediately, then offers a controlled half-open probe after the cooldown rather than waiting for the average to drift back up. The two are complementary: the average handles degradation, the breaker handles collapse.
Do I need the JavaScript token for this?
No. The Normal token is sufficient for the Smart AI Proxy route used here. The JavaScript token exists for targets that need rendering before there is any content to return, which is a question about the target rather than about routing. The rotation logic is identical either way.
Can the same model route more than two options?
Yes, and it is more useful when it does. Nothing in the health model or the selection policy assumes two routes: both operate over a list. Two routes simply make the benchmark easy to read. Adding regional endpoints or a second provider is a matter of appending to the route list, and the weighted selection distributes across whatever is there.
Crawl any site at scale, without fighting infrastructure.
Crawlbase handles proxies, fingerprints, and CAPTCHAs so your team ships data pipelines instead of maintaining crawl plumbing. Up to 5,000 requests free, no card required.