How to Scrape Hotel Data from Agodaprices and ratings with Python

Agoda lists millions of hotels and short-stay properties across Asia and the rest of the world, and each search result carries exactly the structured data that drives price tracking, travel research, and competitive analysis: the hotel name, the nightly price, the guest review score, the location, and a link to the full listing. For anyone watching a market or building a travel comparison tool, that public listing data is the raw material, and collecting it by hand across a results page is slow and quickly out of date.

This guide shows you how to scrape hotel data from Agoda with Python the reliable way. You build a small, runnable scraper that fetches a rendered Agoda search page through the Crawling API, handles Agoda's scroll-based loading so the full result set appears, parses the fields you want with BeautifulSoup, and stores clean records as JSON. The whole walkthrough stays scoped to public listing data, and the legality section near the end is not boilerplate, so read it before you point this at any real volume.

What you will build

A Python script that takes a public Agoda search URL for a city, fetches the rendered results page through the Crawling API, scrolls it so every property card loads, and extracts a structured record per hotel. The running example is hotels in Kuala Lumpur. We pull these fields:

• Hotel name the name of the property as shown on the card.
• Price the nightly price displayed for the selected dates.
• Rating the guest review score that signals quality and popularity.
• Location the city or area the search was run for.
• Link the full URL of the hotel's listing page.

Why a plain request fails on Agoda

If you request an Agoda search URL with a bare HTTP client, you get a response with status 200 and almost none of the listings in the body. Two things work against you. First, Agoda builds its search results in the browser through JavaScript, and it loads more property cards only as you scroll, so the initial HTML is a thin shell that fills in after the page's scripts run and the user scrolls down. Parse that first response and you capture a few cards instead of the full page. Second, Agoda flags automated traffic quickly: datacenter IPs and request patterns that do not look like a real browser get rate-limited, IP-blocked, or challenged before they ever reach the rendered content.

So a working Agoda scraper needs three things in one request: a browser that renders the page, a way to trigger the scroll so the lazy-loaded cards appear, and an IP the platform reads as a real visitor. You can assemble that yourself with a headless browser, a scroll routine, and a pool of rotating residential proxies, but keeping that stack healthy is most of the work. The Crawling API folds all of it into a single call: you send the URL with a JavaScript token and scroll options, it renders and scrolls the page behind a trusted IP, and returns finished HTML for you to parse.

Prerequisites

You need a few things in place before writing any code. None of them take long.

Basic Python. You should be comfortable writing and running a Python script and installing packages with pip. If you are new to the parsing side, the BeautifulSoup guide is a good companion to this tutorial.

Python 3.8 or later. Confirm your version with python --version. If you do not have it, install it from python.org or through a distribution like Anaconda, and make sure Python is on your PATH.

A Crawlbase account and JS token. Sign up, open your dashboard, and copy your JavaScript (JS) token from the account docs page. Crawlbase includes 1,000 free requests to start, which is plenty for working through this guide. Treat the token like a password: it authenticates your requests, so keep it out of version control.

Set up the project

Create a virtual environment so project dependencies stay isolated, then install the libraries the scraper needs.

python --version

python -m venv agoda_env
source agoda_env/bin/activate

pip install crawlbase beautifulsoup4

On Windows, activate the environment with agoda_env\Scripts\activate instead of the source line. Two dependencies do the work: crawlbase is the official client for the Crawling API, and beautifulsoup4 parses the returned HTML so you can pull out individual fields by CSS selector. The json module ships with the standard library, so there is nothing more to install for the export step.

Step 1: Fetch a rendered Agoda page

Start by getting a finished page. Import the CrawlingAPI class, initialize it with your JS token, and request an Agoda search URL. Agoda loads cards as you scroll, so pass the scroll and scroll_interval options to make the API scroll the page before it is captured. Checking the Crawlbase pc_status before you parse keeps failures loud instead of silent.

from crawlbase import CrawlingAPI

api = CrawlingAPI({"token": "YOUR_CRAWLBASE_TOKEN"})

OPTIONS = {
    "scroll": "true",
    "scroll_interval": "20",
}

def fetch_agoda_page(page_url):
    response = api.get(page_url, OPTIONS)
    if response["headers"]["pc_status"] == "200":
        return response["body"].decode("utf-8")
    print(f"Request failed: {response['headers']['pc_status']}")
    return None

if __name__ == "__main__":
    search_url = "https://www.agoda.com/search?city=14524&adults=2&rooms=1"
    html = fetch_agoda_page(search_url)
    print(html[:500] if html else "No HTML returned")

The two scroll options matter for a lazy-loading target like Agoda. scroll tells the API to scroll the page rather than capture it immediately, and scroll_interval sets how many seconds to keep scrolling so late-rendering cards appear before the page is captured. Twenty seconds is the documented maximum and a reasonable start for a busy results page. Run the script with python agoda_scraper.py and you should see real Agoda search markup, not the shell a plain request returns. That confirms rendering and scrolling work before you write a single selector.

Step 2: Inspect the page and parse the hotel cards

Before writing selectors, open the Agoda search page in your browser, right-click a hotel card, and choose Inspect to read its HTML structure. On the Kuala Lumpur results page, each listing sits inside a property card with a handful of stable hooks you can target:

• Hotel name lives in an <h3> with data-selenium="hotel-name".
• Price sits in a <div> with data-element-name="final-price".
• Rating is in a <p> with data-element-name="review-score".
• Listing link is the href of the a.PropertyCard__Link anchor.

Load the rendered HTML into BeautifulSoup, loop over each property card, and pull those four fields. Each lookup is guarded so a card missing one field returns an empty string instead of crashing the run, and the relative link is joined to Agoda's domain to make a complete URL.

from bs4 import BeautifulSoup

CARD_SELECTOR = "div#contentContainer ol.hotel-list-container > li.PropertyCard"

def text_of(card, selector):
    el = card.select_one(selector)
    return el.get_text(strip=True) if el else ""

def extract_agoda_data(html, location):
    soup = BeautifulSoup(html, "html.parser")
    hotels = []
    for card in soup.select(CARD_SELECTOR):
        link_el = card.select_one("a.PropertyCard__Link")
        href = link_el["href"] if link_el and link_el.get("href") else ""
        hotels.append({
            "name": text_of(card, 'h3[data-selenium="hotel-name"]'),
            "price": text_of(card, 'div[data-element-name="final-price"]'),
            "rating": text_of(card, 'p[data-element-name="review-score"]'),
            "location": location,
            "link": f"https://www.agoda.com{href}" if href else "",
        })
    return hotels

The text_of helper queries one element inside a card and returns its stripped text, or an empty string when the element is absent, so a listing that omits a field does not break the loop. The card selector walks from the page's content container down to each li.PropertyCard, and the four field selectors come straight from Agoda's markup. Run this against the scrolled HTML from Step 1 and you get one record per hotel on the page.

Step 3: Assemble the full script

Now wire the pieces into one runnable script: fetch the scrolled search page, parse every hotel card, and export the records to a JSON file.

import json
from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup

api = CrawlingAPI({"token": "YOUR_CRAWLBASE_TOKEN"})

OPTIONS = {
    "scroll": "true",
    "scroll_interval": "20",
}

CARD_SELECTOR = "div#contentContainer ol.hotel-list-container > li.PropertyCard"

def fetch_agoda_page(page_url):
    response = api.get(page_url, OPTIONS)
    if response["headers"]["pc_status"] == "200":
        return response["body"].decode("utf-8")
    print(f"Request failed: {response['headers']['pc_status']}")
    return None

def text_of(card, selector):
    el = card.select_one(selector)
    return el.get_text(strip=True) if el else ""

def extract_agoda_data(html, location):
    soup = BeautifulSoup(html, "html.parser")
    hotels = []
    for card in soup.select(CARD_SELECTOR):
        link_el = card.select_one("a.PropertyCard__Link")
        href = link_el["href"] if link_el and link_el.get("href") else ""
        hotels.append({
            "name": text_of(card, 'h3[data-selenium="hotel-name"]'),
            "price": text_of(card, 'div[data-element-name="final-price"]'),
            "rating": text_of(card, 'p[data-element-name="review-score"]'),
            "location": location,
            "link": f"https://www.agoda.com{href}" if href else "",
        })
    return hotels

def save_to_json(data, filename="agoda_hotels.json"):
    with open(filename, "w") as f:
        json.dump(data, f, indent=2)
    print(f"Saved {len(data)} hotels to {filename}")

def main():
    location = "Kuala Lumpur"
    search_url = "https://www.agoda.com/search?city=14524&adults=2&rooms=1"
    html = fetch_agoda_page(search_url)
    if html:
        hotels = extract_agoda_data(html, location)
        save_to_json(hotels)

if __name__ == "__main__":
    main()

The script fetches the scrolled search page, parses every property card into a record, and writes the list to agoda_hotels.json with two-space indentation. The city=14524 parameter is Agoda's internal id for Kuala Lumpur; swap it for another city id, and add your own check-in, check-out, and occupancy parameters, to point the scraper at a different market. Because the location label is passed in rather than parsed, every record carries the city you searched for.

What the output looks like

Run the full script with python agoda_scraper.py and you get a clean structured record per hotel, ready for analysis, a database, or a spreadsheet. The price and rating values come through as Agoda formats them on the card.

[
  {
    "name": "Summer Suites KLCC By Castle Classy",
    "price": "USD34",
    "rating": "8.4",
    "location": "Kuala Lumpur",
    "link": "https://www.agoda.com/summer-suites-klcc-by-castle-classy/hotel/kuala-lumpur-my.html"
  },
  {
    "name": "Riveria City Kuala Lumpur by Guestonic",
    "price": "USD20",
    "rating": "9.2",
    "location": "Kuala Lumpur",
    "link": "https://www.agoda.com/riveria-city-kuala-lumpur-by-guestonic/hotel/kuala-lumpur-my.html"
  }
]

If you want the same data in a spreadsheet, swap save_to_json for Python's csv.DictWriter using the keys of the first record as the header. The record shape stays the same, so it drops straight into pandas for filtering by price band or sorting by review score.

Scaling across cities and staying unblocked

The single search page above is the building block. To cover more of a market, loop over a list of Agoda city ids and call fetch_agoda_page for each, tagging every batch with its location. Agoda uses scroll-based loading rather than numbered pages, so scroll_interval controls how much of a result set you capture; raise it toward the maximum on dense city pages where cards keep loading.

Even with rendering and scrolling handled, Agoda watches for scraper-shaped traffic. A few habits keep a longer run healthy, and they apply to any hard commercial target.

• Pace your requests. Fetching many city pages in a tight loop is the fastest way to get throttled or challenged. Add a short sleep between requests and vary your targets instead of crawling one path at full speed.
• Lean on rotation. A pool of residential IPs spreads requests across many real-user addresses so no single one trips a rate limit. The Crawling API handles this for you; if you roll your own stack, this is the part to get right.
• Read the status codes. A run that starts returning non-200 pc_status values is telling you the current rate or IP tier is no longer enough. Treat that as a signal to back off, not noise to ignore.

For larger crawls, the async Crawler queues requests and delivers results to a webhook, which suits running many city pages without holding open connections. For the broader playbook, see how to scrape websites without getting blocked and the guide to crawling JavaScript websites. If you want to compare hotel data across portals, the same approach carries over to scraping Expedia, TripAdvisor, and Google Hotels.

Is it legal to scrape Agoda?

Whether scraping Agoda is allowed depends on Agoda's terms of service, your jurisdiction, and what you do with the data. Agoda's terms restrict automated access and bulk data collection, so scraping can run against those terms regardless of how careful your tooling is. None of the code here changes that; it only makes the technical part work. Read Agoda's terms of use and its robots.txt, keep your request volume modest so you are not straining its servers, and treat both documents as the boundary for what you collect. Web scraping sits in a legal gray area, and the safest posture is to collect only what is publicly visible and to stay well within reasonable rate limits.

A few lines worth holding to. Collect only public listing fields: the hotel name, nightly price, guest review score, location, and listing link that anyone can see without an account. Avoid anything tied to identifiable individuals, including guest names attached to reviews or any host or guest contact details, which is personal data and falls outside public-listing scope. Privacy laws such as GDPR and CCPA apply the moment personal data enters the picture, so keep the dataset to factual property information. Do not redistribute Agoda's copyrighted content such as photos or full review text, and do not attempt to reach anything behind a login, a saved booking, or an account.

This guide is deliberately scoped to public search results because that is the line that keeps the work defensible. If your project needs more than that, or you are building something commercial on top of hotel data, the right path is a licensing arrangement: Agoda runs an official partner and affiliate program with a sanctioned API for permitted use cases, which is the correct route for bulk or production use, not a cleverer scraper.

Frequently Asked Questions (FAQs)

Why does a plain request return only a few Agoda hotels?

Because Agoda builds its search results client-side with JavaScript and loads more property cards only as you scroll. The initial HTML is a shell that fills in after the page's scripts run and the page scrolls, so a raw HTTP request returns status 200 with most cards missing. To get the full page you have to render and scroll it first, which is what the Crawling API's JS token and the scroll options handle for you.

Do I need the normal token or the JS token for Agoda?

The JS token. The normal token fetches static HTML, which on Agoda is the same thin shell a plain fetch returns. The JS token renders the page in a real browser and, with the scroll options set, scrolls it before handing back the HTML, so the property cards are present when BeautifulSoup parses them.

What data can I scrape from an Agoda listing?

Public listing fields: the hotel name, the nightly price, the guest review score, the location, and the listing link. Stay on data that is visible to any visitor without an account, and avoid guest names tied to reviews or any host or guest contact details, which are personal data and fall outside the public-listing scope this guide covers.

How do I scrape hotels for a different city?

Agoda identifies each city with a numeric city parameter in the search URL (for example, city=14524 for Kuala Lumpur). Find the id by running the search in your browser and reading it from the URL, then swap it into search_url along with your own check-in, check-out, and occupancy parameters. To cover several cities, loop over a list of ids and call fetch_agoda_page for each.

My selectors return empty strings. What changed?

Almost certainly Agoda's markup. Its data-selenium and data-element-name hooks and its generated class names change without notice, so selectors that worked last month can break. Re-inspect a live page in your browser's dev tools and update the selectors. Periodic selector maintenance is normal for any production scraper.

How do I handle CAPTCHAs and blocks on Agoda?

Agoda uses CAPTCHAs and other bot-detection techniques on automated traffic. The Crawling API handles this for you with browser-based rendering and server-side IP rotation, so most challenges are dealt with before the HTML reaches your code. Beyond that, pace your requests, vary your targets, and back off when you start seeing non-200 pc_status values rather than retrying hard.

HR

Hassan Rehan

Software Engineer · Crawlbase

Software engineer at Crawlbase writing hands-on guides on rotating proxies, scraping, and the practical details of wiring proxies into real code.