A price is the one field on a Walmart product page that changes while everything else stays put. The title, the brand, the model, those hold steady, but the number next to the buy button moves with promotions, demand, and seasonal resets. If you want to track that movement, comparison-shop a basket of items, or watch a competitor's catalog, you need to read the same price again and again over time and write each reading down somewhere you can chart it.
This guide shows you how to scrape Walmart prices with Python and turn them into a running log you can analyze. You build a small, runnable scraper that fetches a rendered Walmart product page through the Crawling API, pulls the title and price out of the HTML with BeautifulSoup, and appends each reading to JSON and CSV with a timestamp so you can track the price over time. We keep the whole walkthrough scoped to public product and listing data, and the legality section near the end is not boilerplate, so read it before you point this at any real volume.
What you will build
A Python script that takes a Walmart product URL, retrieves the rendered page through the Crawling API, extracts a structured record, and appends it to a timestamped price log. We use a refurbished iPhone listing as the running example and pull these fields from each product page:
- Title the product name, for example "Restored Apple iPhone 13, Carrier Unlocked, 128GB Red".
- Price the current listed price shown next to the buy button.
- Discount the dollar amount saved, when the listing shows one.
- Rating the average star rating, when the product has reviews.
- Timestamp an ISO time added at scrape time so each reading is a point you can chart.
Why a plain request fails on Walmart
If you request a Walmart product URL with a bare HTTP client, you get a response with status 200 and almost none of the price data in the body. Two things work against you. First, Walmart builds its product pages client-side: the initial HTML is a shell that only fills in the title, price, and rating after the page's JavaScript runs in a browser. Second, Walmart flags automated traffic quickly. Datacenter IPs and request patterns that do not look like a real browser get challenged with a CAPTCHA or blocked before they ever reach the rendered price.
So a working price scraper needs two things in one request: a browser that actually renders the page, and an IP the platform reads as a real shopper. You can assemble that yourself with a headless browser plus a pool of rotating residential proxies, but stitching those together and keeping them healthy is most of the work. The Crawling API folds both into a single call: you send it the URL with a JavaScript token, it renders the page behind a trusted residential IP, and it returns finished HTML for you to parse.
Crawlbase offers two token types. The normal token fetches static HTML; the JavaScript (JS) token renders the page in a real browser first. Walmart loads price and rating into the page client-side, so you need the JS token here. Using the normal token returns the same empty shell a plain fetch would, and the price element is simply not in it.
Prerequisites
You need a few things in place before writing any code. None of them take long.
Basic Python. You should be comfortable writing and running a Python script and installing packages with pip. If you are new to the language, our walkthrough on scraping a website with Python covers the ground this tutorial assumes.
Python 3.8 or later. Confirm your version with python --version. If you do not have it, install it from python.org or through a distribution like Anaconda.
A Crawlbase account and JS token. Sign up, open your dashboard, and copy your JavaScript (JS) token from the account docs page. The free tier includes 1,000 requests with no card, which is plenty to build and test this scraper. Treat the token like a password: it authenticates your requests, so keep it out of version control.
Set up the project
Create a virtual environment so project dependencies stay isolated, then install the three libraries the scraper needs.
python --version python -m venv walmart_env source walmart_env/bin/activate pip install crawlbase beautifulsoup4 pandas
On Windows, activate the environment with walmart_env\Scripts\activate instead of the source line. Three dependencies do the work: crawlbase is the official client for the Crawling API, beautifulsoup4 parses the returned HTML so you can pull each field out by CSS selector, and pandas handles the CSV export at the end so your price history opens cleanly in any spreadsheet.
Understanding the Walmart product page
A Walmart product page packs a lot into one screen: a title, an image carousel, a price block, a star rating with a review count, a description, and seller details. For price tracking you only care about a few of those. The fields that matter are the title (so you know which item a reading belongs to), the current price, the discount or savings amount when one is shown, and the rating.
Before writing selectors, open a product page in your browser, right-click the price, and choose Inspect. You will see the price exposed through an itemprop="price" attribute inside a price-wrap container, the title in an h1#main-title element, and the savings and rating behind data-testid markers. Those attributes are what you target. Walmart's utility class names change often, but the semantic attributes are more durable, so lean on them where you can.
Step 1: Fetch the rendered product page
Start by getting the finished page. Import the CrawlingAPI class, initialize it with your JS token, and request the product URL. Checking the status code before you parse keeps failures loud instead of silent.
from crawlbase import CrawlingAPI api = CrawlingAPI({"token": "YOUR_CRAWLBASE_TOKEN"}) def crawl(product_url): options = {"ajax_wait": "true", "page_wait": 3000} response = api.get(product_url, options) if response["status_code"] == 200: return response["body"].decode("latin1") print(f"Request failed: {response['status_code']}") return None if __name__ == "__main__": url = "https://www.walmart.com/ip/Restored-Apple-iPhone-13-Carrier-Unlocked-128-GB-Red-Refurbished/462799546" html = crawl(url) print(html[:500] if html else "No HTML returned")
The two wait options matter for a client-rendered target like this. ajax_wait tells the API to wait for asynchronous content to finish loading, and page_wait holds for a fixed number of milliseconds after load so the price block appears before the page is captured. Three seconds is a reasonable start; raise it if the price comes back missing. The body is decoded as latin1 because Walmart pages mix in characters that strict UTF-8 decoding can choke on. Run the script and you should see real product markup, not the empty shell a plain fetch returns. That confirms rendering works before you write a single selector.
That price block only exists after Walmart renders the page behind a trusted IP, which is exactly what the crawl function above leans on. The Crawling API takes a JS token, runs the page in a real browser, rotates through residential IPs server-side, and hands you finished HTML, so you skip running a headless fleet and a proxy pool yourself. Point it at one product URL on the free tier first.
Step 2: Extract the title and price with BeautifulSoup
With rendered HTML in hand, load it into BeautifulSoup and pull each field by its selector. The price lives in a span carrying itemprop="price" inside the price-wrap container, the title in h1#main-title, and the savings and rating behind their data-testid markers. A small helper returns an empty string when an element is missing so one absent field does not crash the run.
from bs4 import BeautifulSoup from datetime import datetime, timezone def text_of(soup, selector): el = soup.select_one(selector) return el.get_text(strip=True) if el else "" def parse_product(html): soup = BeautifulSoup(html, "html.parser") return { "title": text_of(soup, "h1#main-title"), "price": text_of(soup, 'span[data-testid="price-wrap"] span[itemprop="price"]'), "discount": text_of(soup, 'div[data-testid="dollar-saving"] span:last-child'), "rating": text_of(soup, 'div[data-testid="reviews-and-ratings"] span.rating-number'), "scraped_at": datetime.now(timezone.utc).isoformat(), }
The text_of helper queries a single element and returns an empty string when it is missing, instead of throwing on a .get_text() call against nothing. That keeps extraction resilient when a field is absent, which is common since not every listing carries a discount or a rating. The price comes from the itemprop="price" span, the canonical place Walmart marks up the current price, and scraped_at stamps every record with the moment it was read. That timestamp is what turns a one-off scrape into a price series you can chart later.
Walmart's utility class names change without notice, and the data-testid values on the price and savings blocks shift occasionally too. Treat the selectors above as a starting template, not a contract. When the price comes back empty on a page you know shows one, re-inspect the live product page in your browser's dev tools and update the selector. Periodic selector maintenance is normal for any production scraper, not a sign something is broken.
Step 3: Append each reading to a price log
Now wire the fetch and the parse into one runnable script and append every reading to a JSON file and a CSV. Appending rather than overwriting is the whole point: each run adds one dated row per product, and over days or weeks those rows become a price history you can plot.
import json import os import pandas as pd from bs4 import BeautifulSoup from datetime import datetime, timezone from crawlbase import CrawlingAPI api = CrawlingAPI({"token": "YOUR_CRAWLBASE_TOKEN"}) PRODUCTS = [ "https://www.walmart.com/ip/Restored-Apple-iPhone-13-Carrier-Unlocked-128-GB-Red-Refurbished/462799546", ] def crawl(product_url): options = {"ajax_wait": "true", "page_wait": 3000} response = api.get(product_url, options) if response["status_code"] == 200: return response["body"].decode("latin1") print(f"Request failed: {response['status_code']}") return None def text_of(soup, selector): el = soup.select_one(selector) return el.get_text(strip=True) if el else "" def parse_product(html, url): soup = BeautifulSoup(html, "html.parser") return { "title": text_of(soup, "h1#main-title"), "price": text_of(soup, 'span[data-testid="price-wrap"] span[itemprop="price"]'), "discount": text_of(soup, 'div[data-testid="dollar-saving"] span:last-child'), "rating": text_of(soup, 'div[data-testid="reviews-and-ratings"] span.rating-number'), "url": url, "scraped_at": datetime.now(timezone.utc).isoformat(), } def append_json(record, path="walmart_prices.json"): history = [] if os.path.exists(path): with open(path) as f: history = json.load(f) history.append(record) with open(path, "w") as f: json.dump(history, f, indent=2) return history def main(): for url in PRODUCTS: html = crawl(url) if not html: continue record = parse_product(html, url) append_json(record) print(json.dumps(record, indent=2)) history = json.load(open("walmart_prices.json")) pd.DataFrame(history).to_csv("walmart_prices.csv", index=False) if __name__ == "__main__": main()
The append_json function reads the existing history, adds the new reading, and writes the whole list back, so the JSON file grows by one record each run. After the loop, the script rebuilds walmart_prices.csv from that history with pandas, which gives you a flat table that opens in any spreadsheet for charting. Add a URL to the PRODUCTS list and the same code tracks several items at once. To collect readings over time, schedule this script to run on a cron job once a day, and each run appends a fresh dated row per product.
What the output looks like
Run the full script with python scraper.py and each reading prints as JSON and lands in your price log. After a few runs across a couple of days, the JSON file holds a small series per product.
[ { "title": "Restored Apple iPhone 13, Carrier Unlocked, 128 GB Red (Refurbished)", "price": "$449.00", "discount": "$200.00", "rating": "(4.4)", "url": "https://www.walmart.com/ip/...462799546", "scraped_at": "2026-06-09T14:05:33+00:00" }, { "title": "Restored Apple iPhone 13, Carrier Unlocked, 128 GB Red (Refurbished)", "price": "$429.00", "discount": "$220.00", "rating": "(4.4)", "url": "https://www.walmart.com/ip/...462799546", "scraped_at": "2026-06-10T14:04:11+00:00" } ]
The two records share a title and URL but differ on price and date, which is exactly the shape price tracking needs. From here, loading walmart_prices.csv into pandas lets you compute the min, max, and average price per product, flag a drop below a threshold, or plot the series. If you want to turn this log into a side-by-side view across retailers, the companion guide on building a price comparison tool picks up where this leaves off.
Scraping prices across many products
Tracking one listing is a demo; a real job watches a basket of products, and often you do not have the product URLs up front. To build that basket you scrape a search results page first, collect the product links, then run the price scraper above against each one. Extracting those links from a Walmart search page is its own task, covered in detail in our guide on scraping Walmart search pages with Python. Once you have the URLs, feed them into the PRODUCTS list and the loop reads a price for each.
Pace the loop so you are not hammering Walmart in a tight burst. A short delay between product requests keeps your request rate low and your run healthy, the same discipline any hard commercial target rewards.
import time def track_prices(urls): readings = [] for url in urls: html = crawl(url) if not html: continue record = parse_product(html, url) append_json(record) readings.append(record) print(f"{record['title'][:40]}: {record['price']}") time.sleep(2) return readings
The time.sleep(2) between products is the floor, not the ceiling: raise it for larger baskets. Because append_json writes after every product, a run that fails partway still keeps the readings it already collected, so you do not lose a long job to one bad page.
Staying unblocked
Even with rendering handled, Walmart watches for scraper-shaped traffic. A few habits keep a run healthy, and they apply to any hard commercial target.
- Pace your requests. Spread requests out with a delay between products and avoid re-scraping the same listing more often than you need. For price tracking, once or twice a day per product is usually enough to catch every meaningful change.
- Lean on rotation. A pool of residential IPs spreads requests across many real-user addresses so no single one trips a rate limit. The Crawling API handles this for you; if you roll your own stack, this is the part to get right.
- Read the status codes. A run that starts returning challenges or errors is telling you the current rate or IP tier is no longer enough. Treat that as a signal to back off, not noise to ignore.
For the broader playbook, see our guide on how to scrape websites without getting blocked. If you want to scrape a single listing in depth, including variant and spec data, the companion guide on scraping a Walmart product page with Selenium goes deeper than price alone, and the broader case for tracking competitor prices is laid out in using web scraping for price intelligence.
Is it legal to scrape Walmart?
Whether scraping Walmart is allowed depends on Walmart's terms of service, your jurisdiction, and what you do with the data. Walmart's terms restrict automated access, so scraping can run against those terms regardless of how careful your tooling is. None of the code here changes that; it just makes the technical part work. Read Walmart's Terms of Use and its robots.txt, and treat both as the boundary for what you collect.
A few lines worth holding to. Collect only public data: the product titles, prices, discounts, and ratings that anyone can see on a product or search page without an account. Respect Walmart's stated rate expectations and keep your request volume low enough that you are not straining its servers, which for price tracking is easy because one reading a day per product is plenty. Avoid personal data, including anything tied to identifiable shoppers, reviewers, or sellers beyond what is publicly listed. If you plan to reuse the prices commercially, get permission or an official agreement rather than assuming silence is consent.
This guide is deliberately scoped to public product and listing pages because that is the line that keeps the work defensible. It does not cover anything behind a login, account or order data, personal information, payment or checkout flows, or any attempt to bypass authentication. For licensed or bulk access, Walmart offers official APIs and partner programs, and that is the right tool when you need large volumes, guaranteed structure, or commercial rights. If your project needs more than public prices, an official API or a data agreement is the correct path, not a cleverer scraper.
Key takeaways
- Walmart prices are client-side rendered. A plain fetch returns an empty shell, so you must render the page with a JS token before the price element exists to parse.
-
Target durable attributes. Pull the price from the
itemprop="price"span and the title fromh1#main-title; expect the surrounding class names to drift and re-inspect when a field comes back empty. -
A timestamp turns a scrape into a series. Stamp every reading with
scraped_atand append rather than overwrite, and your JSON and CSV become a price history you can chart. - Scale from search to basket. Collect product URLs from a search page, feed them into the loop, pace requests with a delay, and run on a daily schedule to track over time.
- Stay on public data. Respect Walmart's ToS and robots.txt, prefer an official Walmart API for licensed or bulk data, and never touch accounts, orders, or personal information.
Frequently Asked Questions (FAQs)
Why does a plain request return no price from Walmart?
Because Walmart loads the price into the page client-side with JavaScript. The initial HTML is a shell, and the price element only appears after the page's scripts run in a browser, so a raw HTTP request returns status 200 with the price blank. To read it you have to render the page first, which is what the Crawling API's JS token handles for you.
How do I track a Walmart price over time?
Run the scraper on a schedule, once or twice a day, and append each reading to the same JSON and CSV file with a timestamp. The append_json helper in this guide adds one dated record per run instead of overwriting, so the file becomes a price series. Load that CSV into pandas to compute min, max, and average, or to flag a drop below a threshold.
Can I scrape prices for many products at once?
Yes. Add each product URL to the PRODUCTS list, or pass a list of URLs to the track_prices function, and the loop reads a price for each. If you do not have the URLs up front, scrape a Walmart search page first to collect product links, then feed those links into the loop. Keep a short delay between requests so you pace the run.
My price selector returns an empty string. What changed?
Almost certainly Walmart's markup. Its utility class names change without notice, and the data-testid values around the price and savings blocks shift occasionally too. Re-inspect a live product page in your browser's dev tools, prefer the durable itemprop="price" and h1#main-title targets where you can, and update the selectors. Periodic maintenance is normal for any production scraper.
Should I store prices in CSV, JSON, or a database?
For a few products tracked over weeks, the JSON and CSV files in this guide are enough and open anywhere. As the history grows or you track hundreds of items, move to a database like SQLite or PostgreSQL so you can query by date range and product without loading the whole file. The record shape stays the same; only the storage layer changes.
How do I avoid getting blocked while scraping Walmart prices?
Keep your per-IP request rate low, add a delay between products, do not re-scrape the same listing more often than you need, and route through rotating residential IPs so no single address trips a rate limit. The Crawling API manages rotation and a trusted IP pool for you; if you build your own stack, that is the part to invest in. Watch the status codes and back off when you start seeing challenges.
Crawl any site at scale, without fighting infrastructure.
Crawlbase handles proxies, fingerprints, and CAPTCHAs so your team ships data pipelines instead of maintaining crawl plumbing. 1,000 requests free, no card required.