How to Scrape Images from DeviantArtpublic galleries, downloaded

DeviantArt is one of the largest online communities for digital artists, with millions of members publishing paintings, illustrations, photography, pixel art, and concept work every day. Its public search and gallery pages are a useful index of what styles are trending, who is publishing in a given genre, and how a body of work is organised, which is exactly the kind of catalogue a research project, a personal mood board, or a visual-trends study wants to read programmatically instead of scrolling by hand.

This guide shows you how to scrape images from DeviantArt with Python. You will build a small, runnable scraper that fetches a rendered public search page through the Crawling API, parses each result with BeautifulSoup, collects the artwork's metadata, and downloads the thumbnail files to disk. The whole walkthrough stays scoped to public gallery pages, and the legality section near the end is not boilerplate: DeviantArt artwork is copyrighted by the people who made it, so read that part before you point this at anything real.

What you will build

A Python script that takes a public DeviantArt search URL, retrieves the rendered HTML through the Crawling API, and extracts a structured record for every artwork in the results grid. We will use a keyword search as the running example and pull these fields from each result card:

• Title the artwork's title, for example "Owl #6".
• Artist the author's DeviantArt username, read from the deviation link.
• Image URL the thumbnail image source shown in the results grid.
• Page URL the link to the individual deviation page.
• Favourites the public favourites count, when the card exposes it.
• Views the public views count, when the card exposes it.

Why a plain request fails on DeviantArt

If you request a DeviantArt search URL with a bare HTTP client, you get a response with status 200 and almost none of the artwork data in the body. Two things work against you. First, DeviantArt builds its results grid in the browser with JavaScript, so the initial HTML is a shell that only fills in after the page's scripts run. Second, the platform flags automated traffic quickly: datacenter IPs and request patterns that do not look like a real browser get challenged or blocked before they ever reach the rendered thumbnails.

So a working DeviantArt scraper needs two things in one request: a browser that actually renders the page, and an IP the platform reads as a real visitor. You can assemble that yourself with a headless browser plus a pool of rotating residential proxies, but stitching those together and keeping them healthy is most of the work. The Crawling API folds both into a single call: you send it the URL with a JavaScript token, it renders the page behind a trusted IP, and it returns finished HTML for you to parse.

Prerequisites

You need a few things in place before writing any code. None of them take long.

Basic Python. You should be comfortable writing and running a Python script and installing packages with pip. If BeautifulSoup is new to you, our guide to using BeautifulSoup in Python covers the parsing basics this tutorial assumes.

Python 3.8 or later. Confirm your version with python --version. If you do not have it, install it from python.org or through a distribution like Anaconda.

A Crawlbase account and JS token. Sign up, open your dashboard, and copy your JavaScript (JS) token from the account docs page. Treat the token like a password: it authenticates your requests, so keep it out of version control.

Set up the project

Create a virtual environment so project dependencies stay isolated, then install the libraries the scraper needs.

python --version

python -m venv deviantart_env
source deviantart_env/bin/activate

pip install crawlbase beautifulsoup4 requests

On Windows, activate the environment with deviantart_env\Scripts\activate instead of the source line. Three dependencies do the work: crawlbase is the official client for the Crawling API, beautifulsoup4 parses the returned HTML so you can pull out individual fields by CSS selector, and requests streams the image files to disk in the final step.

Step 1: Fetch the rendered search page

Start by getting the finished page. Import the CrawlingAPI class, initialize it with your JS token, and request a search URL built from a keyword. DeviantArt's search path is /search?q=KEYWORD, so a query for "fantasy" gives you a grid of public results. Checking the status before you parse keeps failures loud instead of silent.

from crawlbase import CrawlingAPI

api = CrawlingAPI({"token": "YOUR_CRAWLBASE_JS_TOKEN"})

base_url = "https://www.deviantart.com"

def crawl(page_url):
    options = {"ajax_wait": "true", "page_wait": 5000}
    response = api.get(page_url, options)
    if response["status_code"] == 200:
        return response["body"].decode("latin1")
    print(f"Request failed: {response['status_code']}")
    return None

if __name__ == "__main__":
    keyword = "fantasy"
    search_url = f"{base_url}/search?q={keyword}"
    html = crawl(search_url)
    print(html[:500] if html else "No HTML returned")

The two wait options matter for a client-rendered target. ajax_wait waits for asynchronous content to finish loading, and page_wait holds for a fixed number of milliseconds after load so late-rendering thumbnails appear before the page is captured. Five seconds is a reasonable start; raise it if the grid comes back empty. The body is decoded as latin1 so the raw bytes pass through cleanly. Run the script with python scraper.py and you should see real artwork markup, not the empty shell a plain fetch returns. That confirms rendering works before you write a single selector.

Step 2: Parse the artwork cards with BeautifulSoup

With rendered HTML in hand, load it into BeautifulSoup and pull each result by its selector. DeviantArt wraps each thumbnail in a deviation link, so you select all those links once and then read the same fields from each one. Right-click a thumbnail in your browser, choose "Inspect", and you will find a structure like the one below.

<a data-hook="deviation_link"
   href="https://www.deviantart.com/siobhan-o-wisp/art/Owl-6-925596734"
   aria-label="Owl #6 by Siobhan-o-wisp, visual art">
  <div data-testid="thumb" typeof="ImageObject">
    <img alt="Owl #6" src="src_url_here" property="contentUrl" />
  </div>
</a>

That gives you everything you need from one card. The anchor carries the deviation page URL in href and the title and artist in aria-label, and the nested img carries the thumbnail in src. The CSS selector for a thumbnail image is a[data-hook="deviation_link"] img[property="contentUrl"], and you can reach the artwork title with the image's alt attribute. Here is the parser.

from bs4 import BeautifulSoup

def artist_from_url(page_url):
    # DeviantArt page URLs look like /<username>/art/<slug>
    parts = page_url.split("/")
    return parts[3] if len(parts) > 3 else None

def count_for(link, label):
    el = link.select_one(f'span[aria-label*="{label}"]')
    return el.get_text(strip=True) if el else None

def parse_artworks(html):
    soup = BeautifulSoup(html, "html.parser")
    links = soup.select('a[data-hook="deviation_link"]')
    artworks = []

    for link in links:
        img = link.select_one('img[property="contentUrl"]')
        if not img:
            continue
        page_url = link.get("href")
        artworks.append({
            "title": img.get("alt"),
            "artist": artist_from_url(page_url),
            "image_url": img.get("src"),
            "page_url": page_url,
            "favourites": count_for(link, "Favourites"),
            "views": count_for(link, "Views"),
        })

    return artworks

The title comes from the image's alt attribute, the artist from the username segment of the deviation URL, and the thumbnail from src. The favourites and views counts live in small labelled spans; count_for queries by aria-label text and returns None when a card does not expose that number, so a missing count never crashes the loop. Not every result advertises its stats in the grid, which is why both are optional fields.

Step 3: Put the scraper together

Now wire the fetch and the parse into one runnable script, add pagination so you can read more than the first grid, and write the result to JSON. DeviantArt paginates search with a &page= parameter, so a small loop over page numbers collects a broader set of results. Pause between requests so you are not hammering the site in a tight loop.

import json
import time
from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup

api = CrawlingAPI({"token": "YOUR_CRAWLBASE_JS_TOKEN"})
base_url = "https://www.deviantart.com"

def crawl(page_url):
    options = {"ajax_wait": "true", "page_wait": 5000}
    response = api.get(page_url, options)
    if response["status_code"] == 200:
        return response["body"].decode("latin1")
    print(f"Request failed: {response['status_code']}")
    return None

def artist_from_url(page_url):
    parts = page_url.split("/")
    return parts[3] if len(parts) > 3 else None

def count_for(link, label):
    el = link.select_one(f'span[aria-label*="{label}"]')
    return el.get_text(strip=True) if el else None

def parse_artworks(html):
    soup = BeautifulSoup(html, "html.parser")
    links = soup.select('a[data-hook="deviation_link"]')
    artworks = []
    for link in links:
        img = link.select_one('img[property="contentUrl"]')
        if not img:
            continue
        page_url = link.get("href")
        artworks.append({
            "title": img.get("alt"),
            "artist": artist_from_url(page_url),
            "image_url": img.get("src"),
            "page_url": page_url,
            "favourites": count_for(link, "Favourites"),
            "views": count_for(link, "Views"),
        })
    return artworks

def main():
    keyword = "fantasy"
    total_pages = 2
    results = []
    for page in range(1, total_pages + 1):
        url = f"{base_url}/search?q={keyword}&page={page}"
        html = crawl(url)
        if html:
            results.extend(parse_artworks(html))
        time.sleep(3)

    with open("deviantart_data.json", "w") as f:
        json.dump(results, f, indent=2)
    print(f"Saved {len(results)} artworks")

if __name__ == "__main__":
    main()

Each page shares the same card structure, so the parser you already wrote works across all of them without changes. The time.sleep(3) between requests keeps a multi-page run from hammering the site. Bump total_pages up only as far as you actually need, and watch the status codes as you go.

What the output looks like

Run the full script with python scraper.py and you get a clean structured record for each artwork, ready to write to JSON, CSV, or a database.

[
  {
    "title": "Owl #6",
    "artist": "siobhan-o-wisp",
    "image_url": "https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/.../owl_6.jpg",
    "page_url": "https://www.deviantart.com/siobhan-o-wisp/art/Owl-6-925596734",
    "favourites": "412",
    "views": "3.1K"
  },
  {
    "title": "Magic Forest",
    "artist": "postapodcast",
    "image_url": "https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/.../magic_forest.png",
    "page_url": "https://www.deviantart.com/postapodcast/art/Magic-Forest",
    "favourites": null,
    "views": null
  }
]

The image URLs point at DeviantArt's CDN (the wixmp.com host), and the favourites and views come back as null on cards that do not expose their stats, which is expected. With this list in hand you can index the metadata, study it, or move on to downloading the thumbnail files.

Step 4: Download the image files

The records hold thumbnail URLs, not the image bytes. To save the files locally, stream each URL to disk with requests. Streaming in chunks keeps memory flat even for larger images, and raise_for_status turns a failed download into a caught error rather than a silently empty file.

import os
import requests

def download_image(url, save_path):
    try:
        response = requests.get(url, stream=True, timeout=30)
        response.raise_for_status()
        with open(save_path, "wb") as file:
            for chunk in response.iter_content(chunk_size=8192):
                file.write(chunk)
        print(f"Saved {save_path}")
    except requests.exceptions.RequestException as e:
        print(f"Error downloading {url}: {e}")

def download_all(artworks, folder="downloaded_images"):
    os.makedirs(folder, exist_ok=True)
    for i, art in enumerate(artworks):
        url = art.get("image_url")
        if not url:
            continue
        artist = art.get("artist") or "unknown"
        path = os.path.join(folder, f"{artist}_{i}.jpg")
        download_image(url, path)

Call download_all(results) after the scrape and each thumbnail lands in a downloaded_images folder, named with the artist's username and an index. Naming by artist keeps attribution attached to the file, which matters here because every one of these images belongs to the person who made it. Read the next section before you save anything you intend to do more than glance at.

Staying unblocked

Even with rendering handled, DeviantArt watches for scraper-shaped traffic. A few habits keep a run healthy, and they apply to any hard target.

• Pace your requests. Hammering search pages in a tight loop is the fastest way to get throttled. The time.sleep in the loop is there for this reason; keep it.
• Lean on rotation. A pool of residential IPs spreads requests across many real-user addresses so no single one trips a rate limit. The Crawling API handles this for you; if you roll your own stack, this is the part to get right.
• Read the status codes. A run that starts returning challenges or errors is telling you the current rate or IP tier is no longer enough. Treat that as signal to back off, not noise to ignore.

For the broader playbook, see how to scrape websites without getting blocked. If your target is client-rendered like DeviantArt, our guide on crawling JavaScript websites explains why rendering matters. And if you would rather route your own traffic through a rotating pool instead of using the managed API, the Smart AI Proxy (also called the AI Proxy) gives you the same residential IP rotation as a drop-in proxy endpoint.

Is it legal to scrape DeviantArt?

This is the part that matters most on a creative platform, so do not skim it. The technical work in this guide is the easy half; the rights question is the hard half. Whether scraping DeviantArt is allowed depends on DeviantArt's terms of service, your jurisdiction, and what you do with the data. DeviantArt's terms place limits on automated access, so scraping can run against those terms regardless of how careful your tooling is. Read the DeviantArt Terms of Service and its robots.txt, and treat both as the boundary for what you collect.

The single most important fact here: every artwork on DeviantArt is copyrighted by the artist who created it. Collecting a public thumbnail URL or a title is one thing; what you then do with that image is another, and the law cares about the difference. Do not redistribute, republish, resell, or train models on downloaded artwork without the creator's explicit permission. Scope your work to indexing metadata and saving thumbnails for personal, research, or genuinely internal study, and keep the artist's name attached so attribution never gets lost. A scraped image is not a licensed image, and "it was public" is not a licence.

This guide is deliberately scoped to public gallery and search pages because that is the line that keeps the work defensible. It does not cover anything behind a login, anything mature-gated, account or personal data, or any attempt to bypass authentication or content gates. Public, non-gated artwork metadata only. If your project needs more than that, the sanctioned route is DeviantArt's official DeviantArt OAuth API, which exposes deviations and metadata under terms the platform actually grants. For anything beyond personal use, especially commercial reuse, the official API plus the artist's permission is the correct path, not a cleverer scraper.

Frequently Asked Questions (FAQs)

Why does a plain fetch return no artwork from DeviantArt?

Because DeviantArt builds its search grid client-side with JavaScript. The initial HTML is a shell that only fills in after the page's scripts run in a browser, so a raw HTTP request returns status 200 with the grid empty. To get real data you have to render the page first, which is what the Crawling API's JS token handles for you.

Do I need the normal token or the JS token for DeviantArt?

The JS token. The normal token fetches static HTML, which on DeviantArt's search page is the same empty shell a plain fetch returns. The JS token renders the page in a real browser before handing back the HTML, so the thumbnails and their metadata are present when BeautifulSoup parses them.

How do I scrape more than the first page of results?

DeviantArt paginates search with a &page= parameter. Loop over the page numbers you need, build a URL like /search?q=fantasy&page=2 for each, fetch it through the Crawling API, and run the same parser. Keep a short sleep between requests so a multi-page run does not hammer the site.

My selectors return None for every card. What changed?

Almost certainly DeviantArt's markup. Its class names and internal attributes change without notice, so selectors that worked last month can break. The schema hooks data-hook="deviation_link" and property="contentUrl" are the most durable, but re-inspect a live thumbnail in your browser's dev tools and update the selectors when a field comes back empty.

Can I reuse or sell the images I download from DeviantArt?

No, not without the artist's permission. Every deviation is copyrighted by its creator, so downloading a public thumbnail does not grant you a licence to redistribute, republish, resell, or train on it. Keep downloads to personal, research, or internal study, attribute the artist, and for any reuse get explicit permission or use a licensing arrangement through the official channel.

Is there an official DeviantArt API I should use instead?

Yes. DeviantArt offers an OAuth API that exposes deviations and their metadata under terms the platform grants directly. For anything beyond casual personal use, and especially for commercial projects, the official DeviantArt OAuth API is the sanctioned route and the one that keeps you on the right side of both the terms of service and copyright.

HR

Hassan Rehan

Software Engineer · Crawlbase

Software engineer at Crawlbase writing hands-on guides on rotating proxies, scraping, and the practical details of wiring proxies into real code.