# Headless Browsers (JavaScript Rendering)

Use the JavaScript token when the content you need to crawl is generated via JavaScript, either because it's a JavaScript built page (React, Angular, etc.) or because the content is dynamically generated on the browser.

# How it works?

When you send a request using your JavaScript token to the Crawling API, internally we let your request run on our headless browser infrastructure. Using the JS token allows you to benefit from more features like capturing screenshots of images and bypassing block of websites that only work on browser enabled crawlers. Check the parameters section with the list of features you can use that are only available while JS crawling.

# JavaScript API Rate Limit

By default the Crawling API is rate limited to 20 requests per second (1.728M req/day) for JavaScript crawling. If you require a higher rate limit, please contact support (opens new window) with your usecase.

# JavaScript request example

curl "https://api.crawlbase.com/?token=_JS_TOKEN_&url=https://www.missguidedau.com/petite-black-msgd-insert-mesh-cut-out-gym-leggings-10202499"

require 'net/http'

uri = URI('https://api.crawlbase.com')
uri.query = URI.encode_www_form({
  token: '_JS_TOKEN_',
  url: 'https://www.missguidedau.com/petite-black-msgd-insert-mesh-cut-out-gym-leggings-10202499'})

res = Net::HTTP.get_response(uri)
puts "Response HTTP Status Code: \#{res.code}"
puts "Response HTTP Header Original Status: \#{res['original_status']}"
puts "Response HTTP Response Body: \#{res.body}"

const https = require('https');

const url = encodeURIComponent('https://www.missguidedau.com/petite-black-msgd-insert-mesh-cut-out-gym-leggings-10202499');
const options = {
  hostname: 'api.crawlbase.com',
  path: '/?token=_JS_TOKEN_&url=' + url
};

https.request(options, (response) => {
  let body = '';
  response.on('data', chunk => body += chunk).on('end', () => console.log(body));
}).end();

$url = urlencode('https://www.missguidedau.com/petite-black-msgd-insert-mesh-cut-out-gym-leggings-10202499');

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://api.crawlbase.com/?token=_JS_TOKEN_&url=' . $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

$response = curl_exec($ch);
curl_close($ch);

var_dump($response);

from urllib2 import urlopen
from urllib import quote_plus

url = quote_plus('https://www.missguidedau.com/petite-black-msgd-insert-mesh-cut-out-gym-leggings-10202499')

handler = urlopen('https://api.crawlbase.com/?token=_JS_TOKEN_&url=' + url)

print handler.read()

package main

import (
  "net/http"
  "fmt"
  "io/ioutil"
  "net/url"
)

func main() {
  url := url.QueryEscape("https://www.missguidedau.com/petite-black-msgd-insert-mesh-cut-out-gym-leggings-10202499")
  resp, _ := http.Get("https://api.crawlbase.com/?token=_JS_TOKEN_&url=" + url)

  defer resp.Body.Close()
  body, _ := ioutil.ReadAll(resp.Body)

  fmt.Println("response Body: ", string(body))

# JavaScript Execution

If you want to execute a custom JavaScript code in the pages you want to scrape before we return your HTML, please contact support (opens new window) with your use-case.