HTTP headers are the metadata that travels alongside every request and response: they tell a server what client is calling, what content type the body holds, which credentials to check, and what language or format you expect back. When you want to send HTTP headers with cURL, the whole job comes down to one flag, but the details around it (multiple headers, overriding defaults, removing one, and reading the response) are where most people get stuck.
This guide walks through the practical mechanics from a working engineer's point of view. You will learn the -H / --header syntax, how to send one header or many, the headers that matter most in real work (User-Agent, Accept, Authorization, Cookie, and friends), how to override or strip a default header, and how to inspect what the server sends back. At the end we tie it to scraping: realistic headers make a request look like a browser, but they are only half the story, so we show how to route cURL through a residential proxy when headers alone are not enough.
The -H flag: basic syntax
cURL sends a custom header with the -H flag (or its long form --header), followed by the header name, a colon, and the value, all wrapped in quotes so the shell treats it as one argument.
curl -H "Header-Name: value" https://example.com
That is the entire pattern. The quotes matter: a header value often contains spaces, and without quoting the shell would split the value into separate arguments and cURL would reject it. The header name is case-insensitive per the HTTP spec, but it is good practice to write it in the canonical form (User-Agent, not user-agent) so your commands read clearly and match what the server logs.
If you prefer the verbose long form, --header behaves identically. Use whichever reads better in scripts; most people reach for -H at the command line and spell it out in committed code.
Sending a single header
The most common single header is User-Agent. cURL sends its own default User-Agent (something like curl/8.4.0), which is an instant tell to any server that you are not a browser. Overriding it is one flag.
curl -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" https://example.com
Now the server sees a current Chrome string instead of the cURL identifier. The same one-header pattern covers any other field. To request JSON from an API rather than whatever the server defaults to, set Accept:
curl -H "Accept: application/json" https://api.example.com/data
Sending multiple headers
Real requests rarely carry just one header. To send several, repeat -H once per header. Order is preserved, and cURL sends each line exactly as you write it.
curl -H "Accept: application/json" \ -H "Accept-Language: en-US,en;q=0.9" \ -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \ https://api.example.com/data
The backslashes let you spread the command across lines for readability; on a single line it works the same. There is no practical limit to how many -H flags you can stack, so a request that mimics a browser handshake can carry a dozen headers without any special handling.
The headers that matter most
A handful of headers do the heavy lifting in API work and scraping. Here is what each one is for and how to set it.
User-Agent
Identifies the client. Set a real, current browser string so the server treats the request like ordinary traffic. Outdated or obviously synthetic User-Agents are a common block trigger, so keep yours fresh.
Accept and Accept-Language
Tell the server what content types and languages you can handle. A browser always sends these; a bare cURL request usually does not, which is itself a fingerprint. Sending realistic values closes that gap.
curl -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" \ -H "Accept-Language: en-US,en;q=0.9" \ https://example.com
Referer
Says which page linked to the one you are requesting. Many sites expect a same-origin Referer on internal navigation and treat its absence as suspicious. Setting it to the site's own domain (or the page you would have come from) makes the request more believable.
curl -H "Referer: https://example.com/" https://example.com/products
Content-Type
Declares the format of the body you are sending. It only applies when you have a request body, so you will see it on POST and PUT, not on a plain GET. Send JSON and the server needs to know to parse it as JSON.
Authorization
Carries credentials. Two forms dominate. A Bearer token (OAuth, API keys) goes straight into the header:
curl -H "Authorization: Bearer YOUR_ACCESS_TOKEN" https://api.example.com/me
For HTTP Basic auth you can build the header by hand, but cURL has a shortcut. The -u flag base64-encodes user:pass into an Authorization: Basic header for you, which is less error-prone than encoding it yourself.
# cURL builds the Basic header for you curl -u myuser:mypassword https://api.example.com/secure # Equivalent, written out by hand curl -H "Authorization: Basic bXl1c2VyOm15cGFzc3dvcmQ=" https://api.example.com/secure
Cookie
Sends stored session state. You can pass a cookie inline with -H for a one-off request, or use cURL's -b flag, which also accepts a cookie file written by an earlier -c call so a session persists across requests.
curl -H "Cookie: session_id=abc123; theme=dark" https://example.com/account
HTTP treats header names case-insensitively, so User-Agent and user-agent reach the same field. Values, by contrast, are sent verbatim. A typo in a token or a stray space in a User-Agent is passed through untouched, so when a request behaves oddly, check the value byte for byte before blaming the server.
Overriding versus adding a header
cURL sends a few headers automatically: Host, User-Agent, and Accept among them. When you pass a -H for a header cURL already sends, your value replaces the default rather than appending a second copy. That is why -H "User-Agent: ..." swaps the cURL string out cleanly instead of producing two User-Agent lines.
For a header cURL does not send by default, your -H simply adds it. So the mental model is straightforward: if it is a default header you are overriding it, otherwise you are adding it. Either way the syntax is the same single flag.
Removing a default header
Sometimes you want a header gone entirely, not replaced. To drop a default header, pass its name followed by a colon and nothing after it. cURL reads the empty value as a signal to omit the header from the request.
# Remove the Accept header entirely curl -H "Accept:" https://example.com # Override one default and remove another in the same request curl -H "User-Agent: Mozilla/5.0" -H "Accept:" https://example.com
Note the difference: "Accept: value" sets the header, "Accept:" with no value removes it, and a trailing semicolon ("Accept;") sends the header with an empty value, which is a third, rarer case. Most of the time you want the first two.
Viewing response headers
Sending headers is half the conversation; reading what comes back is the other half. cURL gives you three options depending on whether you want the body too.
Use -I (or --head) to fetch only the response headers with a HEAD request, no body downloaded:
curl -I https://example.com
Use -i (or --include) to print the headers followed by the full body, which is handy when you want both at once:
curl -i https://example.com
Use -D (or --dump-header) to write the response headers to a file while the body goes to stdout. Pass -D - to dump headers to stdout alongside the body, which is useful when you want to keep them in a log.
# Save response headers to a file, body to stdout curl -D headers.txt https://example.com
A typical response header block looks like this, telling you the status, content type, and caching policy at a glance:
HTTP/2 200 content-type: text/html; charset=UTF-8 cache-control: max-age=3600 server: nginx
Sending custom headers on a POST
Headers come into their own on write requests. A typical JSON POST to an API needs at least a Content-Type so the server parses the body correctly, and usually an Authorization header too. The -d flag supplies the body, and cURL switches to POST automatically when it sees one.
curl -X POST https://api.example.com/items \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \ -d '{"name": "widget", "qty": 12}'
The -X POST is technically redundant once you pass -d, but spelling it out makes the intent obvious in a script. If the server expects form data instead of JSON, change Content-Type to application/x-www-form-urlencoded and pass the fields the same way.
If you are building from request basics, our walkthrough on how to send GET requests with cURL covers the read side of the same toolkit.
Headers and web scraping: what they can and cannot do
This is where header knowledge meets reality. A request from a default cURL install is trivially distinguishable from a browser: it sends the curl/x.y.z User-Agent, omits Accept-Language, carries no Referer, and brings none of the client hints a real browser includes. Servers with even basic protection key on exactly those gaps.
Setting realistic headers closes the easy gaps. A current Chrome User-Agent, a plausible Accept and Accept-Language, and a same-origin Referer together make your request look far more like ordinary browser traffic. For lightly protected pages, that alone is often enough to stop getting bounced. Our guide on cURL for web scraping goes deeper on assembling a believable header set.
But headers have a hard ceiling. They describe the request; they say nothing about where it comes from. Anti-bot systems also weigh IP reputation, request rate, TLS fingerprints, and behavioral signals. If you fire a hundred requests a minute from a single datacenter IP, no header set will save you: the address itself is the giveaway. Realistic headers reduce trivial blocks, but they do not beat serious anti-bot defenses or poor IP reputation on their own.
Routing cURL through a residential proxy
The missing half is the IP. When you route cURL through a pool of rotating residential addresses, your requests come from IPs that read as real users while you keep full control over every header. cURL already speaks to proxies natively through the -x (or --proxy) flag, so adding one is a single argument on top of the headers you already send.
The Smart AI Proxy (also called the AI Proxy) exposes residential rotation as a single drop-in endpoint. You point cURL at it with -x, authenticate with your token, and send whatever headers you like; the proxy rotates the exit IP behind the scenes.
curl -x "http://[email protected]:8012" \ -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" \ -H "Accept-Language: en-US,en;q=0.9" \ -k https://example.com
The -x flag sets the proxy, the -H flags carry your realistic headers exactly as before, and the request leaves from a trusted residential IP. You keep the full header control cURL gives you and gain the IP reputation that headers alone cannot provide. For the general mechanics of proxying any cURL request, see how to use cURL with a proxy.
Keep writing your own headers in cURL and let the IP problem solve itself. The Smart AI Proxy rotates residential addresses behind one endpoint, so your carefully crafted User-Agent, Accept-Language, and Referer leave from an IP servers trust. Point cURL at it with one -x flag and keep full control of every header.
If you want the difference between a forward proxy like this and the reverse kind sitting in front of a server, our explainer on forward versus reverse proxies draws the line clearly.
Debugging header problems
When a request behaves unexpectedly, the fastest tool is -v (verbose), which prints the request headers cURL actually sent (prefixed with >) and the response headers it received (prefixed with <). It surfaces exactly what went over the wire, which is usually enough to spot a missing header or a typo.
curl -v -H "Authorization: Bearer YOUR_ACCESS_TOKEN" https://api.example.com/me
Three problems account for most header bugs. First, missing quotes: a value with a space splits into extra shell arguments and cURL errors out, so always quote the whole name: value string. Second, an unintended override: passing -H "User-Agent: ..." replaces the default rather than adding to it, which is what you want for User-Agent but a surprise if you expected both. Third, accidental removal: a stray "Header:" with no value strips the header entirely. Run with -v and the truth is right there in the request line.
Key takeaways
-
One flag does it all.
-H "Name: value"sends any header; repeat-Hfor as many headers as you need, and order is preserved. -
Override, add, or remove. A
-Hfor a default header replaces it, a-Hfor a new one adds it, and"Header:"with no value removes it. -
Read the response too.
-Ifetches headers only,-iprepends them to the body, and-Ddumps them to a file or stdout. - Realistic headers reduce trivial blocks. A real User-Agent, Accept-Language, and Referer make a request look like a browser, but they do not beat serious anti-bot defenses.
-
Pair headers with a trusted IP. Route cURL through the Smart AI Proxy with
-xso requests leave from rotating residential addresses while you keep full header control.
Frequently Asked Questions (FAQs)
How do I add a custom HTTP header in cURL?
Use the -H flag (or its long form --header) followed by the header name, a colon, and the value, all in quotes: curl -H "Header-Name: value" https://example.com. The quotes keep the shell from splitting a value that contains spaces into separate arguments.
How do I send multiple headers in one cURL request?
Repeat the -H flag once per header. For example, curl -H "Accept: application/json" -H "Authorization: Bearer TOKEN" https://api.example.com sends both. There is no practical limit, and cURL preserves the order you write them in.
How do I set a Bearer token or Basic auth with cURL?
For a Bearer token, pass it directly: -H "Authorization: Bearer YOUR_TOKEN". For HTTP Basic auth, the -u user:pass flag is easier than building the header by hand, because cURL base64-encodes the credentials and sets the Authorization: Basic header for you.
How do I view only the response headers with cURL?
Use -I (or --head) to fetch just the headers with a HEAD request and skip the body. If you want the headers and the body together, use -i (or --include). To save the headers to a file, use -D filename.
How do I remove a default header that cURL sends?
Pass the header name with a colon and no value: curl -H "Accept:" https://example.com drops the Accept header entirely. This is different from "Accept: value", which sets it, and "Accept;", which sends it with an empty value.
Will custom headers stop my scraper from getting blocked?
Realistic headers help with lightly protected sites by making your request look like a browser, but they cannot beat serious anti-bot systems or a poor IP reputation on their own. Headers describe the request, not where it comes from. Pair them with rotating residential IPs, for example by routing cURL through the Smart AI Proxy with the -x flag, so your traffic leaves from trusted addresses while you keep full header control.
Crawl any site at scale, without fighting infrastructure.
Crawlbase handles proxies, fingerprints, and CAPTCHAs so your team ships data pipelines instead of maintaining crawl plumbing. 1,000 requests free, no card required.