Asset Indexing Headers
X-Robots-Tag Header Planner
Choose resource rules and generate X-Robots-Tag config for Nginx, Apache, Netlify, and Vercel before launch.
Ready100/100
Critical0
Warnings0
Ready2
Rules2
Generated config
location ~* \.(pdf)$ {
add_header X-Robots-Tag "noindex, nofollow, noarchive" always;
}
location ~* \.(jpg|png|webp)$ {
add_header X-Robots-Tag "noindex, noimageindex" always;
}Report preview
# X-Robots-Tag Header Plan
- Project: Website launch assets
- Bot target: all
- Rules: 2
- Critical: 0
- Warnings: 0
- Ready: 2
## Nginx
```nginx
location ~* \.(pdf)$ {
add_header X-Robots-Tag "noindex, nofollow, noarchive" always;
}
location ~* \.(jpg|png|webp)$ {
add_header X-Robots-Tag "noindex, noimageindex" always;
}
```
## Apache
```apache
# Requires mod_headers
<FilesMatch "\.(pdf)$">
Header set X-Robots-Tag "noindex, nofollow, noarchive"
</FilesMatch>
<FilesMatch "\.(jpg|png|webp)$">
Header set X-Robots-Tag "noindex, noimageindex"
</FilesMatch>
```
## Netlify
```text
/downloads/*.pdf
X-Robots-Tag: noindex, nofollow, noarchive
/assets/originals/*
X-Robots-Tag: noindex, noimageindex
```
## Vercel
```json
{
"headers": [
{
"source": "/downloads/(.*).pdf",
"headers": [
{
"key": "X-Robots-Tag",
"value": "noindex, nofollow, noarchive"
}
]
},
{
"source": "/assets/originals/(.*)",
"headers": [
{
"key": "X-Robots-Tag",
"value": "noindex, noimageindex"
}
]
}
]
}
```Asset indexing rules
X-Robots-Tag Generator
Plan X-Robots-Tag headers for files that do not have HTML meta tags, such as PDFs, images, downloads, and feeds.
Server snippets
Generate Nginx, Apache, Netlify, and Vercel rules from one policy.
Asset-specific directives
Choose noindex, nofollow, noarchive, and noimageindex for different file groups.
Safer publishing
Avoid accidentally indexing private PDFs, originals, or temporary assets.