{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Server snippets" } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Generate Nginx, Apache, Netlify, and Vercel rules from one policy." } }
Asset Indexing Headers
X-Robots-Tag Header Planner
Choose resource rules and generate X-Robots-Tag config for Nginx, Apache, Netlify, and Vercel before launch.
Ready100/100
Critical0
Warnings0
Ready2
Rules2
Generated config
location ~* \.(pdf)$ {
add_header X-Robots-Tag "noindex, nofollow, noarchive" always;
}
location ~* \.(jpg|png|webp)$ {
add_header X-Robots-Tag "noindex, noimageindex" always;
}Report preview
# X-Robots-Tag Header Plan
- Project: Website launch assets
- Bot target: all
- Rules: 2
- Critical: 0
- Warnings: 0
- Ready: 2
## Nginx
```nginx
location ~* \.(pdf)$ {
add_header X-Robots-Tag "noindex, nofollow, noarchive" always;
}
location ~* \.(jpg|png|webp)$ {
add_header X-Robots-Tag "noindex, noimageindex" always;
}
```
## Apache
```apache
# Requires mod_headers
<FilesMatch "\.(pdf)$">
Header set X-Robots-Tag "noindex, nofollow, noarchive"
</FilesMatch>
<FilesMatch "\.(jpg|png|webp)$">
Header set X-Robots-Tag "noindex, noimageindex"
</FilesMatch>
```
## Netlify
```text
/downloads/*.pdf
X-Robots-Tag: noindex, nofollow, noarchive
/assets/originals/*
X-Robots-Tag: noindex, noimageindex
```
## Vercel
```json
{
"headers": [
{
"source": "/downloads/(.*).pdf",
"headers": [
{
"key": "X-Robots-Tag",
"value": "noindex, nofollow, noarchive"
}
]
},
{
"source": "/assets/originals/(.*)",
"headers": [
{
"key": "X-Robots-Tag",
"value": "noindex, noimageindex"
}
]
}
]
}
```{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Asset indexing rules" } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "X-Robots-Tag Generator" } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Plan X-Robots-Tag headers for files that do not have HTML meta tags, such as PDFs, images, downloads, and feeds." } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Asset-specific directives" } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Choose noindex, nofollow, noarchive, and noimageindex for different file groups." } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Safer publishing" } }
{ "t": 0, "b": { "t": 2, "i": [ { "t": 3 } ], "s": "Avoid accidentally indexing private PDFs, originals, or temporary assets." } }