Asset Indexing Headers

X-Robots-Tag Header Planner

Choose resource rules and generate X-Robots-Tag config for Nginx, Apache, Netlify, and Vercel before launch.

Ready100/100
Critical0
Warnings0
Ready2
Rules2

Generated config

location ~* \.(pdf)$ {
  add_header X-Robots-Tag "noindex, nofollow, noarchive" always;
}

location ~* \.(jpg|png|webp)$ {
  add_header X-Robots-Tag "noindex, noimageindex" always;
}

Report preview

# X-Robots-Tag Header Plan

- Project: Website launch assets
- Bot target: all
- Rules: 2
- Critical: 0
- Warnings: 0
- Ready: 2


## Nginx
```nginx
location ~* \.(pdf)$ {
  add_header X-Robots-Tag "noindex, nofollow, noarchive" always;
}

location ~* \.(jpg|png|webp)$ {
  add_header X-Robots-Tag "noindex, noimageindex" always;
}
```

## Apache
```apache
# Requires mod_headers

<FilesMatch "\.(pdf)$">
  Header set X-Robots-Tag "noindex, nofollow, noarchive"
</FilesMatch>

<FilesMatch "\.(jpg|png|webp)$">
  Header set X-Robots-Tag "noindex, noimageindex"
</FilesMatch>
```

## Netlify
```text
/downloads/*.pdf
  X-Robots-Tag: noindex, nofollow, noarchive

/assets/originals/*
  X-Robots-Tag: noindex, noimageindex
```

## Vercel
```json
{
  "headers": [
    {
      "source": "/downloads/(.*).pdf",
      "headers": [
        {
          "key": "X-Robots-Tag",
          "value": "noindex, nofollow, noarchive"
        }
      ]
    },
    {
      "source": "/assets/originals/(.*)",
      "headers": [
        {
          "key": "X-Robots-Tag",
          "value": "noindex, noimageindex"
        }
      ]
    }
  ]
}
```