Select at least one page type to generate robots and noindex recommendations.
Website launch
Robots Crawl Budget Advisor
Choose the page types your site exposes, then generate practical robots.txt, noindex, and canonical recommendations before going live.
Blocked0/100
Selected page types0
Disallow rules0
Noindex rules0
Canonical notes0
Page types to control
Pick crawl-waste or index-risk areas. Noindex pages are intentionally not blocked in robots.txt.
Crawl budget checks
robots.txt preview
User-agent: * Disallow:
Noindex preview
No noindex directives recommended.
Canonical guidance
No canonical changes recommended.
Report preview
# Robots Crawl Budget Advisor - Selected page types: 0 - Disallow rules: 0 - Noindex rules: 0 - Canonical notes: 0 ## Warnings - Select at least one page type to generate robots and noindex recommendations. ## robots.txt User-agent: * Disallow: ## Noindex directives No noindex directives recommended. ## Canonical guidance No canonical changes recommended.