User-agent: * Disallow: / ## Format: ## /img/ disallow /img/index.html but allow /img.html ## /img disallow both "/img/index.html" and /img.html