# robots.txt — Gurug.com # Optimized for SEO, AI crawling, and SEMrush technical guidelines User-agent: * Disallow: /admin/ Disallow: /user-data/ Disallow: /checkout/ Disallow: /cart/ Disallow: /profile/ Disallow: /private/ Disallow: /legal/agreements/ Allow: / # Allow transparent access to AI policy file Allow: /llms.txt # Primary sitemap Sitemap: https://gurug.com/sitemap.xml # Explicitly ensure Google's ad-review bot has full access User-agent: AdsBot-Google Allow: / User-agent: AdsBot-Google-Mobile Allow: /