# robots.txt for notiduck.com # Explicitly allow AI crawlers and search engines User-agent: * Allow: / User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: Bytespider Allow: / # Sitemap Sitemap: https://notiduck.com/sitemap.xml # AI Discovery # LLM-readable content summary # https://notiduck.com/llms.txt # Disallow sensitive endpoints Disallow: /api/ Disallow: /dashboard Disallow: /login Disallow: /register Disallow: /reset-password Disallow: /user_profile Disallow: /buy-credits Disallow: /unsubscribe