# Veloque — veloque.in # # Copyright (c) 2026 Chetty Exports Private Limited. The content of this site # is protected work. Crawling for search and for AI answer engines is welcome # on the terms below. Crawling to build or improve a machine-learning training # corpus is NOT permitted, and this file is one of the three machine-readable # places that reservation is expressed — see also /.well-known/tdmrep.json, # the X-Robots-Tag header on every response, and the human-readable notice at # https://veloque.in/ip.php # # --------------------------------------------------------------------------- # 1. Everything else # --------------------------------------------------------------------------- User-agent: * Allow: / Disallow: /api.php Disallow: /account.php Disallow: /login.php Disallow: /logout.php Disallow: /oauth.php Disallow: /demo.php Disallow: /_backups/ Disallow: /backup.php Disallow: /team.php Disallow: /admin.php Disallow: /support-console.php Disallow: /documents.php Disallow: /brc.php Disallow: /profile.php Disallow: /dashboard.php Disallow: /rp-webhook.php # Reset links carry a single-use token in the query string; keep them out of # crawlers and out of any index that might cache the URL. Disallow: /reset.php Disallow: /forgot.php Disallow: /verify.php # --------------------------------------------------------------------------- # 2. AI answer engines — allowed, with the same exclusions as everyone else # --------------------------------------------------------------------------- # These crawlers fetch a page to answer a question and cite the source, which # is a route to Veloque, so they stay welcome. # # 2026-09-01 correction: each of these previously had a bare "Allow: /" group # of its own. A named group REPLACES the * group outright rather than adding # to it, so GPTBot, ClaudeBot and PerplexityBot were being told they could # crawl /admin.php, /team.php, /profile.php and /dashboard.php — every path # the * group takes care to exclude. Those pages all require a session and # would have served a redirect to a crawler, so nothing private was actually # exposed; the file simply was not saying what it looked like it was saying. # The exclusions are now repeated in full for each named group, which is the # only way robots.txt allows this to be expressed. User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: Claude-User User-agent: Claude-SearchBot User-agent: PerplexityBot User-agent: Perplexity-User Allow: / Disallow: /api.php Disallow: /account.php Disallow: /login.php Disallow: /logout.php Disallow: /oauth.php Disallow: /demo.php Disallow: /_backups/ Disallow: /backup.php Disallow: /team.php Disallow: /admin.php Disallow: /support-console.php Disallow: /documents.php Disallow: /brc.php Disallow: /profile.php Disallow: /dashboard.php Disallow: /rp-webhook.php Disallow: /reset.php Disallow: /forgot.php Disallow: /verify.php # --------------------------------------------------------------------------- # 3. Training-corpus and bulk-scraping crawlers — not permitted # --------------------------------------------------------------------------- # These exist to harvest text in bulk for model training or to resell it as a # dataset. They send no readers back, and they are the specific use this site # reserves against. Applebot-Extended is the training-only opt-out for Apple's # models: it does not affect Siri or Spotlight results, so blocking it costs no # search visibility at all. Google's equivalent token is treated differently # and separately — see the note immediately below. User-agent: CCBot Disallow: / # Google-Extended is DELIBERATELY NOT BLOCKED — decision of 2026-09-01. # # It is the one token on this list where blocking costs something real. Google # bundles two separate things behind it: training future Gemini models, and # GROUNDING — feeding this site's pages into Gemini's answers so it can cite # them. There is no way to permit the second without permitting the first. # # Veloque is allowed in, on purpose, because being citable in Gemini answers is # the same acquisition channel that GPTBot, ClaudeBot and PerplexityBot are # allowed in for. The trade-off was weighed and taken knowingly; it is not an # oversight, and it does not weaken the reservation below — that still binds # every crawler that is listed, and the TDM reservation in # /.well-known/tdmrep.json and in the X-Robots-Tag header still applies to this # site's content generally. # # Note for anyone auditing search visibility: blocking Google-Extended would # NOT have affected Google Search, rankings or AI Overviews — those are # Googlebot's territory and are governed by the * group above. This entry is # only ever about Gemini. User-agent: Applebot-Extended Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: meta-externalagent Disallow: / User-agent: FacebookBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Amazonbot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: Diffbot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Timpibot Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: AI2Bot Disallow: / User-agent: PanguBot Disallow: / User-agent: Scrapy Disallow: / # --------------------------------------------------------------------------- # /sitemap.xml is rewritten to sitemap.php (lastmod generated from filemtime, # so it can never go stale). Both URLs are listed: the conventional one, and # the generator directly in case the rewrite is ever unavailable. Sitemap: https://veloque.in/sitemap.xml Sitemap: https://veloque.in/sitemap.php