GEOReadyBot

❌ Bypasses WAF or Security Protections

We do not attempt to evade IP-blocks, bot-blocking rules, CAPTCHAs, or any security measures. If blocked, the bot stops immediately and records the site as "not AI-crawlable."

❌ Spoofs IP Addresses

No proxy rotation, residential IP masking, or IP spoofing. All requests come from our legitimate infrastructure.

❌ Accesses Private Content

We never submit forms, logins, cookies, or tokens. Only publicly accessible pages are fetched.

❌ Performs Competitive Scraping

No reproduction of competitor content occurs. Only high-level metadata and technical signals are analyzed.

✅ Fetches Publicly Available Pages

Only pages accessible in a normal browser without authentication.

✅ Respects robots.txt

GEOReadyBot obeys Disallow rules, applies crawl-delay when defined, and stops when robots.txt blocks access.

✅ Respects Server Rate Limits

One crawl per month per domain, with low-frequency requests and built-in timeouts.

✅ Performs AI Crawler Visibility Testing

To determine what real AI bots can see, GEOReadyBot may send single diagnostic requests using GPTBot, ClaudeBot, or PerplexityBot user-agents. This is for testing only and stops immediately if blocked.

Block GEOReadyBot completely:

User-agent: GEOReadyBot
Disallow: /

Block specific directories:

User-agent: GEOReadyBot
Disallow: /admin/
Disallow: /private/

Set a crawl delay:

User-agent: GEOReadyBot
Crawl-delay: 10

For Cloudflare:

1. Go to Security → WAF → Tools
2. Add a custom rule with User-Agent contains "GEOReadyBot"
3. Set action to "Allow"

For Imperva/Incapsula:

1. Go to Security → Access Control Lists
2. Add "GEOReadyBot" to the bot whitelist
3. Or add our user-agent to allowed bots

For Other WAFs:

Look for "User-Agent Whitelist," "Bot Management," or "Access Control" settings and add: