1. Make sure ChatGPT can actually reach your site
OpenAI's search crawler is called OAI-SearchBot. OpenAI says that for your content to show up in ChatGPT search summaries and snippets, you must not block OAI-SearchBot in robots.txt. Your host or CDN also has to let OpenAI's published IP addresses through (OpenAI Publishers FAQ).
Check three things:
- Your
robots.txtdoes not includeDisallow: /for OAI-SearchBot. Many sites blocked AI crawlers in bulk in 2023–2024 and never went back to check. - Your firewall or bot protection isn't quietly blocking it.
- Your key facts are in the page HTML, not loaded later by JavaScript. Many AI crawlers read raw HTML, so text that only shows up after scripts run may never be seen.
GPTBot is a separate OpenAI crawler, used for model training. Blocking GPTBot doesn't affect OAI-SearchBot. Decide on each one separately.