Robots & sitemaps

SEO agencies and platform engineers — pull robots.txt and sitemap URLs without driving your local IP.

• /v1/discover — robots body + Sitemap: list
• /v1/robots — raw robots.txt
• /v1/sitemap — fetch a sitemap URL (truncated in JSON if huge)