# Explicitly allow Amazon Ads crawler User-agent: AmazonAdBot Allow: / # Aggressive / low-value crawlers User-agent: AionBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: LinkupBot Disallow: / # Allow normal search engines, social previews, browsers, etc. User-agent: * Disallow: