# Start Bot Craw User-agent: * Allow: / # Disallow Disallow: /.cache/ User-agent: similarweb Disallow: /