User-Agent: * Disallow: /aspnet_client/ Disallow: /bin/ Disallow: /config/ Disallow: /data/ Disallow: /macroScripts/ Disallow: /umbraco/ Disallow: /umbraco_client/ Disallow: /usercontrols/ Disallow: /xslt/ Disallow: /content/ Disallow: /CONTENT/ Disallow: /mediaassets/ User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: BLEXBot Disallow: / User-agent: Amazonbot # Amazon's user agent Disallow: / User-agent: Adsbot Disallow: / User-agent: SemanticScholarBot Disallow: / User-agent: bingbot Disallow: /