如何用Apache限制屏蔽恶意搜索引擎爬虫禁止抓取网站？

求知者

如何用Apache限制屏蔽搜索引擎爬虫访问，禁止蜘蛛抓取网站？

大神

apache屏蔽蜘蛛 下方代码复制到.htaccess文件

1. <IfModule mod_rewrite.c>
   
2. RewriteEngine On
   
3. #Block spider
   
4. RewriteCond %{HTTP_USER_AGENT} "SemrushBot|Webdup|AcoonBot|AhrefsBot|Ezooms|EdisterBot|EC2LinkFinder|jikespider|Purebot|MJ12bot|WangIDSpider|WBSearchBot|Wotbox|xbfMozilla|Yottaa|YandexBot|Jorgee|SWEBot|spbot|TurnitinBot-Agent|mail.RU|curl|perl|Python|Wget|Xenu|ZmEu" [NC]
   
5. RewriteRule !(^robots\.txt$) - [F]
   
6. </IfModule>

复制代码