Adnan123 asked
Member (7 upvotes)

How to use robots.txt crawl-delay

What is robots.txt crawl-delay, and in what case would it be useful to use?

Avatar for member KnowOneSpecial
Moderator (84 upvotes)

You don't use it at all.  It has no effect.

As of July 1st 2019  Google no longer uses, or supports any of the following directives in robots.txt

  • noindex
  • nofollow
  • crawl-delay

Google has many times stated not to use these directives.

Good advice?
Comments 0
login to reply
Copyright ©2024 SEOChat. All Rights Reserved. Owned and operated by Search Ventures Ltd and Chris Chedgzoy.