Adnan123 asked
Member (7 upvotes)
over a year ago

How to use robots.txt crawl-delay

What is robots.txt crawl-delay, and in what case would it be useful to use?

Avatar for member KnowOneSpecial
Moderator (73 upvotes)
over a year ago

You don't use it at all.  It has no effect.

As of July 1st 2019  Google no longer uses, or supports any of the following directives in robots.txt

  • noindex
  • nofollow
  • crawl-delay

Google has many times stated not to use these directives.

Good advice?
Comments 0
login to reply
Copyright ©2021 SEOChat. All Rights Reserved.