Google & WordPress to test on Robots.txt

Google and WordPress

One of many classes realized from the Google Webmaster convention is that if Google tries to entry your robots.txt file is inaccessible, however that exists, Google won’t discover your web site. Google stated about 26% of the time GoogleBot can’t attain a robots.txt file. WordPress may make adjustments to cut back this price of error.

Right here is likely one of the many tweets about it:

Now, with WordPress, Joost of Yoast's Valk stated, "For websites for which you cannot entry the robots.txt file, is there a subset of those WordPress websites?" necessary than the one you usually anticipated? " He added that he "is trying to discover out if we ought to be safer in the way in which WordPress generates robots.txt information".

Google's Gary Illyes stated that he thought WordPress usually accepted this drawback, however he would think about it additional to see if WordPress may make some minor adjustments to it.

I really like this dialogue between Google and Yoast (very a lot associated to WordPress).

Dialogue Discussion board at Twitter.

Replace: I once more upset Gary and, for the document, the most recent piece of knowledge was the share of robots.txt that Google can’t attain.