Robots.txt not updating

by

When I try to google search my web address, the follow comes up as the meta description

“A description for this result is not available because of this site’s
robots.txt – learn more.”

The website
I have tried to edit the robots.txt (downloaded the WP Robot plugin) and although I have changed it to 

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

The robots.txt still appears as 

User-agent: *
Disallow: /

I have Yoast SEO, WP Robots, and Salient’s Theme Nectar.
If anyone might be able to provide me with a solution that would update the robots (or where to go from here) that would be incredibly helpful!

4 s
4

Once I experienced the same issue, this is what I did to fix the issue.

Edit the robots.txt file directly (using FTP/SSH),

User-agent: * 
Disallow: /wp-admin/ 
Disallow: /wp-includes/

There are two reasons if the robots files not updated when you edited using a plugin.

  1. File permission.
  2. Some other plugin is reverting the changes.

Also try to update the search engine visibility settings (this might not be an issue, but just have a look).

Settings -> Reading -> Search Engine Visibility (Uncheck this option) -> Save

I hope this helps.

Leave a Comment