Solution on duplicate content caused by indexed comment feeds

Of recent, I noticed that my self-hosted wordpress site was losing organic traffic, most especially from Bing. I logged into the Bing webmaster dashboard, checked on the SEO report but nothing was found that may have warranted the drop in traffic.

To check very well, I moved over to the Bing search engine, www.bing.com and typed ‘site:www.nameofmysite.com’ and behold, I saw that the search engine has indexed most of my comment feeds, alongside the rss e.t.c.

Now, the implication of this was that by the time I clicked on the indexed feed, what opens up is the post that has the feed, and this automatically means duplicate content.

This is what I mean,

I checked for ‘google homepage’, I got the following: www.ditailz.blogspot.com/google-homepage/ and also www.ditailz.blogspot.com/google-homepage/feed/

but on clicking on the feed below, it still takes you to the actual post. This is duplicate content caused by indexed comment feeds and the search engines will just penalize and downrank your site.

Solution

There are four solutions to these:

  • As it is mostly common with wordpress blogs, if you are using the default feed on your blog i.e. yousite.com/feed, it is better you switch to feedburner provided by Google and use a feedburner redirect plugin to direct this feeds to the feedburner feed. But if your theme is already using aweber or mailchimp, don’t worry as they have taken care of it or better still, they have a settings dashboard that you can use to effect any changes.

To get the feedburner from Google, go to http://feedburner.google.com and sign up and if you already have it, just log into the above address, then follow stated instructions.

  • After doing this, you will need to go the feedburner settings and tell the search engines not to index feed; to do this, follow the steps below:
  • After logging in, click on your feed name ( e.g. name of your site)
  • Click on ‘ publicize’  and towards the left of the page, scroll down and locate ‘noindex’
  • Click on it and check on the two options given; this will noindex and help remove the duplicate content penalty caused by indexed comment

Click save and make sure it shows ‘deactivate’

  • The third option is through the use of wordpress plugins and the use of robots.txt

For WordPress plugin

If you are using SEO ultimate plugin, follow the settings instruction and chose noindex of comment and feed.

Get: http://wordpress.org/plugins/seo-ultimate/

If you are using Yoast SEO, follow the settings instruction and do the same likewise.

Get: http://downloads.wordpress.org/plugin/wordpress-seo.latest-stable.zip

Use of robots.txt

A lot of wordpress users knows about robots.txt and its importance. Just add the following to your robots.txt file.

—————-

User-agent: * Disallow: /comments/feed/

Disallow: /feed/

Disallow: /feed/atom/

Disallow: /feed/rss/

Disallow: /rss/

Disallow: /trackback/

Disallow: /wp/

Disallow: /*/comments/feed/$

Disallow: /*/feed/$

Disallow: /*/feed/atom/$

Disallow: /*/feed/rss/$

Disallow: /*/rss/$

Disallow: /*/trackback/$

—————–

Note: I am not really comfortable using the robots.txt; if you want to use it, make sure that you have noindex the feed from your plugin and wait for the search engines to crawl it again ( probably for three weeks before implementing the robots).

Now, having done all this, what happens to the indexed feeds in both Google and Bing

Read: Steps to remove indexed blog posts and feed in Google and Bing

Resources:

Read more on the above resource links on how to remove duplicate content caused by indexed comment!!

Be the first to comment

Leave a Reply

Your email address will not be published.


*