I have iframe widgets of external sources embedded on my website. For some reason Google seems to be indexing the URLs of the iframes and showing 404 errors for such URLs in the Crawl Section of Google Webmaster Tools. How can I stop Google from doing that?
The URLs don't follow a pattern so I don't think I can use robots.txt for it. Would rel="nofollow" work for iframes?
For some reason Google seems to be indexing the URLs of the iframes
As covered here, Google will indeed try to crawl and associate framed content with the page containing the frames.
Would rel="nofollow" work for iframes?
Within the header section of the iframe page (not the parent page containing the iframe), use:
<meta name="robots" content="noindex, nofollow">
As explained by Google here, this:
instructs web crawlers to not index the page and to not crawl any of the links on the page.
(If you just want to target the Googlebot and not other crawlers, then change name="robots" -> name="googlebot")
Based on pretty extensive experience with iframes, I can confirm that this prevents iframe pages from being indexed, while still allowing the parent page containing the iframe to be indexed.
