1

I am working on a project that just rebranded.

Using Google Search Console I am getting some weird errors. Despite my sitemap_index.xml and robot.txt working properly, google cannot seem to fetch it for some reason.

my sitemap is here: https://example.com/sitemap_index.xml

and my robot.txt: https://example.com/robot.txt

When I try to get my site referenced on google this is what I get: enter image description here enter image description here

If I click on Open sitemaps it opens just fine.

This is what google is saying on my url inspection: enter image description here

I tried reindexing multiple times but nothing changed.

The site has been live for over a month now and is still not referenced despite having backlinks pointing to it from linked in and more.

Where can this be coming from? I asked Google support but no luck and asked my DNS provider to double check everything but it seems fine. I've also hired a DevOps to check my server configuration but apparently everything is fine.

Stephen Ostermiller
  • 99,822
  • 18
  • 143
  • 364
Miles M.
  • 111
  • 3

1 Answers1

2

It isn't clear what the "General HTTP error" is from the screenshot you posted. I can access your site, robots.txt, and sitemap myself. To resolve that problem you should:

  • Click on "1 instance" and try to see what specific error Googlebot is getting.
  • Use "inspect URL" in Google Search Console with a live test to try and reproduce the problem
  • Examine your server logs and see if there are errors.

It could be just a temporary error. If so, Googlebot will try again soon.

You can't force Google to index a site. It doesn't work that way. You can put a site up where Google can find it. If you make sure it can be crawled and make sure you have text content on each page, Google will usually want to index it. Even then, it could take months to see it data in Google Search Console.

Sitemaps have almost nothing to do with SEO. It is one way of letting Google know about the pages on your site, but it isn't the best way. Googlebot usually crawls all the URLs you put in a sitemap, however Google doesn't usually index URLs that are only found in a sitemap. Even if those URLs are indexed, they probably won't rank well without other help in the form of links to them. See The Sitemap Paradox.

The best way to get your site indexed and ranked is to:

  • Use HTML with links to build your site so that it is crawlable. Don't try to use JavaScript, AJAX, or single page application frameworks.
  • Look at your log files to make sure that Googlebot is crawling your site and you don't have too many error pages.
  • Make sure every page has multiple links from other pages on your site.
  • Get some external links to your site from your friends' and family's websites.
  • Continue to build and promote your site over the next few weeks and months.
Stephen Ostermiller
  • 99,822
  • 18
  • 143
  • 364