An XML sitemap generator plugin for WordPress puts the following strings in my robots.txt file:
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
But other WordPress blogs have lots of tags included in it.
Also, my XML file's sitemap name looks like this: sitemap.xml.gz.
How do I add other possible parameters in my robots.txt file? Is the XML file sitemap.xml.gz name valid, and what is the difference between a .xml and .gz extension?