Vanessa Fox announced today on the Google webmaster central blog some pretty exciting news about sitemaps and the sitemaps protocol. First of all, what exactly are XML sitemaps? As apposed to a “Normal” sitemap you see on many sites that are simply a “page of links” to every page on your website, an XML sitemap is a document containing links to pages in your site that ALL MAJOR Search Engines can read and process.
The advantage of Search Engine Sitemaps (or XML sitemaps) over a normal “page of links” sitemap is that you can:
1) Specify the priority of pages to be crawled and/or indexed.
2) Exclude lower priority pages.
3) Insure that Search Engines know about every page on your website.
Until today, only Google, Yahoo and MSN supported this protocol. Today however, there is a new member to the family. Ask.com. On this blog, I mainly cover a lot of Google Search Engine Optimisation and Search Marketing but you may not be aware that my favorite Search Engine is actually Ask. Although it has only minimal market share, over the past year Ask has consistently converted better than all other Search Engines combined.
That’s not the only good news. Until today, it had been quite a daunting task to create a Search Engine Sitemap and submit it correctly to the major Search Engines. In many cases, a webmaster would have to carefully craft a Sitemap and submit it to each Search Engine manually through a special webmaster interface that can be somewhat complicated to the average site owner. No longer! Vanessa announced that all Search Engines have now agreed to accept Sitemap submissions through the robots.txt file on your server.
The robots.txt file is a Search Engine industry standard file that is the VERY FIRST file a LEGIT Search Engine will view when it first comes to your website. Now, you can simply add your sitemap URL to this file in the form of:
Sitemap: http://www.mysite.com/sitemap.xml
Simply create a sitemap with the free Search Engine Sitemap Generator and upload it to your server. Then open the robots.txt file on your server and add the address as above. This makes it as simple as ever to insure that all Search Engines know about your site, know about what pages are in your site and know what pages of your site to list in the search results.
Finally, it should be noted that a Search Engine Sitemap is NO SUBSTITUTION for proper Search Engine Optimisation and link building and is no guarantee of being listed in the Search Engines. Vanessa also makes a very good point that if possible, you should still submit your sitemaps to the dedicated webmaster sections with each search engine as they will be able to provide additional statistics and alert you to any errors with your Sitemap.