Software Review: A1 Sitemap Generator

By Lee Underwood

A sitemap is just that, a map of your Web site. In the beginning they were used to help visitors find their way around large Web sites, especially those with a deep directory structure. Many sites still use them, but as people become more experienced at using the Web, they're not as important. There is, however, another valuable use for sitemaps.

Many search engines now use sitemaps to help with the indexing of Web sites. This makes it easy for a search bot to crawl your site because it now has a map. In 2005 Google created its own XML-based sitemap protocol. Eventually, other search engines such as Yahoo, MSN Search and Ask joined in alliance with Google. The sitemap protocol can be found at sitemaps.org. It's a very useful tool for search engine optimization and can improve your site's listings on the major search engines.

Once you've decided to add sitemaps to your Web site, you need to figure out how to create them. You could create them by hand, but that's not practical, especially if you have a large site. Remember, each time you make changes to your Web site, you need to update your sitemap. While it's not a problem if you make changes and the site is crawled by the search bots before you update your sitemap, updating makes it easier for the bots and ensures they reach all of your site.

The easiest way to make a sitemap is to use a script or program to automatically generate it. One software program that simplifies the process is the A1 Sitemap Generator created by Micro-Sys, a Denmark-based company.

Using A1 Sitemap Generator is straightfoward; it also offers several powerful features. In addition to Google sitemaps you can create other types as well, e.g., HTML, RSS and text.

Program Basics

The program first displays the new project screen (displayed above, at right). Usually, all you need to do is enter the URL of the Web site you want crawled in the "Website directory root path" input box. You can change the type of crawler engine and set the maximum number of simultaneous connections and threads. You can specify how the program crawls your Web site and limit the sitemap to certain areas or directories. If you have a large Web site, it's recommended that you do some fine tuning.

Once the site has been crawled, you can analyze the results. The first screen (displayed above) is the "Collected data" screen. The directory of the Web site is displayed in the left panel. You can also view the links to external files if the option was set in the scan parameters. Detailed information about each file is displayed in the right panel. This includes when the file was last modified, page change frequency and the calculated "page importance score." This last feature is explained on the program's Web site as follows:

  • All links are weighted to calculate better page rank.
    • First, all links to all pages are processed and counted.
    • Some pages might have 100 incoming links and 50 outgoing links.
    • All links and pages are then processed and calculated again for "rank power."
    • A link passes on more "rank" if it resides on a page with many incoming links and few outgoing links.
    • If a page has 100 incoming links and 50 outgoing, each link passes on a page rank score value of 2.
    • The scaled score is based on a logarithm. For example, if the importance "10" equals a "rank power" of 100, "9" equals 50.
    • The algorithm calculation details may vary slightly from above.
  • Calculated priority values in created XML sitemaps.
    • The Google created XML sitemaps protocol includes a priority value field.
    • The calculations used for "page importance" are also used for priority values.
    • More information about the priority field in article priority values in XML sitemaps.

In addition, there are tabs at the bottom of the panel which provide further details, such as what other pages are linking to the current one, where the page was redirected from and a directory summary. These details provide information to help you understand how your Web site is seen by search bots and Web browsers. This information can generate better search engine rankings.

While you're analyzing the results, you can add or remove items from the sitemap. This is sometimes necessary because you might not want certain files listed. You can also change the order of the listing, as well.

There are other tabs across the top of the right panel. These allow you to view the file in an embedded browser and check its source code as a text file. You can also validate the HTML and CSS code of the file using the W3C validator. This can be valuable in aiding search engines to crawl your site.

Creating Sitemaps

The next step is the actual creation of the sitemap. The A1 Sitemap Generator allows you to create five different types of sitemaps: XML, HTML, Text, RSS and ASP.NET. If you're creating an HTML sitemap, there's a template you can customize. (You can view the HTML sitemap for Micro-Sys online.) In addition, there are other options to help you create the exact type of sitemap you need. You can create sitemaps one at a time, or all at once.

Other Features

The program will also aid you in the creation of a robots.txt file. This can help guide search engines around the site and also block access to areas that you don't want listed. (It's important to note that, while most of the search engine bots obey the rules in the robots.txt file, there are some that don't. Be aware that this is not a method of securing your Web site. There are other ways of accomplishing that.)

There's an FTP option included in the program to aid you in uploading the sitemaps to your Web site.

Note: FTPS (FTP/SSL) and SFTP (SSH FTP) aren't currently supported.

Another small bonus is the ping option. This option allows you to ping many of the search engines to notify them of your updated sitemap.

Conclusion

The A1 Sitemap Generator is a quick and easy method for creating different types of sitemaps. Since Google has placed enough importance on them as to create its own protocol, sitemaps can be a major benefit to most Web sites as far as search engines go. This software will create your sitemaps in no time, — and keep them up to date, with not much effort on your part.

Additional Links

This article originally appeared on WebReference.com.



Make a Comment

Loading Comments...

  • Web Development Newsletter Signup

    Invalid email
    You have successfuly registered to our newsletter.
  •  
  •  
  •