Perhaps the most commonly asked question when talking with potential customers about a new website for their business has to do with search engine ranking. The purpose of this document is to clear the air about what a search engine is, who runs them, how to get ranked well within them, and how they affect traffic to your site. Every site owner wants to get a high listing within the popular search engines, including Yahoo, AltaVista, Lycos, AOL Search, and others. The idea here is that a high ranking in search results translates into higher traffic to your site which hopefully means increased business for you.

First, I am going to explain the difference between a "Search Site" and a "Directory Portal". A Search Site is basically a place where you enter a keyword such as "car" and hundreds of thousands of results are returned. The problem with this is that no one can reasonably expect to be guaranteed placement in the first page or two of these results. The other problem is that many of the listings don't necessarily match the user's intended search. Let us say that he entered "car" looking for a "car dealership". Unfortunately, all webpages that have the word "car" mentioned in it, whatever the context, will show up in the search results.

Because of this, the average surfer quickly learns to enter the phrase "used car dealership" if that is what he is looking for. Still, the results are less than focused, as any site that has the words "used", "car", and "dealership" in it will be included in the results. This surfer might actually wind up with more results that are irrelevant than if he just typed in the word "car". This is where the concept of "Directory Portals" comes into play.

Most of the popular search sites are actually primarily directory portals. These sites include the likes of Yahoo, MSN, AltaVista, WebCrawler, Excite, and many others. The best way to think of these sites is that they have two types of results that they return as a result of a search. The primary set of results will be from their own internal "directory" list and the second set of results will be from a general "search results" list.

This concept of a "directory" list is identical to how a normal yellow pages phone book is organized. One can easily flip to the section of the phone book that lists all the used car dealers in a particular location. The upside to this is that the results of a search will be well focused as you can be sure that going to - Home > Business and Economy > Shopping and Services > Automotive > Dealers > By Region > U.S. States > Virginia in Yahoo will yield a list of nothing but car dealers in Virginia. The downside is that since the "dotcom" bust of the last couple of years, most directory portals are now charging money to get a listing in their directory, especially if it is a business listing.

What this means is that all those website promotion companies out there that promise "top ranking" in all searches are lying to you. Getting a top listing in any search engine or directory listing is far from certain, even if you are willing to pay sites like Yahoo for inclusion. Your ranking all depends on how popular a particular category is and how relevant the site reviewer finds your website and how he places you in their system. The bottom line is that most directory portals will require money before even considering your site for their system and inclusion is not guaranteed.

Now this is not all bad news. One has to have a clear idea of the place that a website lives in one's marketing agenda. If you are running a local business such as a car dealership or an antique store, the odds are likely that your customers are not flying in from thousands of miles away to buy your goods. Instead, they live locally to your business and already are aware of your presence. So, what purpose does your website serve? Well, if you have a site composed of just 4 or 5 static pages, the answer is "not much". However, if your site has your inventory listed in it, then it does serve as a valuable selling tool that will bring in more business. This is because more and more people like to surf the net to check out items for potential purchase.

This is especially true for sites that list high ticket inventory items such as cars. These people can view your inventory from the comfort of their own home and statistics have shown that 2 out of 3 car buyers use the web to help them look for a car. So, what do you do to promote your site? The answer here is really quite fundamental.

How I have found the best way to look at your website is that it should be considered an adjunct to what you are already doing in the way of advertising. Treat your web address as you would your phone number. Put it on all your business cards, in all your print advertising, mention it in any radio or TV commercials, even mention it on your answering machine inviting callers on Sunday and after hours to check out your online inventory from the comfort of their home. The bottom line is if you really want to get the top search engines to list you, be prepared to spend in excess of $1000 for the privilege but you should not overlook what you can do inhouse to get your website "out there".

You can spend the money to get listed in these Directory portals but you do have to weigh the cost vs. the benefits and decide if doing so is right for you. Further down this page you will find a concise listing of the major players in the search engine game and what their requirements are for getting listed. I hope the information in this page helps you to understand how search engines work and whether your site will benefit from getting listed in them.

The stockNum System

I have just completed work on a brand new inventory control system that is specifically designed to provide advanced inventory control for website owners that wish to be able to post their goods online. This system is a major improvement over systems that I have created in the past, including those used by Auto-Corner, DianeCrump and Lokers Flat. One of the key differences is that I have coded a system that automatically builds your site in such a way that every page is a static html document instead of using dynamic rendering to browse and display your inventory. Virtually all other inventory posting systems (the old Auto-Corner system included) use a dynamic system that looks up the necessary data to create a webpage on-the-fly and deliver it to the web surfer.

While this might sound like a good idea, it is inherently flawed in a number of aspects. First of all, many search engine spiders will not list any content that is dynamically rendered. What this means is that if your site has a static home page but all of the other pages in your site are dynamically generated, these spiders will only list and rank that one page. This does have an effect on how they determine the ranking of your site when compared to sites with lots of pages in it.

This new system I have just completed is a total rewrite of the Auto-Corner system from the ground up. How it works is that the "engine" that runs the site will dynamically build static pages for your site and those pages will be viewed by your customers instead of ones rendered on-the-fly. This is good for your site because now the search engine spiders will have lots and lots of pages to spider instead of just the home page. Let us say that your dealership has 100 vehicles on the lot in 5 categories. The total number of static pages in your site will be in the ball park of 120 which will significantly improve your search ranking over sites that have just one page listed. This is because one of the methods that a search spider uses to determine ranking is the number of pages that you have in your site that are relevant to a particular search term(s).

Another benefit of having a site that is composed of static vs. dynamic content is that the webserver will deliver these pages in a fraction the time as there is a lot less work for it to do. This will make your customers happy that they didn't have to wait for the server to crunch through a database file and make all kinds of calculations before it delivers the results to their browser.

The principal reason that the other inventory systems don't use this method is that it takes a lot more programming to pull it off. I have developed several systems that use this technology and have the methods down to a science and are now part of the new Auto-Corner system. One example is a system that I wrote last year that used this methodology and was getting 1/4 million hits a day. A check of my logs revealed to me that I was getting several thousand new visitors a day from the just a small handful of the search engines out there.

I then started performing lookups of certain key words that were relevant to my site and was surprised that I was listed on the first page or two for many of these words, even though there were literally hundreds of thousands of other sites listed as well. These sites were all listed after me and not in front of me and it suddenly dawned on me just why this was so. The simple reason was that my "statically generated" site of over 1700 pages in size had a huge advantage over those sites that depended on pages generated on-the-fly. This realization made me rethink my entire strategy when it comes to designing data driven websites. For this reason, I have taken the more difficult avenue of creating an automated system that manages static pages instead of the usual dynamic systems that I have employed in the past.

Inktomi

Inktomi is often regarded as the mother of all search engines. The reason for this is that a number of very large search portals out there (AOL, MSN, and HotBot to name a few) use the Inktomi search engine within their system. If I were to want to submit to only two search engines, Inktomi would be my first choice followed by Google as a close second (see below). Inktomi has both paid premium and free submission systems. To learn more about submitting your site to Inktomi's premium system click here. Submitting your site for free requires that you go thru one of their portals to do so. You can submit your site to Inktomi through HotBot by clicking here. The following information is taken directly from Inktomi's site:

The Inktomi Difference

Inktomi Content Discovery Technology discovers, critiques and harvests content to ensure that the search user experiences the freshest and highest quality search results available on the Web. The system examines more than 1.7 Billion URLs to analyze the characteristics of each page and the link structure that connects the pages to create the Inktomi WebMap. From this WebMap database, Inktomi chooses the highest quality documents for each of these search indexes:

• Eurocluster (110M Documents)
• The Asia Pacific cluster (55M Documents)
• The Best of Web cluster (110M Documents)
• GEN3 (500 M documents).

Google

Next to Inktomi, Google is perhaps the most complete and most widely used of all the search engines out there. Getting a good listing in Google is very important even if you have never heard of them. The simple reason is that many of the top name search portals out there (Yahoo included) use Google's search engine within their system. What this means is even if you can't get a decent listing with Yahoo in their directories, you can still show up in their webpage search results through Google. To submit to Google for free and to learn more about their premium placement system click here. The following is taken directly from Google's site:

Share your place on the net with us. We add and update new sites to our index each time we crawl the web, and we invite you to submit your URL here. We do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about when or if they will appear.

Please enter your full URL, including the http:// prefix. For example: http://www.google.com/. You may also add comments or keywords that describe the content of your page. These are used only for our information and do not affect how your page is indexed or used by Google.

Please note: Only the top-level page from a host is necessary; you do not need to submit each individual page. Our crawler, Googlebot, will be able to find the rest. Google updates its index on a regular basis, so updated or outdated link submissions are not necessary. Dead links will 'fade out' of our index on our next crawl when we update our entire index.

Rules on Spam
Google will remove your site from consideration in their search listings if it is determined that you are spamming their system in an effort to gain a higher position in their search results. The following rules should be followed if you want to be listed by them:

Open Directory Project

Perhaps next in importance to the above mentioned two search engines is the "Open Source Directory Project". This is a system maintained by a group of volunteers all around the globe who check out submitted websites in person and make a judgement call on where each sites should be listed in their system, if at all. What this means is that searching their system will result in a much shorter list of results but these results will be much higher in relevancy to your specific search. The main goal of this project is to maintain quality over quantity. Now, how does submitting to this system affect your search standing over all? Well, most of the biggie portals and search engines (Google, Netscape, Lycos, and AOL Search to name a few) get much of their high relevancy content from the Open Source Directory Project. What this means for you is that getting listed here will just about guarantee a good placement in all the other top dog search portals out there. To submit your site to them, you will have to visit their site here, drill down to the category that you feel is most relevant to your site's content and click on the "Add URL" link found on that page. The following is taken directly from their site:

Procedure After Your Site is Submitted

An ODP editor will review your submission to determine whether to include it in the directory. Depending on factors such as the volume of submissions to the particular category, it may take several weeks or more before your submission is reviewed. Please only submit a URL to the Open Directory once. Again, multiple submissions of the same or related sites may result in the exclusion and/or deletion of those and all affiliated sites. Disguising your submission and submitting the same URL more than once is not permitted. If a site you submitted has not been listed after three weeks, you may submit it again or you may send an e-mail to an editor of the category for which the site was submitted.

Updating Your Site

If a site has been accepted for inclusion in the directory but you are dissatisfied with how the site is described or titled, you may go to the category where it is listed, and fill out the "update URL" form. If you are dissatisfied with the category in which your site is listed, you may send an e-mail to an editor for the category explaining your disagreement. Be polite and civil -- threatening or abusive behavior will not be tolerated. If (but only if) you are dissatisfied with the editor's response, you may then use the Feedback link at the top of the page to appeal the editor's decision. Be specific concerning your disagreement (including why you believe the editor's response is inadequate). Comments made through the feedback link are reviewed by the ODP staff, who will make the final decision. Please do NOT send correspondence to Netscape, as that will only slow down and complicate the process. We take all feedback seriously and give it our thoughtful consideration. But please remember that we must exercise our discretion and make numerous judgment calls as to how to make the ODP as useful as possible -- no matter what decision we make, we may not always satisfy everyone.

Getting Your Site Into Portals and Search Engines Using ODP Data

If your site has been accepted into the Open Directory, it may take anywhere from 2 weeks to several months for your site to be listed on partner sites which use the Open Directory data, such as AOL Search, DirectHit, HotBot, Google, Lycos, Netscape Search, etc. We make updates of the data available weekly, but each partner has their own update schedule.

Yahoo

Yahoo now requires a fee to list a commercial site. This includes car dealerships. This fee is now $299.00 and is one time and non-refundable. This fee does not guarantee a listing with Yahoo but most sites that are reasonably well designed and have content in them that relate to the category that you submit to will be included. Click here for more information on Yahoo Express. The following is taken directly from Yahoo's site:

Looksmart and MSN

Looksmart is yet another search engine system that many search portals use. These include MSN, Excite, and AltaVista. They offer two levels of paid submission which will expedite consideration of your submission over a free listing. The free listing appears to exclude getting listed in MSN's directory system. Apparently MSN wants money before they will list you there. Also, paid listings will appear higher in the search results over free listings. The following is taken directly from Looksmart's site:

Drive Targeted Traffic to Your Web Site

MSN has partnered with LookSmart to offer Express Submit, the fast and easy way to submit your site to the MSN directory and the entire LookSmart Network.

Your site will be reviewed by LookSmart, which powers the search directories of leading portals on the Web including MSN, Excite, AltaVista, iWon and more than 370 ISPs. Together, LookSmart and MSN reach 3 out of 4 US Internet users -- that's 68 million potential new customers!

Express Submit -- Get Listed Fast!

• Guaranteed site review within two business days*
• Your site submitted to the MSN directory and the rest of the LookSmart Network
• Low payment of $299 US

• Site review in approximately eight weeks*
• Low payment of $149 US

AltaVista

AltaVista provides three ways to submit to their site. Two of them cost money and the third can take several weeks before their system will consider you for inclusion. Here is info on submitting to them that comes directly from their site considering these three options:

Express Inclusion
Generate qualified traffic quickly and simply. Express Inclusion lets you quickly add up to 500 pages to the global database that powers AltaVista's search results. Weekly updates keep your listings fresh. Express Inclusion allows you to rapidly insert and frequently update your pages in the global database that powers AltaVista's search results. AltaVista and its partners answer more than 50 million search queries a day, reaching more than 45 million users worldwide.

Express Inclusion lets you:

• Provide up to 500 individual URLs to AltaVista.
• Receive rapid URL entry into AltaVista's global database (All URLs must conform to AltaVista's submission policy).
• Keep your page content up to date with automatic weekly refreshes of your pages.
• Maintain updated URLs in AltaVista's global database for a period of 6 months.
• Track submissions and transactions online through your personal Account Management Center.
• Add your country-specific URLs to the local index that matches your site's domain. Click here for a list of AltaVista's country indices.
• New! Enhance your listings within the AltaVista results by adding logos, icons, custom taglines, and text links to any of your URLs in the Express Inclusion program. Click here for more information.

• Drives qualified traffic to your site.
• Fastest method of having your URLs included in AltaVista's search results.
• Ends the need to continually submit URLs to AltaVista.
• Because your URLs are refreshed weekly, you may frequently modify your site and have these improvements reflected in AltaVista results ranking.
• You can check the status of your URLs at any time through your personal Account Management Center.
• Allows you to target key countries with your products and services.
• Listing enhancements offer you unprecedented control over the look and feel of your results in the AltaVista index.

Six Month Subscription Pricing
AltaVista's Express Inclusion service is priced
for a 6-month subscription period.
 
Unit Price (USD)   
1st URL		$39.00   
URLs #2-10	$24.00 each   
URLs #11-100	$19.00 each   
URLs #101-500	$12.00 each   

Pricing Example:
3 URLs would cost $87 = (1 x $39) + (2 x $24)  

• Increases the number of partner URLs in AltaVista's index.
• Accepts pages that are traditionally difficult for crawlers to index, such as framed pages or pages with dynamic content.
• Customized page information gives partners control of their pages' appearance on AltaVista search result pages.
• Weekly refreshes ensure that AltaVista users are viewing the latest content.
• Topical Search Centers and Multimedia Searches provide additional, targeted traffic.
• Detailed online reporting helps partners optimize their feeds, helping them maximize traffic.
• Ends the need to submit URLs one at a time through AltaVista's "Basic Submit" program.
• Drives quality traffic to your site leading to higher registrations, impressions, etc.

• You may submit up to 5 URLs per session.
• New sites will be considered for inclusion in AltaVista search results in 4-6 weeks.
• This service does not guarantee that your site will be accepted.
• This service does not guarantee frequency of site refresh.

Lycos

Lycos does not appear to have any pay for submission plan and their system is entirely handled by computer run "web spiders" that will automatically visit your site and index it into their system in the manner that they are programmed to do. What this means is that humans aren't in the equation and your site might get a good listing or it might get a poor one, depending on how the spider "reads" the relevancy of your site and where it decides to place it in their system. The good news is that you can submit your site as often as you like and it is entirely free. Go here to submit a site to the Lycos system. The following is taken directly from the Lycos site:

Tips on Adding a Site

• You may submit more than one URL from your site as long as the URLs represent distinct webpages. Multiple pages that contain the same content will not be added.
• The Lycos spider will try to travel through links contained in the webpage you submit. A good rule of thumb is to count on the spider traveling down one level from the page you submit.
• Please do not submit webpages with these symbols in the URL: ampersand (&), percent sign (%), equals sign (=), dollar sign ($) or question mark (?). Our spider does not recognize them.
• Having trouble entering your URL? Make sure you've typed in the whole address, including appropriate prefixes (i.e., http://www.) and a trailing slash ("/") if the URL doesn't include a file name (e.g., http://www.lycos.com/).

• If you change your URL, please add your new site to Lycos using the form above. (Think of it as a change of address form.)
• Keep your site live. Our spider revisits all sites on a periodic basis. If our spider can't connect to your site over an extended period (about 4 weeks), it will be deleted from our rolling catalog. (If your site goes back online, it's easy to add it in again.)

Overture

When you go to most of the search sites above you will notice that there are a set of three results at the top of the page that are usually referred to as "Sponsored Listings", or the equivilant. Well, you can individually contact each of these sites and pay them to get included in this small group of premium listings or you can set up a single account with Overture and get listed on Yahoo, AOL, Lycos, Alta Vista, Hot Bot, and Netscape automatically.

Overture (once known as GoTo.com) is what is known as a "pay per click" search engine. How this works is that each site listed has bid on a keyword or search term. The higher the bid, the higher the resulting rank in their system. The bid amount is the amount that the site owner is willing to pay for each actual click that a visitor to Overture makes to go to their listed site. The top bidder for a particular search term or keyword will always be in the top position in the results for that word. The next highest bidder will be immediately below them and so forth.

You have to create and account with Overture which means that you tell them how much you are willing to pay for your search terms on a per click thru basis. In addition to this you must use your credit card to place money into your account and each click thru will automatically deduct the bid amount from your account balance. One advantage of this system is that your account will not be debited unless a visitor actually clicks on the link to your site. In effect you only pay if Overture provides actual traffic to your site.

Now, how does this get you listed with Yahoo and the others? Well, Overture has a contract with these outfits to automatically include your site as a sponsored listing in these sites if you are one of the top three bidders for your particular search terms in the Overture system. For popular sites, this can mean paying a dollar or more per click thru but for lesser categories and search terms, this can be as little as 5¢ per click thru.

The best way to see how your desired keywords rank is to go to Overture and enter those keywords in their search box and look at the results. In these results you will see a monitary value at the end of each search result. For example, if you entered the search term "car dealer" into the search box the result will show that www.pricequotes.com is paying 52¢ per click thru for this term.

Check it out for yourself and see how this works. You just might find that the terms you desire aren't all that expensive, or then again, you might be willing to pay more than the top lister and get your self instantly ranked right at the top of many of the major search sites out there. Keep in mind that this is an auction environment and just because your bid is the top one, it might not be for long as there are others out there who want that exact same spot and are willing to pay for it.

The following is taken directly from the Overture site:

Premium Listings - Above and Beyond

Take your Pay-For-Performance advertising to the next level with Overture Premium Listings. Premium Listings, the top three Overture results for any keyword, give you what no one else can - exposure to 75% of all Internet users by listing you on leading Web sites, such as Yahoo!, America Online, MSN, Lycos, AltaVista and many others!

Only Premium Listings Give You:

• Reach: Expand your reach to 75% of all Internet's users compared to the 40% reach of our Classic Listings (positions 4-240).
• Exposure: Put your listings on the highest quality, most-frequently visited sites on the Web.
• Maximum Traffic: Receive an average of twice as many quality clicks per month from Premium Listings compared to our Classic Listings.
• Top Position: Position yourself as the #1 search listing on great sites across the Web.
• Convenience: List your site once and manage just one account to be seen on the Web's most popular sites.