Creating Web Pages     (Nov 99)

By Bob Pajot

Back in May’s TXT, I discussed how to register your webpages with a search engine. In this issue we will discuss how to actually use a search engine to find information that you are looking for.

There are many different Search Engines, Web Directories and Meta Search Engines on the Internet. What they do is search their own database of web pages. These databases are compiled in various ways. Most Search Engines have robots that search the Web for new listings and add them to their database. They usually also accept listings of URL’s (universal resource location) from anyone wishing to submit one. (normally for free).

The main reason they offer this as a free service is that while you are visiting their page you will see banner advertising which, they of course get paid for. It is to their advantage to have a large database so they will attract a large number of visitors to their page.

Note: In this on-line edition of the TXT all the following are links to the various search engines.

Popular Search Engines include the following.
AltaVista - THE KING of search engines, indexes 200+ million web pages!
HotBot/ Inktomi - 110MM pages, parallel, scalable system. Search a domain, eg. [.edu].
Scrub The Web - Robot SE claims to have indexed 80 MM pages.
Excite - Indexes 55 million pages, rated as one of the best SEs.
Lycos - "Wolf spider" (Latin). Another one of the SE biggies. Over 55 million URLs.
InfoSeek -30 million URLs. Highly regarded.
Webcrawler - Independent owned by Excite.

Popular Web Directories.
411 Now -The business directory
AllSearchEngines.Com - Handy site lists 'em all! (Here's where I compiled this list)
Bigfoot - Advanced people search.
Four11 - Find people.
SNAP - One of my favorites! A new biggie.
Who Where -Great for searching people's email address.
Yahoo! -The BIGGEST of the big. HUGE directory organized by category.

If you want to search the entire Web then you really want to use a Meta Search Engine.
A Meta Search Engine works by going out to search several different Search Engines and then reports the results. Please note that they may not return all the same results that you would receive if you did the searches yourself on each Engine. The reason for this is that only a specific time is spent at each Search Engine before it gives up the search.

Meta Search Engines.
Dogpile - Searches 13 SEs.
MetaCrawler - Excellent, 8 SEs. Complex searches, can search a continent.
MetaFind - Searches 6 SEs, merges results.
Northern Light - 100MM pages, plus pay "special collection."
Web Info Search - Search websites, search engines, job, MP3 audio, person search, business, reverse phone, reverse address.