So, You Want A Searchable Database, Huh?

Use these to jump around or read it all...

[Search Someone Else's Database]
[But I Want To Search MY Site!]
[Goodies Search -- Java-Driven Search Engine]
[Use AltaVista or HotBot]

I am asked this question time and time again, "How do I set up a searchable database?" There are actually a few different ways -- some are harder than others. Here's a quick look at the three main ways.

Search Someone Else's Database

Ever been into a page and the author invites you to search Yahoo or Webcrawler right from his or her own page? You think this person must be pretty high up the ladder to be able to pull off this kind of deal. Not really -- anyone can do it. Here's an example:

Search Excite Here:

Search Webcrawler Here:

Search Yahoo Here:

Go ahead, enter a word -- nothing dirty, mind you! The results of your search will be returned straightaway.

...waiting ...waiting ...waiting

Ah, good. You're back. Neat, huh? Actually, I lied above (and I may lie down below). My site didn't perform any of the searches. They were done by Yahoo, Webcrawler, or Excite, depending on which one you chose. I just initiated the search. When you got your results, did you notice that you were no longer in Kansas Goodies any more? You were at the site of the search engine you chose.

How I Did It

Let's look at the code I used to create the Yahoo search above:

Notice it's a simple set of form commands, set up much like you would to create a link button or a simple mailto: guestbook. However, in this case you are using the form to send the information contained within the text box to a search engine. That's what you call the actual program that searches the Yahoo database, a "search engine." In the example up above, you are sending the info to Yahoo's CGI bin to be worked on by something called "search."

Here are the Webcrawler and Excite lines from above:

<FORM NAME="wcsearchform" ACTION="http://webcrawler.com/cgi-bin/WebQuery" METHOD="GET">

<FORM NAME="search" ACTION="http://www.excite.com/search.gw" METHOD="get">

Notice they also sent the output of the text box to a search engine. One goes to something called "WebQuery" and the other goes to "search.gw." Once the data is sent to the search engine, then the site's database is searched and you get your results.
But note again that you do not use my site to search -- you only send the information from my site. Once that's done, I'm totally out of the picture and you're at the search engine's site.

Once more thing... Notice the "METHOD" in the two above, and the name="p" in the Yahoo search above? Those are little items that each search engine uses to denote how to manipulate the data it receives and what to name the output sent through the text box. Each search engine will work differently and you must make sure you use the search engine's format, exactly, on your page.

Where Do I Get The Format

Right from the search engine itself. It's not that tough. Go to the search engine site and look at the source code. It's written right there. Just grab the part that starts the form through the /form. Now, you may need to knock out some stuff in the middle, like extra text or table commands, but it's all right there. Then put it on your page.
One thing though, you may notice when you get the code that the ACTION section does not include the entire address to the search engine. That's because it didn't need it when it was on their site. Now it's on your site and it needs it. Add the full address before the search stuff. Without it, the data looks for something on your site that will do the search. No dice... errors all over the place.

How Many Search Engines Are There

Tons. Now wait... yes. Tons. Here are few of the biggies:

[Alta Vista] [Apollo] [Bizwiz] [ComFind] [Excite] [The Web Magazine] [Pronett] [HotBot] [Infoseek] [Lycos] [Nerd World Media] [LinkStar] [Mallpark] [WebCrawler] [ImageSurfer] [AAA Matilda] [What's New Too!] [DejaNews] [WebDirect!]

Here are some pages with multiple search engines....

[More] [Dr. Webster's Big Page of Search Engines] [Beaucoup Search Engines] [Search.com]

But I Want To Search MY Site!

Nice emphasis on the word "my" there, pal. This is a tough call. I have gone through the process and I can tell you that it is rough. The process happens in two steps:

You need to create a database and place it on your site (or run it from another site).
This database has to be in a format the server understands. Excel works a lot, as does Lotus.
You need to write a CGI Perl script to do the searching.
Maybe you know someone who can write one of these for a small amount of money. I didn't -- $250 did it for me.

Try Number 1
My Goodies Search was up for about a month. Problems galore. I took it down.

Try Number 2
Next I tried the Excite search engine. This is a search engine put together by the people at Excite. It is a self-contained package that your Webmaster may already have. Mine did. $275.
The search engine was wonderfully easy. It installed quickly and made an entire directory searchable. All I did was point it in the right direction. It compiled the data and was up and running in a matter of minutes. After a month, I had to take it down. So many people were using it that the search engine was taking up 3 out of every 5 cycles of the computer's brain. In layman's terms, it was about to crash the whole system by overworking it. I still have it and I still use it to search, but I don't make it available to others. It slowed the site tremendously.

Try Number 3 This is the current try, and I think this may work. I am running a Java-driven searchable database. Want to see it? I knew you would:

Goodies Search

The database is one created by Satadip Dutta. The reason I like this database so darn much is because my site is not involved in the search. You are doing all the work.

Huh?

Don't look so dazed. It's all you, man! The search engine page you just went to and probably ran is really just a huge JavaScript. When you enter a word, the JavaScript isn't searching my site -- it's searching itself.

Huh, again?

Yeeepper! When you logged into the page, you actually downloaded the entire database with the page. Clicking the button made the JavaScript search itself. What it returned was simply what it found contained within itself. So your computer did all the work and the Goodies server was free to go on about its business.

I Want One!

I thought you might. So I put together a quick tutorial on using the JavaScript to make one for yourself. Follow this link:

So, You Want Your Own Searchable Database, Huh?

Please take the time to read the requirements, Satadip Dutta has put a lot of time into this work and wants you to use it. Just make sure you give him credit where he asks for it. It's the least you can do.

Use AltaVista or HotBot

I also have a tutorial that teaches you to use the search engines to search just your site. It's very clever. Read it here.

Well, enjoy. There's not a whole lot more I can tell you. You know how to search other databases from your site and a couple of suggestions about how to set up Excite and other CGI driven searches. You may want to contact your site administrator to look into those paths. Plus you have a link to get started on making your own Java-driven searchable database. Any of them will work. You just need to pick the one that you like best.

Enjoy!

[Search Someone Else's Database]
[I Want To Search My Own Site]
[Goodies Search - Java Driven Search Engine]
[Use AltaVista or HotBot]