March 26, 2001-- Newsletter #124
Full Text Search: The Key to Better Natural Language Queries for NoSQL in Node.js
Goodies to Go (tm)
March 26, 2001--Newsletter #124
This newsletter is part of the internet.com network.
Please visit http://www.htmlgoodies.com
Greetings, Weekend Silicon Warriors,
I have decided to coin a term, "Space Junk". Yes, I know it's not a new term, but I intend to use it in a new way. Space junk will now mean all the pages contained within a search engine that are either dead, not what they claim to be, or basically worthless to anyone who clicks to visit.
There. It's settled. Space Junk.
The hacker gauntlet has been thrown down once more. The new Charlie Pride album "A Tribute to Jim Reeves" will be out round about the middle of April. It is expected to be the first CD encoded so that the material cannot be copied onto a computer. Should a person try to copy a song, the CD will open the computer's browser and direct the copier to the SunnComm Web site where you can download MP3 versions of the songs. That's pretty clever, huh? We'll see how it goes...
We're just over two weeks away from the tax deadline here in the good old U. S. of A. Luckily I didn't go bonkers buying up tech stock options. Those who did and then watched the value drop like a stone through water might be in for a surprise when their accountant mentions the Alternative Minimum Tax. Here's the way the tax works. If you bought shares in 2000 but didn't sell them before the end of the year, you must pay tax on the purchase price even if the stock has fallen below the purchase price. I'm sure there was some kind of logic employed when that little ditty was written but I can't seem to come up with it right off the bat.
Now onto today's topic...
Remember when you were in school and the teacher made a mistake or something went horrible wrong? Wasn't it wonderful? There was this sense of accomplishment even though you did little more than watch a science experiment fail or view a gym coach miss an easy lay-up.
Well, it happened to me during the last hours of my Executive MBA class this past Saturday.
I had been lecturing on META commands and how to get pages up and into the search engines. The students were feverishly taking notes when I went to the big computer to show them how to get their pages into the search engine databases for free. I assured them that it would take a little time if they wanted to go the free route, but the search engines would still accept their pages and would no doubt give them a listing for free.
I went first to Webcrawler. I slid down to the familiar "Add you URL" link towards the bottom. I clicked expecting to get the text box.
Errrrt! (That's my basketball buzzer sound effect)
It's now pay-to-play. If you want a link in two days, it's $199. The "Basic Submit" is only $99.
"Woah," I thought, "I'd better try another, and quick."
AltaVista wants $199 too. However, you can still submit for free...five pages. They're now using some sort of code that tracks your submitting. LookSmart wants $199 for two days or $99 for a listing in about eight weeks. Excite wants the same.
"Yahoo must still be free", I think. I go. Nope. I
attempted to submit a site and ran into the now familiar
$199 deal. When I attempted to go beyond and submit
for free, I was greeted by an FAQ that starts with the
question, "Why can't I submit my site for free anymore?"
My lecture was falling apart. I finally found that Lycos still allowed a free submission so I submitted and quickly moved along to a new topic.
Wow. Right there in front of my class, I learned the new direction of the search engine world. It's going to be Pay to Play. Now, before you write, I am quite sure there are hundreds of sites out there that still allow free submissions and I also know the tricks of submitting to certain search engines knowing that it "feeds" other search engines. LookSmart is just such a search engine.
All that aside, what's happening? Why have the search engines brought the hammer down?
I would think it's a combination of reasons. For one, by allowing free submissions, they over-filled themselves with space junk. (There's my new term - see above.)
Every person who could code even the smallest amount of HTML submitted page after page, site after site to the search engines. It became way too much to search and then hope to produce reasonably decent results. Furthermore, once people understood that when a search engine would "spider" a site, it also took every page attached to the submitted page, people began creating homepages with a jillion links. Submit one page and thirty would go.
What's more, the world of the Web is not the nice and easy static HTML arena it used to be. Now sites are made dynamically. I know. StreetArtist.com only contains about ten pages total but can display over 1000. Except for a few pages on the site, every display is created through PHP and only exists as long as the user is viewing the page. Now there's a clash of search engine thought and dynamic page creation.
At some point in time, the search engine people had to throw their hands up in the air and proclaim, "enough". Tens of thousands of pages were rolling in. Multiple submit sites had programs running that would resubmit pages again and again every week. The wait time to get a site up and on the engine was pushing two months.
The thought of waiting two months was inconceivable to my wife and me so we paid the $199 three times to three different search engines. It was a tax deduction and we needed it done. I guess I took a little comfort in the fact that those who didn't have that kind of cash could still submit for free. I mean, the world could wait a little while longer for Billy's World of Pokemon Wonder and The Park Bench, Idaho WCW Fan Page, right?
I know many of you are right now seething over the thought that many of the major search engines are asking for money for listings.
Well...I'm not. I just don't like their fee structure.
I think the search engines are asking for such a large fee right now because they can get it. I wonder, however, if in the near future a more acceptable amount could be asked. Allow me to suggest this:
$75 per e-commerce page
$15 per non-profit, university, or personal site page If the page is not clicked on in three months...delete it.
I would make that last one retroactive.
I can hear those of you right now screaming, "what?" at your computer screen. But wait. Before you proclaim that I am completely wrong and that all should be free and something about the first amendment, stop and look at it from the perception of the search engine.
They are overwhelmed. They obviously cannot keep up with the vast number of pages being submitted. Many of the pages being submitted are being submitted "just because".
I believe that many of the problems could be alleviated if a little personal cost was attached to the submission. If a person wants to get his or her site into a search engine, I think they could find $15. It's a one-time fee. If a business truly cares to get their pages on the search engine, they should look at the submissions as a cost of doing business and take the cost off of taxes as if it were advertising.
Furthermore, attaching a cost of every single page will help to submit many more static pages making for successful searches. Attaching a cost might help to submit more pages that are kept up and cared for.
Attaching a cost might mean greater numbers of employees at the search engines so that every page submitted could be posted within 48 hours.
Think about it. Is it worth $15 to you to get your page posted in a reasonable amount of time? It is to me.
If a person or a business feels it isn't worth a payment and a two-month (or more) wait is OK, then why exactly was the page submited to the search engine?
Because I could be?
Is that enough of a reason?
That's that. Thanks for reading.
Joe Burns, Ph.D.
And Remember: Did you know the great English playwright William Shakespeare Actually mentioned America in one of his plays? It was "The Comedy of Errors." Act III, Scene II.