The Magic Café
Username:
Password:
[ Lost Password ]
  [ Forgot Username ]
The Magic Cafe Forum Index » » Announcements (Please Read) » » Search Engine Announcement » » TOPIC IS LOCKED (0 Likes) Printer Friendly Version

 Go to page 1~2 [Next]
Gene
View Profile
Café Forum Developer
Howard City, MI
444 Posts

Profile of Gene
[ANNOUNCEMENT]

As of yesterday at 5PM (Eastern US), the Café's search engine was shut down due to being the cause of the recent slowdowns. The specific cause is keyword searches. All other functions of the search engine do not appear to cause any problems. Links to the search engine have been removed, except the "view posts in the last 30 days" link on the member profile pages. I will be removing that soon so there is no confusion.

I will be looking at everything as soon as I can. There will be a lot of thought put into this, so it will not be a quick process. My goal is to come up with a solution that does not change any of the search engine's features, yet still provides accurate results within an acceptable period of time.

[SEARCH ENGINE HISTORY]

For the benefit of those who have not been with us since the beginning, the search engine has been a focal point of many discussions and has undergone several revisions. The original version was very slow and extremely limited, but it did offer one thing that nearly all present-day forums don't: Exact phrase searches. This was possible because it used a very slow database function. It didn't take long for that to start crashing the Café, so we had to come up with a solution. I don't think anybody ever really realized it, but version 2 was an almost complete copy of the search engine used with phpBB2. That one, too, started to crash the forums. I thought it would last a long time, but in reality, it didn't last that long at all. There were still complaints about it, too, most of which were because it didn't offer exact phrase searches. I made some changes to the original phpBB design to include certain limits so that it wouldn't crash the Café. It worked - but there was a deluge of complaints. People were very vocal about it. Everybody seemed to want the whole smash: Two-character words, being able to search all posts without regard to when posted and, of course, exact phrase searches. It took me a while, but I came up with a version I really thought would do it. It answered (nearly) everyone's needs and wants, but at a delayed price. That one was shut down yesterday. Soon I will begin considering options and thinking about the next version: V4. Who knows where that one will end up?

[GENERAL THOUGHTS]

Trust me, this is not going to be an easy task. I fully understand how important the search engine is for the Café - indeed, for any site - and I will do my best, but please allow me the time I need to work it out. Technically speaking, there is nothing wrong with the search engine we have been using, it's just not efficient enough to handle the massive amount of data in use here. In fact, I am using the same search engine structure for two other sites with perfect success. It's simply a matter of data. For those who have any interest in the figures, the total size of the Café database is over 1.4GB. (That's 1.4 gigabytes, folks. Smile ) The search engine itself currently contains 282,387 keywords, with a total keyword/post-ID data size of nearly 235MB. These are gargantuan numbers to deal with on a single server. Is there an answer? I don't know that yet. I'm only one guy. Other forum packages have several developers, or a complete team. However, if there *is* a solution, I will find it. Hey, I figured out how to offer exact phrase searches in a modern forum, didn't I? Smile

And now, let the process begin ... hang in there, everybody!
Somebody kick me off the computer, will ya? Smile Smile
pegasus
View Profile
Inner circle
United Kingdom
8252 Posts

Profile of pegasus
Apparently you can use google to search specific websites, including the Café, but I cannot remember how exactly. If anyone knows, then could they please post it here for others?
pegasus
View Profile
Inner circle
United Kingdom
8252 Posts

Profile of pegasus
As a quick fix then you can use Google as an internal site search engine.

Example= torn and restored site:www.themagiccafe.com

Type in what you are searching for, then leave a space, then type the following. site:wwwthemagiccafe.com
GarySumpter
View Profile
Special user
UK
857 Posts

Profile of GarySumpter
The only problem is that the google results wont be up to date, but better than nothing I suppose.

Can you not just move th ecafe to a more poweful server?

Gary
Edith
View Profile
Regular user
Germany
130 Posts

Profile of Edith
I am looking forward to see the result... of all your good and hard work, Gene!

Edith
pegasus
View Profile
Inner circle
United Kingdom
8252 Posts

Profile of pegasus
The following link allows google to operate as an internal search engine. Perhaps it can be used until a proper fix has been found.

http://www.javascriptkit.com/script/scri......ch.shtml
Marco S.
View Profile
Inner circle
1017 Posts

Profile of Marco S.
Hope you will find a solution quickly.
evolve629
View Profile
Inner circle
A stack of
3838 Posts

Profile of evolve629
Perhaps you can install an auto delete function on the members search so that key words are not memorized by the search engine to prevent overloading the system. Just a thought!
One hundred percent of the shots you don't take don't go in - Wayne Gretzky
My favorite part is putting the gaffs in the spectators hands...it gives you that warm fuzzy feeling inside! - Bob Kohler
Tom Jorgenson
View Profile
Inner circle
LOOSE ANGLES, CALIFORNIA
4452 Posts

Profile of Tom Jorgenson
We all appreciate your past efforts and grand successes....we'll survive this one....good luck. Does Starbucks deliver???
We dance an invisible dance to music they cannot hear.
mshanker
View Profile
New user
Frankfort, KY
88 Posts

Profile of mshanker
Gene, several sites--including U.S. government ones--embed the Google search engine into their sites. Couldn't that be a solution to the problem?

I for one think it's a shame to disable the search feature.

Manoj
Manoj
pegasus
View Profile
Inner circle
United Kingdom
8252 Posts

Profile of pegasus
Quote:
On 2006-02-22 08:37, pegasus wrote:
The following link allows google to operate as an internal search engine. Perhaps it can be used until a proper fix has been found.

http://www.javascriptkit.com/script/scri......ch.shtml


Already mentioned manoj
Gene
View Profile
Café Forum Developer
Howard City, MI
444 Posts

Profile of Gene
Yes, I'll be doing something about that when I can get to it.
Somebody kick me off the computer, will ya? Smile Smile
Bill Hallahan
View Profile
Inner circle
New Hampshire
3219 Posts

Profile of Bill Hallahan
Someone mentioned searching using http://www.google.com/ in another topic. Just add "site:themagiccafe.com" to your search string without the double quotes.

You can even search titles in a specific forum using Google. For example, to search the Pick a card...any card forum titles for an exact match to the string "three card monte" you would enter the following into the Google search text box.
Quote:
"three card monte" "Pick a card...any card » »" site:themagiccafe.com

The double quotes cause a case insensitive exact string match. You could leave them off on the first quoted string and you'd find topics that contain "three", "card", and "monte", but not necessarily together or even in that order.

Of course, someone could put "Pick a card...any card » »" in a post too and it would show up in the title search I describe, but I doubt anyone has posted that, and I hope they won't.

This is not quite as good as the Magic Café's title search was since each forum's titles have to be searched separately, and also the search data is not updated in real time. But it does work pretty well.
Humans make life so interesting. Do you know that in a universe so full of wonders, they have managed to create boredom. Quite astonishing.
- The character of ‘Death’ in the movie "Hogswatch"
Foucault
View Profile
Elite user
New Jersey, USA
424 Posts

Profile of Foucault
Quote:
On 2006-02-22 18:32, Bill Hallahan wrote:
but I doubt anyone has posted that, and I hope they won't.


erm..you just did! Smile
MikeRaffone
View Profile
Special user
620 Posts

Profile of MikeRaffone
I can't find the link "view posts in the last 30 days". Has this feature been removed too?
I am trying find a post from a few days ago and can't find it.
Mike Perrello
DC Area's Leading Family Entertainment Expert
Gene
View Profile
Café Forum Developer
Howard City, MI
444 Posts

Profile of Gene
Yes. Read my first post.
Somebody kick me off the computer, will ya? Smile Smile
stannmaple
View Profile
Elite user
487 Posts

Profile of stannmaple
Gene, Do you really live in Rockford, MI? That's crazy! You're like my next door neighbor. I live in Grandville. Why did I never notice this before? Crazy!
Dann
Gene
View Profile
Café Forum Developer
Howard City, MI
444 Posts

Profile of Gene
Actually, you just reminded me that I have to change that. I live in Howard City now. If you ever want to stop by and learn how it all started just let me know. Smile
Somebody kick me off the computer, will ya? Smile Smile
willmorton
View Profile
New user
79 Posts

Profile of willmorton
Gagging for the search engine to return! Google is ok, but the Café-specific one is vital!
ClouDsss
View Profile
Inner circle
1799 Posts

Profile of ClouDsss
I really miss the search function Smile

all the best and hopefully a solution is found Smile
Think outside the box, cos people are all thinking inside now!! - ClouDsss
The Magic Cafe Forum Index » » Announcements (Please Read) » » Search Engine Announcement » » TOPIC IS LOCKED (0 Likes)
 Go to page 1~2 [Next]
[ Top of Page ]
All content & postings Copyright © 2001-2020 Steve Brooks. All Rights Reserved.
This page was created in 0.12 seconds requiring 5 database queries.
The views and comments expressed on The Magic Café
are not necessarily those of The Magic Café, Steve Brooks, or Steve Brooks Magic.
> Privacy Statement <

ROTFL Billions and billions served! ROTFL