Working on code line by line, that is heavy duty. It amazes me how you guys can break down the task to do the line by line code writing and updating. This lowly user thanks you.

Ken

Ah!!!! Great news, Gene!! Thank you so much!

Great job Gene, you people that work behind the scenes at the Café are fantastic.

Thanks so much!!!!

You may not realize this, but I created the search engine from scratch to begin with, personally writing every line of code used by the search engine, so I know it well enough to make certain changes fairly quickly. However, the new search engine will pretty much be a complete rewrite, that's why it's going to take awhile. Certain parts of it will use existing code, but much of the actual search routines are going to be changed. As you can easily imagine, creating a fully functional search engine isn't easy, and the current version is the third one I created (not including the short-lived phpBB version I modified for use here).

The current search engine took me about five months to complete, and it has worked without any major problems for six-plus years. But, alas, as the Café grows, so does the amount of data that needs to be searched. This latest problem is just a natural part of that growth and needs to be addressed accordingly. In a perfect world I would have forseen this problem and made allowances for it all those years ago, but as they say (whoever "they" are), hindsight is 20/20. This same search engine is in use at four other forums and is operating wonderfully for those, but those forums do not have the mass of data as exists here. There are limits to the current version, and I now have to work around those limitations.

The solution, as I presently see it, is to limit the amount of data the Café has to search per request. (Limiting searches by specific forum was a simple way to do the same thing and could be done quickly.) With that in mind, I have decided to change the search engine so it only has to search three-month blocks at a time, either in ascending or descending order as requested by the searcher. (Descending will be the default.) That will mean a few extra clicks by the searcher, but I will make it as painless as possible. Three-month searches will (or should) be pretty fast, and if there are no results in the first block the searcher will be provided a link to the next block, and so on. As it is right now, the search engine has to search *all* existing posts, regardless of date, and with a 3.5-million post database, that's obviously too much for a single server to handle.

I previously mentioned this idea to someone and was asked if I could just make the search engine search for results and stop when it reaches the first 30 (or however many) matches. I explained that their suggested method would only be a variation on the same processes currently used. For example, if a search request resulted in only 30 total matches in the entire dataset, the search engine would still be required to search as much data as it does now so it would be of no benefit. The solution demands that the new search engine search limited data at a time, and the time-limit idea seems to be the best scenario based on the resources we have at present.

Sorry to be long-winded, but I'm sure there are people out there who are curious about what we're planning, and I didn't really want it to seem like some big secret. It isn't. Some people have asked me via PM about it and I've explained it to them, though not quite in this kind of detail.

And so, for those who are or were curious, there you are.

p.s. ... Thanks to all for the kind words. It's very appreciated!

Gene, You're efforts are really appreciated! Many do not realize what it takes to provide this service.

Mega Kudos to you and Steve! As Gomer Pile would say - THANK YOU, THANK YOU, THANK YOU!

Gene, you rock!

Thanks a lot!

This being back online is one of the best presents a poor man could possibly ask for the holidays.
What else is there?
Thanks
again!!!

While I am chomping at the bit for the full search feature to return, I have to say thank you for thinking of us and giving us this limited version to work with until the full one comes back.

Gene, Thanks for all your hard work & dedication in making the Café such a success!

Thanks for all the hard work Gene. As a fellow web developer I admire your resolve!

While we wait for Gene to do his magic on the new search tool you can always use Google. It constantly amazes me how bad most people are at using search engines. If you just want to search The Café using Google, type the following before or after your search term(s):

site:themagiccafe.com

You can also use the 'Advanced Search' page to do the same thing:

http://www.google.co.uk/advanced_search

This page also allows you to filter by date, and search specific areas of a web page.

HTH,

Marty

P.S. This trick works on other search engines too - yes, Google isn't the only one . I often use Ask Jeeves.

An easier way to search a site through google is like this:

themagiccafe.com: Slydini

type in the url of the site then the colon then the space and then the subject matter. This usually works better than the search engines on the site itself.

Cheers,
Brian

Thanks Brian, I didn't know about that technique. Less typing is always better!

Marty

Ummm, Where is it? Can't find a search anywhere

Upper right corner of every page, third line down it says search

Does not work. I choose all forums for a search and I get an error saying choose a forum.

Your answer is in Gene's post above.

JD

Rob, you must choose a forum. That is one of things that was changed so we could bring the engine back. Pitboss is correct. Read Gene's post earlier in this topic for an explanation.

I must be doing something wrong using Google Advanced Search.

All of the results I get date around 2006

I appreciate anyone working hard on the issue of the search, but don't understand why it's a problem.

Couldn't we either:

a) Incorporate a search engine search into the Café (google/bing) or
b) Move the whole Café over to use some more modern forum software?

There's lots of free forum software out there. From basic to very advanced, and then full search would just be part of the built in system.

Obviously any transition would have to be done carefully, but we've had a pretty much unusable search function in here for what seems like a long time now and if the problem is difficult to solve and requires lots of time that people don't have spare, then why not just get rid of the problem a different way?

If we are a few days away from a solution and full search again, then obviously ignore everything I've just said. If we are more than a few days away from a solution, then I'm sure that going down one of the two routes above would be a lot simpler and faster.

I don't know how most others use the Café, but it used to be the first place I'd come for anything magic related. Reviews/info etc. Since the search problems, I've stopped coming half as much. I'll general web search, which is not as good, but at least gets me what I want a bit slower. Every now and again I forget about the search problem and try to find something again (I did this just now) and it's just too frustrating to use.

I'm sure it'll be sorted at some point. It's really affected my use of the Café and regularity of visits.

Damian