Searching a big mysql database with relevance

MySQL is good for storing data but not great when it comes down to fast text based search.

Apart from Sphinx which has been already suggested I recommend two fantastic search engines:

  1. Solr with http://pecl.php.net/package/solr - very popular search engine. Used on massive services like NetFlix.

  2. Elastic Search - relatively new software but with very active community and lots of respect

Both solution are based on the same library Apache Lucene


If the "ORDER BY" is really the bottleneck, the straight-forward solution would be to remove the "ORDER BY" logic from your query, and re-implement the sorting directly in your application's code using C# sorting. Unfortunately, this means you'd also have to move your pagination into your application, since you'd need to obtain the complete result set before you can sort & paginate it. I'm just mentioning this because no-one else so far appears to have thought of it.

Frankly (like others have pointed out), the query you showed at the top should not need full-text indexing. A single suffix wildcard (e.g., LIKE 'ABC%') should be very effective as long as a BTREE (and not a HASH) index is available on the column in question.

And, personally, I have no aversion to even double-wildcard (e.g., LIKE '%ABC%"), which of course can never make use of indexes, as long as a full table scan is cheap. Probably 250,000 rows is the point where I'll start to seriously consider full-text indexing. 100,000 is definitely no problem.

I always make sure my SELECT's are dirty-reads, though (no transactionality applied to the select).

It's dirty once it gets to the user's eyeballs in any case!


You should try sphinx search solution which is full-text search engine will give you very good performance along with lots of options to set relevancy.

Click here for more details.


Seems like the index doesn't cover Premium, yet that is the first ORDER BY argument.

Use EXPLAIN your query here to figure out the query plan and change your index to remove any table scans as explained in http://dev.mysql.com/doc/refman/5.0/en/using-explain.html