Please upgrade here. These earlier versions are no longer being updated and have security issues.
HackerOne users: Testing against this community violates our program's Terms of Service and will result in your bounty being denied.

Improved Search (Sphinx)

mcu_hqmcu_hq yippie ki-yay ✭✭✭
edited August 2012 in Vanilla 2.0 - 2.8

First off, thanks to core devs and everyone in the community, I sure learned a lot from everyone!

Live Demo

pics

Login with User: test Pass: test

Try to add a comment or discussion with tags and then see if you can find it by searching for it. Indexes are rotated every 5 minutes. Site will probably be slow since running off of old laptop with low bandwidth.

This big plugin enables your forum to run its own dedicated search engine. Once installed, you should hopefully see a massive increase in search relevancy and enjoyment in finding things.

There is a ton of documentation stuff that goes along with this, so I encourage anyone who seems interested in getting this to work to read EVERYTHING in the readme on the github repo. It will also go over the 9 widgets.

This does come with an install wizard, so you don't need to do anything other than click your way through it. Just be forewarned that some hosts will not allow you to run a dedicated engine on your server! The wizard will configure everything for you, including your config file and cron files. Most of the settings that sphinx supports can be changed via the admin settings page. The control panel allows you to manually index your database as well as start/stop sphinx and look at other things such as document count.

My main goal was to make your visitors STAY on your site and to have them find things easily. This plugin will show related threads on every discussion topic, whether it be in the side panel or below the discussion. Also, when asking a new question, your members will be presented with already existing titles much like stackoverflow does it:

image

All of the links will have a tool tip associated with it that will show the first 200 words of the original title to save everyone time from clicking on it only to find that it is not relevant. The main search also highlights hit words.

One of the neat things about sphinx is the extended query syntax. You can now do things like:

@title Vanilla Forums @user 422 @body "Photoshop | Australia"

I don't doubt that people will have a tuff time installing this, so feel free to post your frustration here. I got mine to install on this old piece of crap. Took me about 1.5 hrs to compile and install. If the demo site is slow, just imagine this dusty thing with its 100MB of RAM.

Another confusing topic will be all of the settings that sphinx provides. Your default config will cover what most people probably want, but some of the cool features allow you to do the following:

  1. Morphology - match dogs with dog and running with run
  2. Min Word Length - only index words that are xx characters long
  3. Stopwords - create a list of common words that you DON'T want to be indexed (filler stuff)

I encourage people to add other widgets to this. There are some things that I had in mind to complete, but lost track of time. One of them is a "Did you mean" feature. Locale is only English ATM and has 20 or so definitions with it.

Let me know how it goes! @2Olivaw @Apreche

Comments

  • Thank you so much for making this plugin, I'm sure lots of users will have great benefits from it!

    There was an error rendering this rich post.

  • and @mcu_hq goes for the gold medal

    I may not provide the completed solution you might desire, but I do try to provide honest suggestions to help you solve your issue.

  • maybe this can get installed on the vanilla forum asap.

    is there a limit on results? if so it would be nice to get more results than the current vanilla search which will produce at maximum two pages of results, which may only go back two weeks in some discussions.

    I haven't read the docs yet :)

    I may not provide the completed solution you might desire, but I do try to provide honest suggestions to help you solve your issue.

  • mcu_hqmcu_hq yippie ki-yay ✭✭✭

    is there a limit on results? if so it would be nice to get more results than the current vanilla search which will produce at maximum two pages of results, which may only go back two weeks in some discussions.

    I haven't read the docs yet :)

    You only get two pages of results on the demo site because there just are not a lot of discussions. You can set the maximum per page in the admin settings. I should have left this up to the user, but maybe for the next rev I'll put this in.

  • You can't imagine how much I love you for this add-on! Thank you so much!

  • You great! My db guy is involved around Lucene but without good results, so this is a great news. In this moment We have summer holidays, I can feedback you at the end of month.
    Bests

  • mattmatt ✭✭

    Great work. Search has been lacking or years!

    One things I would like to see is this made simpler. Not just the installation, but also the interface. There are too many options right now, which will scare people away even though it gives better results than standard Vanilla search. I think defaults should be implicit, with a button to get to advanced options. What do you think?

  • fh111fh111 Vanilla Padawan ✭✭
    edited August 2012

    thanks for the nice plugin. will probably use it even though i never had issues with the built in search.

    matt said:

    Great work. Search has been lacking or years!

    just curious now, i really wonder what kind of issues everybody has with the built in search? am i missing something? i just tested the built in search in our community again and it seems to work flawlessly

  • mcu_hqmcu_hq yippie ki-yay ✭✭✭
    edited August 2012

    matt said:
    One things I would like to see is this made simpler. Not just the installation, but also the interface. There are too many options right now, which will scare people away even though it gives better results than standard Vanilla search. I think defaults should be implicit, with a button to get to advanced options. What do you think?

    Yea, there are a lot of options, but for most people they should just stay as default. The problem is that some people are going to say, "Why can't I do xxx xxx xx?"

    The install wizard honestly can't get any easier. I could just make it 1 step and cram everything into it I guess.

    I think defaults should be implicit, with a button to get to advanced options

    Are you talking about the admin side of things or the main search front? There should be a link to the adv search landing page. The other button is collapsible div that covers what people are going to probably use the most often.

    The admin side could have something like this for the control panel.

    fh111 said:
    really wonder what kind of issues everybody has with the built in search? am i missing something?

    I know that some people have decent search if they enable "Like" in the config, but this plugin is more for huge communities then anything. You should notice that using "LIKE" in MYSQL is damn slow, which is why the Vanilla team is cautious to enable this on default. Most people won't notice this because their community is less than 500 members with around 4K discussions (if that). Sphinx is meant to scale just fine.

    When I worked on my main site years ago, I had to put in 200K "dummy" records to figure out which queries were slow, otherwise they go unnoticed.

  • Just wanted to mention here that everyone should try this out, it's a HUGE improvement over the default search which is next to useless on a large forum. With Sphinx everything is fast and you get far more relevant search results.

  • Wow this looks pretty exciting! What are the chances that installing this will break my forum? Is it easy to get rid of if something goes wrong? Does it automatically replace the default search or do I need to change something in the theme? Thanks!

  • Wow this looks pretty exciting! What are the chances that installing this will break my forum?

    Anybody can break their forum temporarily not following the steps closely.
    This plugin requires some knowledge and the right environment, but it is well worth it.

    it can be a bit of trial and error, to get it working, depending on your expertise.

    Is it easy to get rid of if something goes wrong?

    Absolutely, just disable it via dashboard or (remove the lines from the config).

    Does it automatically replace the default search or do I need to change something in the theme?

    No change to theme is necessary. It automatically add the search when sphinx is enabled and running, if you look at the images displayed you will see examples.

    mcu_hq put a lot of hard work into this plugin and it shows, but read the requirements and speak to your host. and read other comments relating to the plugin.

    I may not provide the completed solution you might desire, but I do try to provide honest suggestions to help you solve your issue.

  • mcu_hqmcu_hq yippie ki-yay ✭✭✭
    edited September 2012

    ditto what @peregrine said.

    If you host yourself, it will most likely work. You can download and install sphinx through your distro and tell the plugin where to look for that installation. Some other people installed it successfully through the wizard, but their host shuts down sphinx from running. I can almost guarantee that shared hosting will NOT let you run this. I would expect that dedicated hosts that DO allow it will cost you $30/month or more. The default search will override the vanilla one when it detects sphinx is running. If it is not, like when you index, the default one will then be enabled.

Sign In or Register to comment.