Search engine for your site

htDig Installation

htDig is an indexing and searching tool for Web sites. It seems a little complex at first but is well worth it in the end. The best resource for complete information on htDig is the htDig website. What follows below are instructions on how to install htDig on a Web server.

Requirements

htDig requires several programs and compilers to be installed on the server side. The web server we used for this example is pair.com. The following are required on the server (and are usually present):

• Unix machine
• a C compiler
• C++ compiler

Installing htDIG

When installing htDig, be sure you are referencing the correct directory. Also remember that UNIX machines are case sensitive. Follow the directions in the order given below to have a successful installation of the search engine.

1. Download latest version

Go to www.htdig.org/where.html and download the latest version of htDig (these instructions used htdig-3.1.6.tar.gz). Upload the file into the directory you would like to install the htDig engine.

2. Log in to your server

In order to execute and compile htDig, you will need to log into your machine via SSH terminal window (to execute files) and an FTP client (to transfer files).

3. Extract htDig

In the terminal window change to the directory where you want to install the htDig engine. After you are in the right directory, type one of the two following:

tar xzf htdig-3.1.6.tar.gz
(If you do not have GNU tar do the following)
gunzip -c htdig-3.1.6.tar.gz | tar xf

4. Run the configuration

In the terminal window, change to the directory that you just created (cd htdig-3.1.6.tar.gz). Next run the configuration by entering the following

./configure
If you get any critical errors, then your server may 
not support everything needed to run htDig.

5. Configure the CONFIG file

This is one of the more challenging parts, you must be sure you reference the correct directories or else it will not work. Download the CONFIG file that was created in the htDig-3.1.6 folder via FTP. Open it up for editing using a text editor. Read through the file and change the value of the following variables:

prefix, CGIBIN_DIR, IMAGE_DIR, IMAGE_URL_PREFIX, SEARCH_DIR

6. Build the system

In the terminal window, make sure you are still in the htDig-3.1.6 directory. Run the Make file by typing the following

make
(after this finishes compile the make install as follows)
make install

"Make install" will attempt to create directories into the htDig-3.1.6 directory. If it fails, you will see errors such as "No such file or directory" or "Permission denied". "No such file or directory" means that you made a mistake on referencing directories in the CONFIG file. "Permission Denied" can mean the same thing or it can mean that you are trying to write to a folder that does not have write privileges (right click the folders to see if write privileges are on).

7. Configuring htdig.conf

In the terminal window, change directories to the conf folder located inside htDig-3.1.6. Download the file that has been created title htdig.conf. This file is where you will tell htDig what you want to search and what you do not want searched. The following are variables that need to be changed or checked to make sure they are correct:

8. Running the Dig

Change directories in the terminal window to the "bin" folder located inside "htDig-3.1.6". Once there, type the following to run the dig:

./rundig -vvv

This will run the dig and show exactly what it is doing. If you have done everything correct, it can take up to 5 minutes searching and building the database. If you made a mistake, then you probably have messed up in your htdig.conf or CONFIG file. Go back to step 4 if you get errors and be sure to go in order.

9. Creating the HTML search form

After running the dig and completing the steps above, a file should be created in your htdig-3.1.6 folder called search.html. This file has the code you will need to search your website. The following is what www.classanytime.com uses:

<form method="post"
      action="http://www.classanytime.com/cgi-bin/htsearch">
    <font size="-1">
    Match: <select name="method">
    <option value="and">All</option>
    <option value="or">Any</option>
    <option value="boolean">Boolean</option>
    </select>
    Format: <select name="format">
    <option value="builtin-long">Long</option>
    <option value="builtin-short">Short</option>
    </select>
    Sort by: <select name="sort">
    <option value="score">Score</option>
    <option value="time">Time</option>
    <option value="title">Title</option>
    <option value="revscore">Reverse Score</option>
    <option value="revtime">Reverse Time</option>
    <option value="revtitle">Reverse Title</option>
    </select>
    </font>
    <input type="hidden" name="config" value="htdig">
    <input type="hidden" name="restrict" value="">
    <input type="hidden" name="exclude" value="">
    <br>
    Search:
    <input type="text" size="30" name="words" value="">
    <input type="submit" value="Search">
</form>
    

UNIX help

Here are some UNIX commands that you will need to know:

• ls - Lists all the files in the current directory
• cd foldername - Changes the current directory to that folder
• cd ../ - Moves up one directory
• ./filename - Executes the file