Spider Options

MSpider MUST be capable of being run from the command line in batch-mode and MUST NOT require a graphical user interface. Essentially, the user MUST be able to run the spider client with a command line similar to the following:

        java MSpider [options] >&! logfile &

and then coming back some later time to view the resutls.

The spider client MUST support at least the following command-line options. The designer MAY provide additional options, but all such options MUST be fully described in the user documentation.

-c #

Set the CRAWL-MAX parameter to # for this run. Note that different CRAWL-MAXs may be used on different runs.

-i iPage

Set the initial page from which to begin the crawl.

-m modelFileName

Load/save spider database to/from modelFileName.

-d dumpFileName

Dump statistics on the REVERSE INDEX into the specified file and then exit.

In addition, the system SHOULD provide the following option:

-h

Print a help message and then exit.