Intelligent spider for Internet searching

Hsinchun Chen, Yi Ming Chung, Marshall Ramsey, Christopher C. Yang, Pai Chun Ma, Jerome Yen

Research output: Contribution to journalConference articlepeer-review

13 Scopus citations


As the World-Wide Web (WWW) based Internet services become more popular, information overload also becomes a pressing research problem. Difficulties with searching on Internet get worse as the amount of information that available on the Internet increases. A scalable approach to support Internet search is critical to the success of Internet services and other current or future National Information Infrastructure (NII) applications. A new approach to build intelligent personal spider (agent), which is based on automatic textual analysis of Internet documents, is proposed in this paper. Best first search and genetic algorithm have been tested to develop the intelligent spider. These personal spiders are able to dynamically and intelligently analyze the contents of the users selected homepages as the starting point to search for the most relevant homepages based on the links and indexing. An intelligent spider must have the capability to make adjustments according to progress of searching in order to be an intelligent agent. However, the current searching engines do not have the communication between the users and the robots. The spider presented in this paper use Java to develop the user interface such that the users can adjust the control parameters according to the progress and observe the intermediate results. The performances of the genetic algorithm based and best first search based spiders are also reported.

Original languageEnglish (US)
Pages (from-to)178-188
Number of pages11
JournalProceedings of the Hawaii International Conference on System Sciences
StatePublished - 1997
EventProceedings of the 1997 30th Annual Hawaii International Conference on System Sciences. Part 1 (of 6) - Wailea, HI, USA
Duration: Jan 7 1997Jan 10 1997

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'Intelligent spider for Internet searching'. Together they form a unique fingerprint.

Cite this