WWW2008 Posters - WWW 2008: Posters
Skip to main content.

Poster


Track: Posters

Paper Title:
Low-load Server Crawler: Design and Evaluation

Authors:

  • Katsuko T. Nakahira(Nagaoka University of Technology)
  • Tetsuya Hoshino(Nagaoka University of Technology)
  • Yoshiki Mikami(Nagaoka University of Technology)

Abstract:
This paper proposes a method of crawling Web servers connected to the Internet without imposing a high processing load. We are using the crawler for a field survey of the digital divide, including the ability to connect to the network. Rather than employing normal Web "page" crawling algorithm, which usually collect all pages found on the target server, we have developed "server" crawling algorithm, which collect only minimum pages from the same server and achieved low-load and high-speed crawling of servers.

PDF version












Inquiries can be sent to: Email contact: program-chairs at www2008.org

Valid XHTML 1.0 Transitional