Proceedings Template - WORD

Verifying Genre-based Clustering Approach to Content Extraction

 



 

 


ABSTRACT

The content of a webpage is usually contained within a small body of text and images, or perhaps several articles on the same page; however, the content may be lost in the clutter, particularly hurting users browsing on small cell phone and PDA screens and visually impaired users relying on speed rendering of web pages. Using the genre of a web page, we have created a solution, Crunch that automatically identifies clutter and removes it, thus leaving a clean content-full page. In order to evaluate the improvement in the applications for this technology, we identified a number of experiments. In this paper, we have those experiments, the associated results and their evaluation.

Categories and Subject Descriptors

I.7.4 [Document and Text Processing]: Electronic Publishing; H.3.5 [Information Storage and Retrieval]: Online Information Services Web-based Services

General Terms

Human Factors, Algorithms, Standardization.

Keywords

Website classification, clustering, content extraction, reformatting, HTML, context, accessibility, speech rendering.

1.     INTRODUCTION

Web pages are often cluttered with extraneous materials, perhaps attempting to attract the users attention or improve the users efficiency, but they may end up distracting the user from the actual content. These features may include script and flash-driven animations, other kinds of images not directly associated with a main text body, menus and guides, links scattered around the screen, etc. The automatic extraction of heuristically-defined content from webpages has many applications, including enabling end-users to access the web more easily over constrained devices and providing better access to the web for the blind or otherwise disabled. We have developed a framework, Crunch [1] [2] [3], as a web proxy that employs various heuristics in the form of filters and filter settings that are applied to the DOM (www.w3.org/DOM) of the web page to achieve content extraction via clutter reduction.

In order to reduce human involvement in selecting the heuristic filter settings, we consider utilizing a websites genre classification. Crunch can then obtain some previously (manually) adjusted settings for a newly visited website by automatically classifying it as sufficiently similar to a genre-cluster of known websites, at least one of which has known good settings which, we found empirically, produces better content extraction results than any possible one-size-fits-all default settings. The clustering algorithm and genre-based adaptive application of filters have been described in our prior publication. [3]

To evaluate the effectiveness of our clustering approach and the subsequent clutter free web pages, we conducted experiments to test Crunchs benefits in the two areas mentioned above. In this paper, we present our experiments and associated results.

2.     EXPERIMENTS & RESULTS

2.1     Speech rendering experiment

The goal of our first experiment was to consider the readability, by conventional screen reader software, of the extractions produced by Crunch compared to the raw webpages. We measured the length of time, in seconds, that it took the screen reader to render (speak) each variant of a given page, considering a variety of webpages. We ran trials with both the free demo version of JAWS (www.freedomscientific.com) and a licensed copy of Home Page Reader (www.ibm.com), which vary slightly in how they preprocess the raw HTML although the measured time differences were under 3 seconds in all cases, so we average the results in our reporting below. The idea was to determine the amount of time a visually disabled user might save by using our content extraction technology. The notion of content is inherently subjective and our determination of what is the content vs. non-content was performed by visual and auditory inspection.

For this experiment, we chose 11 websites that represent a variety of layout formats. We included websites from all the major genres that appeared in our corpus (e.g., news, shopping, tech news, astronomy), but were also careful to cover different structures (columns, single-body articles, portal-based and blog-style sites), as well as W3C-compliant accessible sites vs. non-compliant sites. We passed the original webpages vs. the Crunch outputs through the screen readers. We then measured the time it took, in minutes, to read (speech render) the entire webpages.

Table 1 Speech rendering results

Site

(accessed on May 18th, 2005)

Read original webpage (minutes)

Read page produced by Crunch (minutes)

CNN.com front page

10 :09

1 :08

CNN.com subsidiary page

7:35

2:44

Slashdot.org front page

25:20

17:53

Slashdot.org article page

14 :13

6 :15

MSNBC front page

10 :47

2 :40

MSNBC article page

11:12

3:43

Yahoo News front page

25:15

16:39

Yahoo News article page

14:08

5:13

NASA Ames front page

2 :18

1 :48

NASA Ames Research page

1:57

1:17

Amazon.com front page

13:28

7:42

From these tests and from the anecdotal accounts of visually impaired users (e.g., attendees at the 2005 W4A meeting), it is clear that blind web users typically spend tens of minutes listening to nearly any single webpage using a commercial screen reader alone - and this is absolutely unacceptable! We found that using Crunch together with such a screen reader reduces by 10-80% the time spent in reading the page while the content on the webpage remains qualitatively accurate. The least significant improvement (< 10% speedup in reading) using Crunch was on the main page of a given site, where the settings preserved a larger percentage of navigation links (Crunchs heuristics distinguish between front and auxiliary pages since front pages are often intended to operate as portals). The greatest improvement noticed was on subsidiary pages of websites, usually containing contentful articles.

2.2     Constrained screen testing

We also evaluated how well Crunch compared to other content extraction and webpage reformatting technologies designed for devices with limited screen real-estate. We used the same samples as for the speech rendering tests above, and displayed both the original webpages and the pages output by Crunch on various combinations of handheld devices and browsers. We tested the system on the Toshiba e805 and HP iPaq 2215 PDAs running Microsofts PocketPC OS, with Pocket Internet Explorer and BitStreams Thunderbird browsers, respectively. We measured the amount of content on the first screenful at both 320x240 and 640x480 resolutions. We also used a Blackberry 7100t running a proprietary Blackberry browser and a Microsoft Smartphone i600 running Internet Explorer and Opera Mobile Browser.

The purpose of these tests was to demonstrate the increase in relevant and useful content displayed on a small screen when using Crunch vs. not using Crunch. We would like to reiterate that, in the general-purpose case absent any model of the authors or readers intents, content is subjective. For this experiment, we define content as the number of relevant words shown on the screen, measured by visual inspection.

Table 2 - Constrained device testing results

Number of words (PDA 320x240)

Number of words (PDA 640x480)

Number of words (Blackberry)

Number of words (Opera on Smartphone)

I

II

I

II

I

II

I

II

29

38

102

217

17

30

10

32

29

185

158

338

17

68

13

59

49

154

134

270

48

68

43

63

45

80

215

215

48

68

43

63

56

56

111

111

27

27

27

27

123

123

370

370

14

53

13

48

20

34

20

75

27

33

25

29

20

93

20

93

27

51

25

45

7

34

185

185

3

19

3

19

15

112

112

112

12

30

12

28

28

35

247

247

12

34

43

43

I without Crunch, i.e., original webpage.

II with Crunch, i.e., the page is passed through Crunch, with automatic genre-based settings.

From the data presented in Table 2, we see that Crunch with genre-based automatic selection of filter settings is very useful towards maximizing the amount of content displayed on constrained devices. The most significant difference was on a 320x240 resolution PDA screen, where there was on average a 215% increase in the amount of content displayed on the screen. This increase jumped dramatically up to a 750% when considering only news articles. With 640x480 resolution, we found an average increase of 133% of content on the first screenful. Several of the pages tested were able to fully render within that screenful. When testing with the cell phone browsers, we found the results to be almost identical, almost 185% improvement in both cases, presumably because of the very similar screen sizes. The main difference was due to the Opera browsers default behavior of jumping to the middle of the page where it found the largest concentration of text attempting to skip over anticipated non-content, which was lacking on the Blackberry. In none of our trials did a webpage rendered using Crunch display less content on the first screenful than the original page rendered on the same constrained device. However, the Opera comparison is somewhat problematic, not always counting the same words, due to Operas skip-to-the-middle heuristic.

3.     CONCLUSION

In this paper we have presented experiments that demonstrate the substantial improvement achieved by the application of content extraction via clutter removal to web pages, especially in the domains tested. We also hope that the experimental process shown here is used towards future content extraction applications so as to evaluate the goodness rate achieved.

4.     REFERENCES

[1]     Suhit Gupta, Gail Kaiser, David Neistadt, Peter Grimm, "DOM-based Content Extraction of HTML Documents", 12th International World Wide Web Conference, May 2003

[2]     Suhit Gupta, Gail Kaiser, "CRUNCH - Web-based Collaboration for Persons with Disabilities", W3C WAI, Teleconference on Making Collaboration Technologies Accessible for Persons with Disabilities, Apr 2003

[3]     Suhit Gupta, Context-Based Content Extraction of HTML Document, PhD Dissertation, New York, NY, June 2005