www8.dvi
<- Previous | First | Next ->

All Intersection Non-Netscape

Avg. Precision Avg. Precision Avg. Precision

Algorithm Precision at 10 Precision at 10 Precision at 10

Companion 0.541 0.417 0.666 0.501 0.540 0.401

Cocitation 0.518 0.363 0.605 0.435 0.434 0.325

Netscape 0.343 0.241 0.502 0.357 n/a

Table 5: Precision metrics for each algorithm for three groups of URLs

Companion better than Netscape 0.0026 0.0041 0.0340

n/a

Cocitation better than Netscape 0.0136 0.0164 0.1685 0.2340

n/a

Companion better than Cocitation 0.1922 0.3898 0.0793 0.2628 0.2643 0.4180

Table 6: Sign Test and Wilcoxon Sum of Ranks Test for algorithm pairs

For a given URL u, the u of an algorithm is the sum of the precision at

each rank where the answer of the algorithm for u received a '1' score divided by the total number

of the answers of the algorithm for u receiving a '1' score. If the algorithm does not return any

answers for u, its average precision for u is 0. The overall for an algorithm is the

sum of all the average precisions for all the URLs divided by the total number of URLs.

For each of the three groups of URLs , , and ), Table 5 shows the

average precision and the precision at 10 for each algorithm. Figure 1 shows the precision at r for

each of these groups of URLs in graphs (a), (b), and (c). Figures 1 (a) and 1 (b) illustrate that the

Companion and Cocitation algorithms substantially outperform Netscape's algorithm at all ranks,

and the Companion algorithm almost always outperforms the Cocitation algorithm.

The group is the most interesting comparison, since it avoids penalizing an algorithm for not returning at least one answer. For the group, Netscape's algorithm

achieves a precision at 10 of 0.357, while the Companion algorithm achieves a precision at 10 of
0.501 (40% better), and the Cocitation algorithm achieves a precision at 10 of 0.435 (22% better). The average precision in the group does not penalize an algorithm for returning

fewer than 10 answers. Under this metric, the Companion algorithm is 32% better than Netscape's

algorithm, while the Cocitation algorithm is 20% better than Netscape's algorithm.

In the group that includes all URLs, all three algorithms had drops in their precision at 10

values. There are two reasons for this. The rst is that algorithms were given a precision of 0 for a

Companion Cocitation Netscape


1.0


1.0


1.0


0.8


0.8


0.8

Precision

DeviceRGB 3 bits

0.6


0.6


0.6


0.4

DeviceRGB 3 bits

0.4


0.4

DeviceRGB 3 bits

0.2


0.2


0.2


0.0


0.0


0.0

1 10

1 10

1 10

r

r

r

(a) All

(b) Intersection

(c) Non-Netscape

Figure 1: Precision at r for the three groups of URLs

10


<- Previous | First | Next ->