Search Results

Now showing 1 - 3 of 3

Some of the metrics are blocked by your
consent settings
On the reliability of diversity and redundancy-based search metrics
(2015-01-01)
Tangsomboon, Ake
;
Leelanupab, Teerapong
Traditional approaches to ranking documents in Information Retrieval (IR) are under the assumption that the representation of information needs is clear and well-defined. This representation, which is usually in the form of a search query, is arguably considered ambiguous or underspecified. To deal with this uncertainty, much recent research has focused on creating IR systems that diversify search results so as to satisfy the multiple possible information needs underlying the query. To validate these IR systems, many new evaluation metrics have been proposed to quantify their effectiveness in terms of diversity and redundancy. Among these, a new diversity-based metric, called normalized Coverage Frequency (nCF), has lately been proposed to quantify diversity in a ranking. When a new metric is proposed, its reliability needs to be validated. This paper conducts an empirical experiment to compares and contrast state-of-the-art diversity and redundancy-based metrics, in term of discriminative power and stability of system rankings. Our experiment shows that the nCF is rated the best among all the studied metrics. Moreover, this finding is confirmed by when nCF is interpolated with other redundancy-based metrics (i.e., ERR-SA and A-nDCG). the nCF is considered more relatively robust than another diversity metric, subtopic-recall.
Some of the metrics are blocked by your
consent settings
Evaluating diversity and redundancy-based search metrics independently
(2014-11-26)
Tangsomboon, Ake
;
Leelanupab, Teerapong
This paper proposes a new evaluation metric, normalized Coverage Frequency (nCF), which aims to explicitly evaluate the diversity of search results, going beyond the drawbacks of previously proposed measures. In fact, two of the most widely adopted metrics for the diversity retrieval task, namely-nDCG and Intent-Aware Expected Reciprocal Rank (ERR-IA), explicitly evaluate redundancy, but not diversity. While there exists a genuine diversity-based metric called Intent Recall (I-rec), it has some drawbacks. These drawbacks may be inherited by other derived metrics such as D#-nDCG, which combines I-rec with a modified version of nDCG. The proposed nCF metric assesses how often query-intents are successfully covered throughout a ranked list up to a given rank position. A comprehensive study is conducted using both real and synthetic data to compare nCF with-nDCG, ERR-IA, I-recall and D#-nDCG. Results show that the proposed metric correlates well with the existing ones while it is capable of capturing other factors, e.g., a series of coverage. In addition, we categorize the existing metrics into two distinct groups, i.e., diversity and novelty, based upon their intuitive measurements and suggest that they be used independently according to what they quantify for the ease of performance interpretation.
Some of the metrics are blocked by your
consent settings
CoZpace: A proposal for collaborative web search for sharing search records and interactions
(2014-10-14)
Kruajirayu, Hannarin
;
Tangsomboon, Ake
;
Leelanupab, Teerapong
Today's search systems are mainly designed for one user to use individually. Search activities in practice can, however, be conducted by two or more users working together. This is due to the fact that information seeking tasks are complex and often requires multiple users' efforts to collaboratively assess and search for relevant information. In some tasks, there is a large amount of information to search or such information requires human intelligence to judge whether it is relevant to information needs. This paper introduces the development of collaborative search system, named "CoZpace". CoZpace is a Web-based application, which allows a pair or a group of users to create a collaborative search task. Within the created task, a user can invite and communicate with other members in a group, share search history records, and mark search results considered relevant. We also propose a newly developed feature, called "snapboard", to support collaborative search activities. With the snapboard, users are allowed to take a snapshot of a part of focused information in a Web site and then share it among collaborators on a display board. A user can mark it as relevant or comment it for further reviews of other collaborators.

KMITL

Browse

Filters

Settings

Sort By

Results per page

Search Results