Multilingual and Multimodal Information Access Evaluation

by Agosti, Maristella; Ferro, Nicola; Peters, Carol; De Rijke, Maarten; Smeaton, Alan

• ISBN13: 9783642159978
• ISBN10: 3642159974
• Format: Paperback
• Copyright: 2010-10-18
• Publisher: Springer-Verlag New York Inc

Summary

This book constitutes the refereed proceedings of the 11th symposium of the Cross-Language Evaluation Forum, CLEF 2010, held in Padua, Italy, in September 2010 as the First International Conference on Multilingual and Multimodal Information Access Evaluation - in continuation of the popular CLEF campaigns and workshops that have run for the last decade.The 12 revised full papers presented together with 2 keynote talks and 2 panel presentations were carefully reviewed and selected from numerous submissions. The papers include advanced research into the evaluation of complex multimodal and multilingual information systems in order to support individuals, organizations, and communities who design, develop, employ, and improve such systems. The papers are organized in topical sections on resources, tools, and methods; experimental collections and datasets, and evaluation methodologies.

Table of Contents

Keynote Addresses
IR between Science and Engineering, and the Role of Experimentation	p. 1
Retrieval Evaluation in Practice	p. 2
Resources, Tools, and Methods
A Dictionary- and Corpus-Independent Statistical Lemmatizer for Information Retrieval in Low Resource Languages	p. 3
A New Approach for Cross-Language Plagiarism Analysis	p. 15
Creating a Persian-English Comparable Corpus	p. 27
Experimental Collections and Datasets (1)
Validating Query Simulators: An Experiment Using Commercial Searches and Purchases	p. 40
Using Parallel Corpora for Multilingual (Multi-document) Summarisation Evaluation	p. 52
Experimental Collections and Datasets (2)
MapReduce for Information Retrieval Evaluation: "Let's Quickly Test This on 12 TB of Data"	p. 64
Which Log for Which Information? Gathering Multilingual Data from Different Log File Types	p. 70
Evaluation Methodologies and Metrics (1)
Examining the Robustness of Evaluation Metrics for Patent Retrieval with Incomplete Relevance Judgements	p. 82
On the Evaluation of Entity Profiles	p. 94
Evaluation Methodologies and Metrics (2)
Evaluating Information Extraction	p. 100
Tie-Breaking Bias: Effect of an Uncontrolled Parameter on Information Retrieval Evaluation	p. 112
Automated Component-Level Evaluation: Present and Future	p. 124
Panels
The Four Ladies of Experimental Evaluation	p. 136
A PROMISE for Experimental Evaluation	p. 140
Author Index	p. 145
Table of Contents provided by Ingram. All Rights Reserved.