Evaluation of Rich and Explicit Feedback for Exploratory Search

Tools

Sørig, Esben; Collignon, Nicolas; Fiebrink, Rebecca and Kando, Noriko. 2019. 'Evaluation of Rich and Explicit Feedback for Exploratory Search'. In: Workshop on Evaluation of Personalisation in Information Retrieval (held at CHIIR 2019). Glasgow, United Kingdom 14 March 2019. [Conference or Workshop Item]

Preview

Text
SørigEtAlWEPIR2019.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.
Download (958kB) | Preview

Official URL: http://www.ir.disco.unimib.it/wepir2019/

Abstract or Description

A user’s goals and interests during an exploratory search task are often ambiguous and complex. When engaging with new documents, people regularly use annotations to help better understand them and make them their own. These annotations can also pro- vide rich information to gain insight into a reader’s interests. In particular, it is possible to use highlights as a richer form of feed- back compared to traditional document-level relevance feedback. We first show that this form of feedback leads to improvements in document retrieval in exploratory search tasks with simulated users when compared to relevance feedback. We then present an evaluation platform which will allow us to understand the retrieval performance, user experience, and behavioral characteristics of human subjects using highlights as feedback. Finally, we propose an experimental design with human subjects. We hope that our experimental findings will help improve current simulated user evaluations for such systems.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

https://doi.org/10.1145/1122445.1122456