Online learning to rank

Abstract

"Ranking of the documents has been always a prominent area of research in IR community. Most of the techniques for learning to rank evolved around Cranfield paradigm, in which supervised and semi-supervised machine learning algorithms were developed.Ranking framework developed according to this model generally have training phase and testing phase, during training phase the loss function is optimized ie. some specific parameters are learned from the training data set. Usually, this training data set consists of the algorithm’s input and its output (so what the algorithm should do with a certain input). In document ranking, this can be labeled documents with a relevancy score associated with them in correspondence to a query. Here the documents and query are inputs and relevancy score or ranked list of the document in decreasing order is a output. Since there are various algorithms, the learned parameters vary as well. In the case of neural networks, the edges ’weights’ are being learned. The output or ranking generated by the trained system is evaluated by measures like NDCG [8]. But instead of Cranfiled paradigm we will consider learning to rank in an online environment from the view point of reinforcement learning, where with the help of some base line methods we will see that how retrieval system behaves when user preferences changes. In this thesis we introduce the concepts and previous work on which this thesis is based. Most immediately relevant are the baseline approaches for online learning to rank, which are presented in chapter 3. However, we build up related concept in several steps. First, we start with general terminology from IR such as learning to rank, evaluation measures etc.Because online learning to rank in IR relies on implicit feedback, such as click data, we review existing work on uses of such data for improving IR systems. We also detail the feedback mechanisms. build on,the interleaved comparison methods balanced interleave (BI),documentn constraints (DC) and team draft (TD)[5].Besides IR, this thesis draws on concepts and techniques developed in reinforcement learning (RL), a machine learning paradigm in which systems learn from interactions with their environment. Many ideas developed within RL are applicable to online learning to rank for IR, and we review relevant research in this area. For our experiments we will observe the performance of various interleaved methods in the simulated setting and evaluate their performances."