# Preference learning

> Mediated Wiki article. Canonical URL: https://mediated.wiki/source/Preference_learning
> Markdown URL: https://mediated.wiki/source/Preference_learning.md
> Source: https://en.wikipedia.org/wiki/Preference_learning
> Source revision: 1316868925
> License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/)

{{Short description|Subfield of machine learning}}
'''Preference learning''' is a subfield of [machine learning](/source/machine_learning) that focuses on modeling and predicting preferences based on observed preference information.<ref>{{Cite Mehryar Afshin Ameet 2012}}</ref> Preference learning typically involves [supervised learning](/source/supervised_learning) using datasets of pairwise preference comparisons, rankings, or other preference information.

==Tasks==

The main task in preference learning concerns problems in "[learning to rank](/source/learning_to_rank)". According to different types of preference information observed, the tasks are categorized as three main problems in the book ''Preference Learning'':<ref>{{Cite book |url=https://books.google.com/books?id=nc3XcH9XSgYC&pg=PA4 |title=Preference learning |date=2010 |publisher=Springer |isbn=978-3-642-14124-9 |editor-last=Fürnkranz |editor-first=Johannes |location= |pages=3–8 |editor-last2=Hüllermeier |editor-first2=Eyke}}</ref>

===Label ranking===

In label ranking, the model has an instance space <math>X=\{x_i\}\,\!</math> and a [finite set](/source/finite_set) of labels <math>Y=\{y_i|i=1,2,\cdots,k\}\,\!</math>. The preference information is given in the form <math>y_i \succ_{x} y_j\,\!</math> indicating instance <math>x\,\!</math> shows preference in <math>y_i\,\!</math> rather than <math>y_j\,\!</math>. A set of preference information is used as [training data](/source/Training%2C_validation%2C_and_test_data_sets) in the model. The task of this model is to find a preference ranking among the labels for any instance.

It was observed that some conventional [classification](/source/Classification_in_machine_learning) problems can be generalized in the framework of label ranking problem:<ref>{{Cite journal |date=2002 |title=Constraint classification for multiclass classification and ranking |url=https://proceedings.neurips.cc/paper_files/paper/2002/file/16026d60ff9b54410b3435b403afd226-Paper.pdf |journal=NeurIPS}}</ref> if a training instance <math>x\,\!</math> is labeled as class <math>y_i\,\!</math>, it implies that <math>\forall j \neq i, y_i \succ_{x} y_j\,\!</math>. In the [multi-label](/source/Multi-label_classification) case, <math>x\,\!</math> is associated with a set of labels <math>L \subseteq Y\,\!</math> and thus the model can extract a set of preference information <math>\{y_i \succ_{x} y_j | y_i \in L, y_j \in Y\backslash L\}\,\!</math>. Training a preference model on this preference information and the classification result of an instance is just the corresponding top ranking label.

===Instance ranking===

Instance ranking also has the instance space <math>X\,\!</math> and label set <math>Y\,\!</math>. In this task, labels are defined to have a fixed order <math>y_1 \succ y_2 \succ \cdots \succ y_k\,\!</math> and each instance <math>x_l\,\!</math> is associated with a label <math>y_l\,\!</math>. Giving a set of instances as training data, the goal of this task is to find the ranking order for a new set of instances.

===Object ranking===

Object ranking is similar to instance ranking except that no labels are associated with instances. Given a set of pairwise preference information in the form <math>x_i \succ x_j\,\!</math> and the model should find out a ranking order among instances.

==Techniques==

There are two practical representations of the preference information <math>A \succ B\,\!</math>. One is assigning <math>A\,\!</math> and <math>B\,\!</math> with two real numbers <math>a\,\!</math> and <math>b\,\!</math> respectively such that <math>a > b\,\!</math>. Another one is assigning a binary value <math>V(A,B) \in \{0,1\}\,\!</math> for all pairs <math>(A,B)\,\!</math> denoting whether <math>A \succ B\,\!</math> or <math>B \succ A\,\!</math>. Corresponding to these two different representations, there are two different techniques applied to the learning process.

===Utility function===

If we can find a mapping from data to real numbers, ranking the data can be solved by ranking the real numbers. This mapping is called [utility function](/source/utility_function). For label ranking the mapping is a function <math>f: X \times Y \rightarrow \mathbb{R}\,\!</math> such that <math>y_i \succ_x y_j \Rightarrow f(x,y_i) > f(x,y_j)\,\!</math>. For instance ranking and object ranking, the mapping is a function <math>f: X \rightarrow \mathbb{R}\,\!</math>.

Finding the utility function is a [regression](/source/Regression_analysis) learning problem{{citation needed|date=March 2025}} which is well developed in machine learning.

===Preference relations===

The binary representation of preference information is called preference relation. For each pair of alternatives (instances or labels), a binary predicate can be learned by conventional supervised learning approach. Fürnkranz and Hüllermeier proposed this approach in label ranking problem.<ref name=":0">{{Cite book |last1=Fürnkranz |first1=Johannes |last2=Hüllermeier |first2=Eyke |chapter=Pairwise Preference Learning and Ranking |series=Lecture Notes in Computer Science |date=2003 |volume=2837 |editor-last=Lavrač |editor-first=Nada |editor2-last=Gamberger |editor2-first=Dragan |editor3-last=Blockeel |editor3-first=Hendrik |editor4-last=Todorovski |editor4-first=Ljupčo |title=Machine Learning: ECML 2003 |chapter-url=https://link.springer.com/chapter/10.1007/978-3-540-39857-8_15 |language=en |location=Berlin, Heidelberg |publisher=Springer |pages=145–156 |doi=10.1007/978-3-540-39857-8_15 |isbn=978-3-540-39857-8}}</ref> For object ranking, there is an early approach by Cohen et al.<ref>{{Cite journal |last1=Cohen |first1=William W. |last2=Schapire |first2=Robert E. |last3=Singer |first3=Yoram |date=1998-07-31 |title=Learning to order things |url=https://dl.acm.org/doi/10.5555/302528.302736 |journal=NeurIPS |series= |location=Cambridge, MA, USA |publisher=MIT Press |pages=451–457 |doi= |isbn=978-0-262-10076-2}}</ref>

Using preference relations to predict the ranking will not be so intuitive. Since observed preference relations may not always be transitive due to inconsistencies in the data, finding a ranking that satisfies all the preference relations may not be possible or may result in multiple possible solutions. A more common approach is to find a ranking solution which is maximally consistent with the preference relations. This approach is a natural extension of pairwise classification.<ref name=":0" />

==Uses==

Preference learning can be used in ranking search results according to feedback of user preference. Given a query and a set of documents, a learning model is used to find the ranking of documents corresponding to the [relevance](/source/relevance_(information_retrieval)) with this query. More discussions on research in this field can be found in [Tie-Yan Liu](/source/Tie-Yan_Liu)'s survey paper.<ref>{{Cite journal |last=Liu |first=Tie-Yan |date=2007 |title=Learning to Rank for Information Retrieval |url=http://www.nowpublishers.com/article/Details/INR-016 |journal=Foundations and Trends in Information Retrieval |language=en |volume=3 |issue=3 |pages=225–331 |doi=10.1561/1500000016 |issn=1554-0669|url-access=subscription }}</ref>

Another application of preference learning is [recommender systems](/source/recommender_systems).<ref>{{Citation |last1=Gemmis |first1=Marco de |title=Learning Preference Models in Recommender Systems |date=2010 |work=Preference Learning |pages=387–407 |editor-last=Fürnkranz |editor-first=Johannes |url=http://link.springer.com/10.1007/978-3-642-14125-6_18 |access-date=2024-11-05 |place= |publisher=Springer |language=en |doi=10.1007/978-3-642-14125-6_18 |isbn=978-3-642-14124-9 |last2=Iaquinta |first2=Leo |last3=Lops |first3=Pasquale |last4=Musto |first4=Cataldo |last5=Narducci |first5=Fedelucio |last6=Semeraro |first6=Giovanni |editor2-last=Hüllermeier |editor2-first=Eyke|url-access=subscription }}</ref> Online store may analyze customer's purchase record to learn a preference model and then recommend similar products to customers. Internet content providers can make use of user's ratings to provide more user preferred contents.

==References==

{{Reflist}}

Category:Information retrieval techniques
Category:Machine learning

---
Adapted from the Wikipedia article [Preference learning](https://en.wikipedia.org/wiki/Preference_learning) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/Preference_learning?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.