# Relevance vector machine

> Mediated Wiki article. Canonical URL: https://mediated.wiki/source/Relevance_vector_machine
> Markdown URL: https://mediated.wiki/source/Relevance_vector_machine.md
> Source: https://en.wikipedia.org/wiki/Relevance_vector_machine
> Source revision: 1304537313
> License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/)

{{Short description|Machine learning technique}}
{{Machine learning|Supervised learning}}
In [mathematics](/source/mathematics), a '''Relevance Vector Machine (RVM)''' is a [machine learning](/source/machine_learning) technique that uses [Bayesian inference](/source/Bayesian_inference) to obtain [parsimonious](/source/Occam's_razor) solutions for [regression](/source/Regression_analysis) and [probabilistic classification](/source/probabilistic_classification).<ref>{{cite journal | last=Tipping | first=Michael E. |title=Sparse Bayesian Learning and the Relevance Vector Machine |year=2001 |journal = [Journal of Machine Learning Research](/source/Journal_of_Machine_Learning_Research) |volume=1 |pages=211&ndash;244 |url=http://jmlr.csail.mit.edu/papers/v1/tipping01a.html }}</ref> A greedy optimisation procedure and thus fast version were subsequently developed.<ref>{{cite journal |last1=Tipping |first1=Michael |last2=Faul |first2=Anita |title=Fast Marginal Likelihood Maximisation for Sparse Bayesian Models |journal=Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics |date=2003 |pages=276–283 |url=https://proceedings.mlr.press/r4/tipping03a.html |access-date=21 November 2024}}</ref><ref>{{cite journal |last1=Faul |first1=Anita |last2=Tipping |first2=Michael |title=Analysis of Sparse Bayesian Learning |journal=Advances in Neural Information Processing Systems |date=2001 |url=https://proceedings.neurips.cc/paper_files/paper/2001/file/02b1be0d48924c327124732726097157-Paper.pdf |access-date=21 November 2024}}</ref>
The RVM has an identical functional form to the [support vector machine](/source/support_vector_machine), but provides probabilistic classification.

It is actually equivalent to a [Gaussian process](/source/Gaussian_process) model with [covariance function](/source/covariance_function):
:<math>k(\mathbf{x},\mathbf{x'}) = \sum_{j=1}^N \frac{1}{\alpha_j} \varphi(\mathbf{x},\mathbf{x}_j)\varphi(\mathbf{x}',\mathbf{x}_j) </math>
where <math>\varphi</math> is the [kernel function](/source/kernel_function) (usually Gaussian), <math>\alpha_j</math> are the variances of the prior on the weight vector
<math>w \sim N(0,\alpha^{-1}I)</math>, and <math>\mathbf{x}_1,\ldots,\mathbf{x}_N</math> are the input vectors of the [training set](/source/training_set).<ref>{{cite thesis
|type=Ph.D.
|last=Candela
|first=Joaquin Quiñonero
|date=2004
|title=Learning with Uncertainty - Gaussian Processes and Relevance Vector Machines
|publisher=Technical University of Denmark |url=http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3237/pdf/imm3237.pdf |chapter=Sparse Probabilistic Linear Models and the RVM
|access-date=April 22, 2016
}}</ref>

Compared to that of [support vector machine](/source/support_vector_machine)s (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an [expectation maximization](/source/expectation_maximization) (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard [sequential minimal optimization](/source/sequential_minimal_optimization) (SMO)-based algorithms employed by [SVM](/source/Support_vector_machine)s, which are guaranteed to find a global optimum (of the convex problem).

The relevance vector machine was [patented in the United States](/source/Software_patents_under_United_States_patent_law) by [Microsoft](/source/Microsoft) (patent expired September 4, 2019).<ref>{{cite patent
|country = US
|number = 6633857
|title = Relevance vector machine
|inventor = Michael E. Tipping
}}</ref>

== See also ==
* [Kernel trick](/source/Kernel_trick)
* [Platt scaling](/source/Platt_scaling): turns an SVM into a probability model

== References ==
{{reflist}}

Category:Classification algorithms
Category:Kernel methods for machine learning
Category:Nonparametric Bayesian statistics

---
Adapted from the Wikipedia article [Relevance vector machine](https://en.wikipedia.org/wiki/Relevance_vector_machine) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/Relevance_vector_machine?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.