# Graphical lasso

> Mediated Wiki article. Canonical URL: https://mediated.wiki/source/Graphical_lasso
> Markdown URL: https://mediated.wiki/source/Graphical_lasso.md
> Source: https://en.wikipedia.org/wiki/Graphical_lasso
> Source revision: 1344941162
> License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/)

In statistics, the **graphical lasso**[1] is a penalized likelihood [estimator](/source/Estimator) for the [precision matrix](/source/Precision_(statistics)) (also called the concentration matrix or inverse [covariance matrix](/source/Covariance_matrix)) of a multivariate [elliptical distribution](/source/Elliptical_distribution). Through the use of an L_1 penalty, it performs [regularization](/source/Regularization_(mathematics)) to give a sparse estimate for the precision matrix. In the case of [multivariate Gaussian distributions](/source/Multivariate_Gaussian_distribution), sparsity in the precision matrix corresponds to [conditional independence](/source/Conditional_independence) between the variables therefore implying a Gaussian [graphical model](/source/Graphical_model).

The graphical lasso was originally formulated to solve Dempster's covariance selection problem[2][3] for the [multivariate Gaussian distribution](/source/Multivariate_Gaussian_distribution) when observations were limited. Subsequently, the optimization algorithms to solve this problem were improved[4] and extended[5] to other types of estimators and distributions.

## Setting

Let S be the [sample covariance matrix](/source/Sample_mean_and_covariance#Definition_of_sample_covariance) of an [independent identically distributed](/source/Independent_and_identically_distributed_random_variables) sample from a multivariate [Gaussian distribution](/source/Gaussian_distribution) X \sim N(\mu, \Sigma). We are interested in estimating the [precision matrix](/source/Precision_matrix) \Sigma^{-1} = \Theta = (\Theta_{ij}).

The graphical lasso estimator \hat{\Theta} is the maximiser of the L_1 penalised [log-likelihood](/source/Likelihood_function#Log-likelihood):

- \hat{\Theta} = \operatorname{argmax}_{\Theta \succ 0} \left(\log \det(\Theta) - \operatorname{tr}(S \Theta) - \lambda \sum_{i,j} |\Theta_{ij}| \right)

where \lambda is a penalty parameter,[4] \operatorname{tr} is the [trace](/source/Trace_(linear_algebra)) function and \Theta \succ 0 refers to the set of [positive definite matrices](/source/Definite_matrix).

A popular alternative form of the graphical lasso removes the penalty on the diagonal, only penalising the off-diagonal entries:[6]

- \hat{\Theta} = \operatorname{argmax}_{\Theta \succ 0} \left(\log \det(\Theta) - \operatorname{tr}(S \Theta) - \lambda \sum_{i \neq j} |\Theta_{ij}| \right)

Because the graphical lasso estimate is not invariant to scalar multiplication of the variables,[7] it is important to [normalize](/source/Normalization_(statistics)) the data before applying the graphical lasso.

## Application

To obtain the estimator in programs, users could use the R package [glasso](https://cran.r-project.org/web/packages/glasso/glasso.pdf),[8] [GraphicalLasso() class](https://scikit-learn.org/stable/modules/generated/sklearn.covariance.GraphicalLasso.html) in the [scikit-learn](/source/Scikit-learn) [Python](/source/Python_(programming_language)) library,[9] or the [skggm](https://github.com/skggm/skggm) Python package[10] (similar to scikit-learn).

## See also

- [Graphical model](/source/Graphical_model)
- [Lasso (statistics)](/source/Lasso_(statistics))

## References

1. Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert (2008-07-01). "Sparse inverse covariance estimation with the graphical lasso". *Biostatistics*. **9** (3): 432–441. [doi:10.1093/biostatistics/kxm045](https://doi.org/10.1093/biostatistics/kxm045). [ISSN 1465-4644](https://www.worldcat.org/issn/1465-4644). [PMC 3019769](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3019769). [PMID 18079126](https://pubmed.ncbi.nlm.nih.gov/18079126)

1. Dempster, A. P. (1972). "Covariance Selection". *Biometrics*. **28** (1): 157–175. [doi:10.2307/2528966](https://doi.org/10.2307/2528966). [ISSN 0006-341X](https://www.worldcat.org/issn/0006-341X). [JSTOR 2528966](https://www.jstor.org/stable/2528966)

1. Banerjee, Onureena; d'Aspremont, Alexandre; Ghaoui, Laurent El (2005-06-08). "Sparse Covariance Selection via Robust Maximum Likelihood Estimation". [arXiv:cs/0506023](https://arxiv.org/abs/cs/0506023)

1. Friedman, Jerome and Hastie, Trevor and Tibshirani, Robert (2008). ["Sparse inverse covariance estimation with the graphical lasso"](http://statweb.stanford.edu/~tibs/ftp/graph.pdf). *Biostatistics*. **9** (3): 432–41. Biometrika Trust. [doi:10.1093/biostatistics/kxm045](https://doi.org/10.1093/biostatistics/kxm045). [PMC 3019769](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3019769). [PMID 18079126](https://pubmed.ncbi.nlm.nih.gov/18079126)

1. Cai, T. Tony; Liu, Weidong; Zhou, Harrison H. (April 2016). "Estimating sparse precision matrix: Optimal rates of convergence and adaptive estimation". *The Annals of Statistics*. **44** (2): 455–488. [arXiv:1212.2882](https://arxiv.org/abs/1212.2882). [doi:10.1214/13-AOS1171](https://doi.org/10.1214/13-AOS1171). [ISSN 0090-5364](https://www.worldcat.org/issn/0090-5364). [S2CID 14699773](https://api.semanticscholar.org/CorpusID:14699773)

1. Yuan, Ming & Lin, Yi (2007). ["Model selection and estimation in the Gaussian graphical model"](https://academic.oup.com/biomet/article/94/1/19/228821). *Biometrika*. **94** (1): 19–35. [doi:10.1093/biomet/asm018](https://doi.org/10.1093/biomet/asm018)

1. Carter, Jack Storror; Rossell, David; Smith, Jim Q. (2024). ["Partial correlation graphical lasso"](https://onlinelibrary.wiley.com/doi/full/10.1111/sjos.12675). *Scandinavian Journal of Statistics*. **51** (1): 32–63. [doi:10.1111/sjos.12675](https://doi.org/10.1111/sjos.12675)

1. Jerome Friedman; Trevor Hastie; Rob Tibshirani (2014). [*glasso: Graphical lasso- estimation of Gaussian graphical models*](https://cran.r-project.org/package=glasso)

1. Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E. (2011). ["Scikit-learn: Machine Learning in Python"](http://scikit-learn.org/stable/about.html). *Journal of Machine Learning Research*. **12**: 2825. [arXiv:1201.0490](https://arxiv.org/abs/1201.0490). [Bibcode:2011JMLR...12.2825P](https://ui.adsabs.harvard.edu/abs/2011JMLR...12.2825P)

1. Jason Laska & Manjari Narayan (2017). "skggm 0.2.7: A scikit-learn compatible package for Gaussian and related Graphical Models". *Zenodo*. [Bibcode:2017zndo....830033L](https://ui.adsabs.harvard.edu/abs/2017zndo....830033L). [doi:10.5281/zenodo.830033](https://doi.org/10.5281/zenodo.830033)

---
Adapted from the Wikipedia article [Graphical lasso](https://en.wikipedia.org/wiki/Graphical_lasso) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/Graphical_lasso?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.