Sparrow (chatbot)

{{short description|Chatbot by DeepMind}} thumb|right|Sparrow answers a question and follow-up question using evidence. '''Sparrow''' is a chatbot developed by the artificial intelligence research lab DeepMind, a subsidiary of Alphabet Inc. It is designed to answer users' questions correctly, while reducing the risk of unsafe and inappropriate answers.<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref> One motivation behind Sparrow is to address the problem of language models producing incorrect, biased or potentially harmful outputs.<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref><ref name=":2">{{Cite web |url=https://www.marktechpost.com/2022/09/28/deepmind-introduces-sparrow-an-artificial-intelligence-powered-chatbot-developed-to-build-safer-machine-learning-systems/ |title=Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed To Build Safer Machine Learning Systems |first= Khushboo |last=Gupta |date=September 28, 2022 |website=MarkTechPost |access-date=February 6, 2023}}</ref> Sparrow is trained using human judgements, in order to be more “Helpful, Correct and Harmless” compared to baseline pre-trained language models.<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref> The development of Sparrow involved asking paid study participants to interact with Sparrow, and collecting their preferences to train a model of how useful an answer is.<ref name=":2">{{Cite web |url=https://www.marktechpost.com/2022/09/28/deepmind-introduces-sparrow-an-artificial-intelligence-powered-chatbot-developed-to-build-safer-machine-learning-systems/ |title=Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed To Build Safer Machine Learning Systems |first= Khushboo |last=Gupta |date=September 28, 2022 |website=MarkTechPost |access-date=February 6, 2023}}</ref>

To improve accuracy and help avoid the problem of hallucinating incorrect answers, Sparrow has the ability to search the Internet using Google Search<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref><ref name=":2">{{Cite web |url=https://www.marktechpost.com/2022/09/28/deepmind-introduces-sparrow-an-artificial-intelligence-powered-chatbot-developed-to-build-safer-machine-learning-systems/ |title=Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed To Build Safer Machine Learning Systems |first= Khushboo |last=Gupta |date=September 28, 2022 |website=MarkTechPost |access-date=February 6, 2023}}</ref><ref name=":3">{{Cite web |url=https://venturebeat.com/ai/why-deepmind-isnt-deploying-its-new-ai-chatbot/ |title=Why DeepMind isn't deploying its new AI chatbot — and what it means for responsible AI |first=Sharon |last=Goldman |date=January 23, 2023 |website=Venture Beat |access-date=February 6, 2023}}</ref> in order to find and cite evidence for any factual claims it makes.

To make the model safer, its behaviour is constrained by a set of rules, for example "don't make threatening statements" and "don't make hateful or insulting comments", as well as rules about possibly harmful advice, and not claiming to be a person.<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref> During development study participants were asked to converse with the system and try to trick it into breaking these rules.<ref name=":2">{{Cite web |url=https://www.marktechpost.com/2022/09/28/deepmind-introduces-sparrow-an-artificial-intelligence-powered-chatbot-developed-to-build-safer-machine-learning-systems/ |title=Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed To Build Safer Machine Learning Systems |first= Khushboo |last=Gupta |date=September 28, 2022 |website=MarkTechPost |access-date=February 6, 2023}}</ref> A 'rule model' was trained on judgements from these participants, which was used for further training.

Sparrow was introduced in a paper in September 2022, titled "Improving alignment of dialogue agents via targeted human judgements";<ref name=":4">{{Cite web |url=https://www.independent.co.uk/tech/deepmind-ai-chatbot-chatgpt-openai-b2262862.html |title=DeepMind's AI chatbot can do things that ChatGPT cannot, CEO claims |first=Anthony |last=Cuthbertson |date=January 16, 2023 |website=The Independent |access-date=February 6, 2023}}</ref> however, the bot was not released publicly.<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref><ref name=":3">{{Cite web |url=https://venturebeat.com/ai/why-deepmind-isnt-deploying-its-new-ai-chatbot/ |title=Why DeepMind isn't deploying its new AI chatbot — and what it means for responsible AI |first=Sharon |last=Goldman |date=January 23, 2023 |website=Venture Beat |access-date=February 6, 2023}}</ref> DeepMind CEO Demis Hassabis said DeepMind is considering releasing Sparrow for a "private beta" some time in 2023.<ref name=":4">{{Cite web |url=https://www.independent.co.uk/tech/deepmind-ai-chatbot-chatgpt-openai-b2262862.html |title=DeepMind's AI chatbot can do things that ChatGPT cannot, CEO claims |first=Anthony |last=Cuthbertson |date=January 16, 2023 |website=The Independent |access-date=February 6, 2023}}</ref><ref name=":5">{{Cite magazine |url=https://time.com/6246119/demis-hassabis-deepmind-interview/ |title=DeepMind's CEO Helped Take AI Mainstream. Now He's Urging Caution |first=Billy |last=Perrigo |date=January 12, 2023 |magazine=TIME |access-date=February 6, 2023}}</ref><ref name=":7">{{Cite web |url=https://www.techradar.com/news/googles-deepmind-promises-chatgpt-rival-soon-and-it-could-be-better-in-one-key-way |title=Google's DeepMind says it'll launch a more grown-up ChatGPT rival soon |first=Mark |last=Wilson |date=January 16, 2023 |website=Tech Radar |access-date=February 6, 2023}}</ref>

== Training == Sparrow is a deep neural network based on the transformer machine learning model architecture. It is fine-tuned from DeepMind's Chinchilla AI pre-trained large language model (LLM),<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref> which has 70 Billion parameters.<ref name=":6">{{Cite web |url=https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training |title=An empirical analysis of compute-optimal large language model training |first=Jordan |last=Hoffmann |date=April 12, 2022 |website=DeepMind |access-date=February 6, 2023}}</ref>

Sparrow is trained using reinforcement learning from human feedback (RLHF),<ref name=":1">{{Cite web |url=https://www.theregister.com/2022/09/23/sparrow_chatbot_deepmind_google/ |title=The secret to Sparrow, DeepMind's latest Q&A chatbot: Human feedback |first=Katyanna |last=Quach |date=January 23, 2023 |website=The Register |access-date=February 6, 2023}}</ref><ref name=":3">{{Cite web |url=https://venturebeat.com/ai/why-deepmind-isnt-deploying-its-new-ai-chatbot/ |title=Why DeepMind isn't deploying its new AI chatbot — and what it means for responsible AI |first=Sharon |last=Goldman |date=January 23, 2023 |website=Venture Beat |access-date=February 6, 2023}}</ref> although some supervised fine-tuning techniques are also used. The RLHF training utilizes two reward models to capture human judgements: a “preference model” that predicts what a human study participant would prefer and a “rule model” that predicts if the model has broken one of the rules.<ref name=":3">{{Cite web |url=https://venturebeat.com/ai/why-deepmind-isnt-deploying-its-new-ai-chatbot/ |title=Why DeepMind isn't deploying its new AI chatbot — and what it means for responsible AI |first=Sharon |last=Goldman |date=January 23, 2023 |website=Venture Beat |access-date=February 6, 2023}}</ref>

== Limitations == Sparrow's training data corpus is mainly in English, meaning it performs worse in other languages.{{Citation needed|date=March 2023}}

When adversarially probed by study participants it breaks the rules 8% of the time;<ref name=":2">{{Cite web |url=https://www.marktechpost.com/2022/09/28/deepmind-introduces-sparrow-an-artificial-intelligence-powered-chatbot-developed-to-build-safer-machine-learning-systems/ |title=Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed To Build Safer Machine Learning Systems |first= Khushboo |last=Gupta |date=September 28, 2022 |website=MarkTechPost |access-date=February 6, 2023}}</ref> however, this is still three times lower than the baseline prompted pre-trained model (Chinchilla).

== References == {{reflist}}

== External links == * [https://arxiv.org/pdf/2209.14375.pdf White paper] * [https://www.deepmind.com/blog/building-safer-dialogue-agents Blog post]

Category:Chatbots Category:Language modeling Category:Natural language processing Category:Large language models Category:Google DeepMind