{{Short description|Use of speech recognition to verify pronunciation}} {{Redirect-distinguish|Speech verification|speaker verification}}
Automatic '''pronunciation assessment''' uses computer speech recognition to determine how accurately speech has been pronounced,<ref name="El Kheir">{{citation |last1=El Kheir |first1=Yassine |last2=Ali |first2=Ahmed |display-authors=1 |title=Automatic Pronunciation Assessment — A Review |publisher=Conference on Empirical Methods in Natural Language Processing |date=October 2023 |arxiv=2310.13974 |s2cid=264426545 }}</ref><ref>{{cite journal |last1=Lounis |first1=Meriem |last2=Dendani |first2=Bilal |last3=Bahi |first3=Halima |title=Mispronunciation detection and diagnosis using deep neural networks: a systematic review |journal=Multimedia Tools and Applications |date=January 2024 |volume=83 |issue=23 |pages=62793–62827 |doi=10.1007/s11042-023-17899-x |url=https://www.researchgate.net/publication/377264348 |access-date=12 July 2025}}</ref> instead of relying on a human instructor or proctor.<ref name="Isaacs">{{cite journal |last1=Isaacs |first1=Talia |last2=Harding |first2=Luke |title=Pronunciation assessment |journal=Language Teaching |date=July 2017 |volume=50 |issue=3 |pages=347–366 |doi=10.1017/S0261444817000118 |s2cid=209353525 |language=en |issn=0261-4448|doi-access=free }}</ref> It is also called '''speech verification''', '''pronunciation evaluation''', and '''pronunciation scoring'''.<ref name="EhsaniKnodt1998">{{cite journal |last1=Ehsani |first1=Farzad |last2=Knodt |first2=Eva |title=Speech technology in computer-aided language learning: Strengths and limitations of a new CALL paradigm |journal=Language Learning & Technology |date=July 1998 |volume=2 |issue=1 |pages=54–73 |url=https://www.lltjournal.org/item/10125-25032/ |access-date=11 February 2023 |publisher=University of Hawaii National Foreign Language Resource Center; Michigan State University Center for Language Education and Research |doi=10.64152/10125/25032 |language=en|doi-access=free |hdl=10125/25032 |hdl-access=free }}</ref> This technology is used to grade speech quality, for language testing,<ref>{{cite journal |last1=Jones |first1=Daniel Marc |last2=Cheng |first2=Liying |last3=Tweedie |first3=M. Gregory |title=Automated Scoring of Speaking and Writing: Starting to Hit its Stride |journal=Canadian Journal of Learning and Technology |volume=48 |issue=3 |year=2022 |pages=1–22 |doi=10.21432/cjlt28241 |url=https://cjlt.ca/index.php/cjlt/article/view/28241|doi-access=free }}</ref> for '''computer-aided pronunciation teaching''' ('''CAPT''') in computer-assisted language learning (CALL), for speaking skill remediation, and for accent reduction.<ref name="EhsaniKnodt1998"/>
Pronunciation assessment is different from dictation or automatic transcription, because instead of determining unknown speech, it verifies learners' pronunciation of known word(s), often from prior transcription of the same utterance; ideally scoring the intelligibility of the learners' speech.<ref name=Loukina2015>{{cite conference |last1=Loukina |first1=Anastassia |last2=Lopez |first2=Melissa |display-authors=1 |title=Pronunciation accuracy and intelligibility of non-native speech |conference=Interspeech 2015 |pages=1917–1921 |publisher=ISCA |location=Dresden, Germany |date=September 2015 |url=https://www.isca-archive.org/interspeech_2015/loukina15_interspeech.html |quote=only 16% of the variability in word-level intelligibility can be explained by the presence of obvious mispronunciations.}}</ref><ref name="obrien">{{cite journal |last1=O’Brien |first1=Mary Grantham |last2=Derwing |first2=Tracey M. |display-authors=1 |title=Directions for the future of technology in pronunciation research and teaching |journal=Journal of Second Language Pronunciation |date=December 2018 |volume=4 |issue=2 |pages=182–207 |doi=10.1075/jslp.17001.obr |s2cid=86440885 |language=en |issn=2215-1931 |quote=pronunciation researchers are primarily interested in improving L2 learners’ intelligibility and comprehensibility, but they have not yet collected sufficient amounts of representative and reliable data (speech recordings with corresponding annotations and judgments) indicating which errors affect these speech dimensions and which do not. These data are essential to train ASR algorithms to assess L2 learners’ intelligibility.|doi-access=free |hdl=2066/199273 |hdl-access=free }}</ref> Sometimes pronunciation assessment evaluates the prosody of the learners' speech, such as intonation, pitch, tempo, rhythm, and syllable and word stress, although those are usually not essential for being understood in most languages.<ref name="Eskenazi">{{cite journal |last1=Eskenazi |first1=Maxine |title=Using automatic speech processing for foreign language pronunciation tutoring: Some issues and a prototype |journal=Language Learning & Technology |date=January 1999 |volume=2 |issue=2 |pages=62–76 |doi=10.64152/10125/25043 |url=https://www.lltjournal.org/item/10125-25043/ |access-date=11 February 2023 |language=en|doi-access=free |hdl=10125/25043 |hdl-access=free }}</ref> Pronunciation assessment is also used in reading tutoring, for example in products from Google,<ref>{{cite web |title=Read Along |url=https://readalong.google.com/ |website=Google.com |date=August 2022 |access-date=29 September 2025}}</ref> Microsoft,<ref>{{cite web |title=Reading Coach |url=https://coach.microsoft.com/ |website=Microsoft.com |date=March 2025 |access-date=29 September 2025 }}</ref><ref>{{cite news |last1=Tholfsen |first1=Mike |title=Reading Coach in Immersive Reader plus new features coming to Reading Progress in Microsoft Teams |url=https://techcommunity.microsoft.com/t5/education-blog/reading-coach-in-immersive-reader-plus-new-features-coming-to/ba-p/3734079 |access-date=12 February 2023 |work=Techcommunity Education Blog |publisher=Microsoft |date=February 2023 |language=en}}</ref> and Amira Learning.<ref>{{cite news |last1=Banerji |first1=Olina |title=Schools Are Using Voice Technology to Teach Reading. Is It Helping? |url=https://www.edsurge.com/news/2023-03-07-schools-are-using-voice-technology-to-teach-reading-is-it-helping |access-date=7 March 2023 |work=EdSurge News |date=March 2023 |language=en}}</ref> Automatic pronunciation assessment can also be used to help diagnose and treat speech disorders such as apraxia.<ref>{{cite conference |last1=Hair |first1=Adam |last2=Monroe |first2=Penelope |conference=Proceedings of the 17th ACM Conference on Interaction Design and Children |title=Apraxia world: A speech therapy game for children with speech sound disorders |date=June 2018 |pages=119–131 |doi=10.1145/3202185.3202733 |isbn=9781450351522 |s2cid=13790002 |url=https://psi.engr.tamu.edu/wp-content/uploads/2018/04/hair2018idc.pdf }}</ref><ref>{{cite journal |last1=Kim |first1=Do Hyung |last2=Jeong |first2=Joo Won |display-authors=1 |title=Usefulness of Automatic Speech Recognition Assessment of Children With Speech Sound Disorders: Validation Study |journal=Journal of Medical Internet Research |date=14 January 2025 |volume=27 |issue=1 |article-number=e60520 |doi=10.2196/60520 |language=EN |doi-access=free|pmid=39576242 |pmc=11775490 }}</ref>
== Intelligibility ==
Intelligibility refers to how well a learner's utterance is understood by a listener, rather than how much it sounds like a native speaker.<ref name="obrien"/> This is separate from measures of fluency, such as so-called "Goodness of Pronunciation" (GoP) scores, which estimate how closely an utterance aligns with those of native speakers.<ref>{{cite thesis |last=Kyriakopoulos |first=Konstantinos |title=Automatic Assessment of English as a Second Language |url=https://mi.eng.cam.ac.uk/foswiki/pub/Main/KK492/IIB_kk492.pdf |publisher=University of Cambridge, Department of Engineering |date=May 2016 |type=IIB Project Final Report |pages=9–10 |location=Cambridge, United Kingdom }}</ref> Intelligibility is widely regarded as the most important communicative goal in pronunciation teaching and assessment.<ref>{{cite journal |last1=Levis |first1=John |title=Revisiting the Intelligibility and Nativeness Principles |journal=Journal of Second Language Pronunciation |date=November 2020 |volume=6 |issue=3 |pages=310–328 |doi=10.1075/jslp.20050.lev |url=https://homepage.ntu.edu.tw/~karchung/jslp.20050.levNative.pdf |access-date=18 October 2025}}</ref> For example, in the Common European Framework of Reference for Languages (CEFR) assessment criteria for "overall phonological control", intelligibility outweighs formally correct pronunciation at all levels.<ref>{{Cite book |url=https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989 |title=Common European framework of reference for languages learning, teaching, assessment: Companion volume with new descriptors |date=February 2018 |publisher=Language Policy Programme, Education Policy Division, Education Department, Council of Europe |oclc=1090351600 |page=136}}</ref>
Studies in applied linguistics have shown that accent reduction does not always increase intelligibility because listeners can often comprehend heavily accented speech without difficulty.<ref name=Loukina2015/> Pronunciation assessment systems often rely on acoustic methods such as GoP which compare learner speech to reference models to produce phoneme-level scores, which are in turn aggregated to produce word and phrase scores.<ref name="El Kheir"/> While these methods are effective for identifying deviations from native speakers' utterances, they do not effectively measure how understandable speech is to human listeners.<ref name="obrien"/> Intelligibility is influenced by broader linguistic and contextual factors such as stress placement, speech rate, and coarticulation, which are not represented in purely segmental scores.<ref name="Eskenazi"/><ref name="Isaacs"/>
The earliest work on pronunciation assessment avoided measuring genuine listener intelligibility,<ref>{{citation |last1=Bernstein |first1=Jared |last2=Cohen |first2=Michael |display-authors=1 |contribution=Automatic Evaluation and Training in English Pronunciation |title=First International Conference on Spoken Language Processing (ICSLP 90) |date=November 1990 |pages=1185–1188 |contribution-url=https://www.isca-archive.org/icslp_1990/bernstein90_icslp.html |access-date=11 February 2023 |publisher=International Speech Communication Association |location=Kobe, Japan |quote=listeners differ considerably in their ability to predict unintelligible words.... Thus, it seems the quality rating is a more desirable... automatic-grading score.}} (Section 2.2.2.)</ref> a shortcoming corrected in 2011 at the Toyohashi University of Technology,<ref>{{cite conference |last1=Hiroshi |first1=Kibishi |last2=Nakagawa |first2=Seiichi |title=New feature parameters for pronunciation evaluation in English presentations at international conferences |conference=Interspeech 2011 |pages=1149–1152 |publisher=ISCA |location=Florence, Italy |date=August 2011 |url=https://www.isca-archive.org/interspeech_2011/kibishi11_interspeech.html |access-date=11 February 2023 |quote=we investigated the relationship between pronunciation score / intelligibility and various acoustic measures, and then combined these measures.... As far as we know, the automatic estimation of intelligibility has not yet been studied.}}</ref> and included in the Versant high-stakes English fluency assessment from Pearson<ref>{{cite web |last1=Bonk |first1=Bill |title=New innovations in assessment: Versant's Intelligibility Index score |url=https://www.english.com/blog/intelligibility-index-score-versant/ |website=Resources for English Language Learners and Teachers |publisher=Pearson English |access-date=11 February 2023 |date=August 2020 |archive-url=https://web.archive.org/web/20230127122339/https://www.english.com/blog/intelligibility-index-score-versant/ |archive-date=2023-01-27 |quote=you don’t need a perfect accent, grammar, or vocabulary to be understandable. In reality, you just need to be understandable with little effort by listeners.}}</ref> and mobile apps from 17zuoye Education & Technology,<ref>{{cite book |last1=Gao |first1=Yuan |last2=Srivastava |first2=Mohan Lal Srivastava |display-authors=1 |contribution=Spoken English Intelligibility Remediation with PocketSphinx Alignment and Feature Extraction Improves Substantially over the State of the Art |date=May 2018 |title=2nd IEEE Advanced Information Management, Communication, Electronic and Automation Control Conference (IMCEC 2018) |pages=924–927 |doi=10.1109/IMCEC.2018.8469649 |arxiv=1709.01713 |isbn=978-1-5386-1803-5 |s2cid=31125681 }}</ref> but still missing in 2023 products from Google Search,<ref name="Snir2019">{{cite web |last1=Snir |first1=Tal |title=How do you pronounce quokka? Practice with Search |url=https://blog.google/products/search/how-do-you-pronounce-quokka-practice-search/ |website=The Keyword |publisher=Google |access-date=11 February 2023 |language=en-us |date=November 2019}}</ref> Microsoft,<ref>{{cite web |title=Pronunciation assessment tool |url=https://speech.microsoft.com/portal/pronunciationassessmenttool |website=Azure Cognitive Services Speech Studio |publisher=Microsoft |access-date=11 February 2023}}</ref> Educational Testing Service,<ref>{{cite book |last1=Chen |first1=Lei |last2=Zechner |first2=Klaus |display-authors=1 |title=Automated Scoring of Nonnative Speech: Using the SpeechRater v. 5.0 Engine |date=December 2018 |publisher=Educational Testing Service |location=Princeton, NJ |url=https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12198 |doi=10.1002/ets2.12198 |series=ETS Research Report Series |volume=2018 |issue=1 |pages=1–31 |s2cid=69925114 |issn=2330-8516 |access-date=11 February 2023 |language=en}}</ref> Speechace,<ref>{{citation |last1=Alnafisah |first1=Mutleb |contribution=Technology Review: Speechace |series=no. 40 |title=Proceedings of the 12th Pronunciation in Second Language Learning and Teaching Conference (Virtual PSLLT) |publisher=Iowa State University Digital Press |location=St. Catharines, Ontario |date=September 2022 |volume=12 |contribution-url=https://www.iastatedigitalpress.com/psllt/article/id/14315/ |issn=2380-9566 |access-date=14 February 2023}}</ref> and ELSA.<ref>{{Cite AV media |last1=Gorham |first1=Jon |last2=Raine |first2=Paul |display-authors=1 |title=Speech Recognition for English Language Learning |url=https://www.youtube.com/watch?v=SdC16s-GEEo&t=1325s |time=22:05 |date=March 2022 |publisher=Education Solutions |work=Technology in Language Teaching and Learning |format=video |language=en |access-date=2023-02-14}}</ref> Assessing authentic listener intelligibility is essential for avoiding inaccuracies from accent bias,<ref name=Loukina2015/> especially in high-stakes assessments;<ref>{{cite news |title=Computer says no: Irish vet fails oral English test needed to stay in Australia |url=https://www.theguardian.com/australia-news/2017/aug/08/computer-says-no-irish-vet-fails-oral-english-test-needed-to-stay-in-australia |access-date=12 February 2023 |agency=Australian Associated Press |work=The Guardian |date=8 August 2017}}</ref><ref>{{cite news |last1=Ferrier |first1=Tracey |title=Australian ex-news reader with English degree fails robot's English test |url=https://www.smh.com.au/technology/australian-exnews-reader-with-english-degree-fails-robots-english-test-20170809-gxsjv2.html |access-date=12 February 2023 |work=The Sydney Morning Herald |date=9 August 2017 |language=en}}</ref><ref>{{cite news |last1=Main |first1=Ed |last2=Watson |first2=Richard |title=The English test that ruined thousands of lives |url=https://www.bbc.com/news/uk-60264106 |access-date=12 February 2023 |work=BBC News |date=February 2022}}</ref> from words with multiple correct pronunciations;<ref>{{cite web |last1=Joyce |first1=Katy Spratte |title=13 Words That Can Be Pronounced Two Ways |url=https://www.rd.com/list/words-that-can-be-pronounced-two-ways/ |publisher=Reader's Digest |access-date=23 February 2023 |date=January 2023}}</ref> and from phoneme coding errors in machine-readable pronunciation dictionaries.<ref>E.g., CMUDICT, {{cite web |title=The CMU Pronouncing Dictionary |url=http://www.speech.cs.cmu.edu/cgi-bin/cmudict |website=www.speech.cs.cmu.edu |access-date=15 February 2023}} Compare "four" given as "F AO R" with the vowel AO as in "caught," to "row" given as "R OW" with the vowel OW as in "oat." This mistake is due to the "horse–hoarse merger," often called the "north–force merger."</ref>
In 2022, researchers found that some newer speech-to-text systems, based on end-to-end reinforcement learning to map audio signals directly into words, produce word and phrase confidence scores (from 10-25ms audio frame logit aggregation) closely correlated with genuine listener intelligibility.<ref>{{cite conference |last1=Tu |first1=Zehai |last2=Ma |first2=Ning |last3=Barker |first3=Jon |title=Unsupervised Uncertainty Measures of Automatic Speech Recognition for Non-intrusive Speech Intelligibility Prediction |conference=Interspeech 2022 |pages=3493–3497 |publisher=ISCA |date=2022 |doi=10.21437/Interspeech.2022-10408 |url=https://www.isca-archive.org/interspeech_2022/tu22b_interspeech.html |access-date=17 December 2023}}</ref><ref>{{cite conference |last1=Kuhn |first1=Korbinian |last2=Kersken |first2=Verena |last3=Zimmermann |first3=Gottfried |title=Evaluating ASR Confidence Scores for Automated Error Detection in User-Assisted Correction Interfaces |date=April 2025 |pages=1–7 |book-title=Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25) |volume=229 |doi=10.1145/3706599.3720038 |arxiv=2503.15124 }} See Table 1, column 5, "𝑟<sub>𝑝𝑏</sub>" and section 4.1.3.</ref> Others have been able to assess intelligibility using Levenshtein or dynamic time warping distance measures from Wav2Vec2 representation of good speech.<ref>{{cite conference |last1=Yang |first1=Mu |last2=Hirschi |first2=Kevin |display-authors=1 |title=Interspeech 2022 |chapter=Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment |date=2022 |volume=Interspeech 2022 |pages=4481–4485 |doi=10.21437/Interspeech.2022-11039 |chapter-url=https://www.isca-archive.org/interspeech_2022/yang22v_interspeech.html |publisher=ISCA}}</ref><ref>{{cite arXiv |last1=Anand |first1=Nayan |last2=Sirigiraju |first2=Meenakshi |last3=Yarra |first3=Chiranjeevi |title=Unsupervised speech intelligibility assessment with utterance level alignment distance between teacher and learner Wav2Vec-2.0 representations |date=June 2023 |class=cs.SD |eprint=2306.08845}}</ref> Further work through 2025 has focused specifically on measuring intelligibility.<ref>{{cite conference |title=A Perception-Based L2 Speech Intelligibility Indicator: Leveraging a Rater's Shadowing and Sequence-to-sequence Voice Conversion |last1=Geng |first1=Haopeng |last2=Saito |first2=Daisuke |last3=Minematsu |first3=Nobuaki |conference=Interspeech 2025 |pages=2420–2424 |location=Rotterdam, The Netherlands |publisher=ISCA |date=August 2025 |url=https://www.isca-archive.org/interspeech_2025/geng25_interspeech.html }}</ref><ref>{{cite conference |title=Aligning ASR Evaluation with Human and LLM Judgments: Intelligibility Metrics Using Phonetic, Semantic, and NLI Approaches |last1=Phukon |first1=Bornali |last2=Zheng |first2=Xiuwen |last3=Hasegawa-Johnson |first3=Mark |conference=Interspeech 2025 |pages=5708–5712 |location=Rotterdam, The Netherlands |publisher=ISCA |date=August 2025 |url=https://www.isca-archive.org/interspeech_2025/phukon25_interspeech.html }}</ref>
A 2025 study of 42 pronunciation and speech coaching apps (32 mobile and 10 web) found that none offered intelligibility assessment. Instead, most provided only segmental and accent-focused scoring. About two-thirds of the apps provided some form of specific pronunciation feedback, usually with phonetic transcriptions, but accompanied by visual cues (such as animations of the vocal tract or the lips and tongue from the front) in only about 5% of the apps. Less than a third provided feedback on learner perception of exemplar speech.<ref>{{cite journal |last1=Walesiak |first1=Beata |last2=Talley |first2=Jim |title=Feedback Mechanisms in Pronunciation and Speech Coaching Apps |journal=Pronunciation in Second Language Learning and Teaching Proceedings |date=15 September 2025 |volume=15 |issue=1 |doi=10.31274/psllt.18444 |url=https://www.iastatedigitalpress.com/psllt/article/id/18444/ |access-date=25 September 2025 |issn=2380-9566}}</ref>
== Evaluation ==
Although there are as yet no industry-standard benchmarks for evaluating pronunciation assessment accuracy, researchers occasionally release evaluation speech corpuses for others to use for improving assessment quality.<ref>{{cite conference |last1=Zhang |first1=Junbo |last2=Zhang |first2=Zhiwen |display-authors=1 |title=speechocean762: An Open-Source Non-Native English Speech Corpus for Pronunciation Assessment |conference=Interspeech 2021 |date=August 2021 |pages=3710–3714 |publisher=ISCA |doi=10.21437/Interspeech.2021-1259 |arxiv=2104.01378 |s2cid=233025050 |url=https://www.isca-archive.org/interspeech_2021/zhang21x_interspeech.html |access-date=19 February 2023}}; [https://github.com/jimbozhang/speechocean762 GitHub corpus repository.]</ref><ref>{{cite conference |last1=Vidal |first1=Jazmín |last2=Ferrer |first2=Luciana |display-authors=1 |title=EpaDB: A Database for Development of Pronunciation Assessment Systems |conference=Interspeech 2019 |date=September 2019 |pages=589–593 |publisher=ISCA |doi=10.21437/Interspeech.2019-1839 |hdl=11336/161618 |s2cid=202742421 |url=https://www.isca-archive.org/interspeech_2019/vidal19_interspeech.html |access-date=19 February 2023|hdl-access=free }}; [https://drive.google.com/file/d/1zEFNnyp8gmgJbOuSUwh1avccAw00FN5z/view database .zip file.]</ref><ref>{{cite conference |last1=Menzel |first1=Wolfgang |last2=Atwell |first2=Eric |last3=Bonaventura |first3=Patrizia |last4=Herron |first4=Daniel |last5=Howarth |first5=Peter |last6=Morton |first6=Rachel |last7=Souter |first7=Clive |display-authors=1 |title=The ISLE Corpus of Non-Native Spoken English |conference=Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00) |publisher=European Language Resources Association |location=Athens, Greece |date=May 2000 |url=https://aclanthology.org/L00-1234/ |access-date=13 August 2025}}</ref><ref>{{cite conference |last1=Zhao |first1=Guanlong |last2=Sonsaat |first2=Sinem |last3=Silpachai |first3=Alif |last4=Lucic |first4=Ivana |last5=Chukharev-Hudilainen |first5=Evgeny |last6=Levis |first6=John |last7=Gutierrez-Osuna |first7=Ricardo |display-authors=1 |title=L2-ARCTIC: A Non-native English Speech Corpus |conference=Interspeech 2018 |publisher=ISCA |year=2018 |pages=2783–2787 |doi=10.21437/Interspeech.2018-1110 |url=https://www.isca-archive.org/interspeech_2018/zhao18b_interspeech.html |access-date=13 August 2025}}</ref> Such evaluation databases often emphasize formally unaccented pronunciation to the exclusion of genuine intelligibility evident from blinded listener transcriptions.<ref name="obrien" /> As of mid-2025, state of the art approaches for automatically transcribing phonemes typically achieve an error rate of about 10% from known good speech.<ref>{{cite conference |title=Towards Accurate Phonetic Error Detection Through Phoneme Similarity Modeling |last1=Zhou |first1=Xuanru |last2=Lian |first2=Jiachen |last3=Cho |first3=Cheol Jun |last4=Prabhune |first4=Tejas |last5=Li |first5=Shuhe |last6=Li |first6=William |last7=Ortiz |first7=Rodrigo |last8=Ezzes |first8=Zoe |last9=Vonk |first9=Jet |last10=Morin |first10=Brittany |last11=Bogley |first11=Rian |last12=Wauters |first12=Lisa |last13=Miller |first13=Zachary |last14=Gorno-Tempini |first14=Maria |last15=Anumanchipalli |first15=Gopala |display-authors=1 |conference=Interspeech 2025|pages=4738–4742 |location=Rotterdam, The Netherlands |publisher=ISCA |date=August 2025 |url=https://www.isca-archive.org/interspeech_2025/zhou25h_interspeech.html }}</ref><ref>{{cite arXiv |last=Alon |first=Yonatan |date=March 2021 |title=Real-time low-resource phoneme recognition on edge devices |eprint=2103.13997 |class=cs.CL}}</ref><ref>{{cite web |last1=Yeo |first1=Eunjung |title=wav2vec2-large-english-TIMIT-phoneme_v3 |date=October 2022 |url=https://huggingface.co/speech31/wav2vec2-large-english-TIMIT-phoneme_v3 |website=huggingface.co |publisher=Seoul National University Spoken Language Processing Lab |access-date=19 August 2025}}</ref><ref>{{cite web |last1=Lee |first1=Jooyoung |title=wav2vec2-large-lv60_phoneme-timit_english_timit-4k |date=June 2024 |url=https://huggingface.co/excalibur12/wav2vec2-large-lv60_phoneme-timit_english_timit-4k |website=huggingface.co |publisher=Seoul National University Spoken Language Processing Lab |access-date=19 August 2025}}</ref>
The International Speech Communication Association (ISCA) 2025 Workshop on Speech and Language Technology in Education (SLaTE) administered a ''Speak & Improve Challenge: Spoken Language Assessment and Feedback,'' introducing benchmarks for evaluating pronunciation assessment and remediation systems across languages, accents, and learner populations. The challenge emphasized cross-lingual generalization and alignment with human intelligibility judgments, for more robust and interpretable assessment systems.<ref>{{cite conference |title=Speak & Improve Challenge 2025 |last1=Qian |first1=Mengjie |last2=Knill |first2=Kate M. |display-authors=1 |conference=Proceedings of the 3rd Workshop on Speech and Language Technology in Education (SLaTE 2025) |date=August 2025 |publisher=ISCA |location=Leuven, Belgium |url=https://www.isca-archive.org/slate_2025/qian25_slate.html |access-date=4 October 2025}}</ref>
Ethical issues in pronunciation assessment are present in both human and automatic methods. Authentic validity, fairness, and mitigating bias in evaluation are all crucial. Diverse speech data should be included in automatic pronunciation assessment models. Combining human judgments, especially blinded transcriptions from a wide diversity of listeners, with automated feedback can improve accuracy and fairness.<ref>{{cite journal |last=Babaeian |first=Ali |year=2023 |title=Pronunciation Assessment: Traditional vs Modern Modes |journal=Journal of Education for Sustainable Innovation |volume=1 |issue=1 |pages=61–68 |doi=10.56916/jesi.v1i1.530 |url=https://ejournal.papanda.org/index.php/jesi/article/view/530 |access-date=2024-12-31 |doi-access=free}}</ref>
Second language learners benefit substantially from their use of widely available speech recognition systems for dictation, virtual assistants, and AI chatbots. In such systems, users naturally try to correct their own errors evident in speech recognition results that they notice. Such use improves their grammar and vocabulary development along with their pronunciation skills. The extent to which explicit pronunciation assessment and remediation approaches improve on such self-directed interactions remains an open question.<ref>{{cite journal |last1=Akhter |first1=Elmoon |title=The Impact of Human-Machine Interaction on English Pronunciation and Fluency: Case Studies Using AI Speech Assistants |journal=Review of Applied Science and Technology |date=June 2025 |volume=4 |issue=2 |pages=473–500 |doi=10.63125/1wyj3p84 |doi-access=free}}</ref> Similarly, automatic dictation results have been shown to reflect intelligibility about as well as human scorers.<ref>{{cite journal |last1=Johnson |first1=Carol |last2=Cardoso |first2=Walcir |display-authors=1 |title=Assessing pronunciation using dictation tools: The use of Google Voice Typing to score a pronunciation placement test |journal=Journal of Second Language Pronunciation |date=July 2024 |volume=10 |issue=1 |pages=10–34 |doi=10.1075/jslp.23033.joh |url=https://www.researchgate.net/publication/381244671 |access-date=18 October 2025}}</ref><ref>{{cite journal |last1=Inceoglu |first1=Solène |last2=Chen |first2=Wen-Hsin |display-authors=1 |title=Assessment of L2 intelligibility: Comparing L1 listeners and automatic speech recognition |journal=ReCALL |date=January 2023 |volume=35 |issue=1 |pages=89–104 |doi=10.1017/S0958344022000192}}</ref>
== Recent developments ==
During 2021–22, a smartphone-based CAPT system was used to sense articulation through both audible and inaudible signals, providing feedback at the phoneme level.<ref>{{cite book |last1=Wong |first1=Aslan B. |last2=Chen |first2=Xia |last3=Liao |first3=Qianru |last4=Wu |first4=Kaishun |chapter=Articulation Motion Sensing for Pronunciation Training |title=2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON) |date=July 2021 |pages=1–2 |doi=10.1109/SECON52354.2021.9491610 |isbn=978-1-6654-4108-7 }}</ref><ref>{{cite journal |last1=B. Wong |first1=Aslan |last2=Huang |first2=ZiQi |last3=Wu |first3=Kaishun |title=Leveraging audible and inaudible signals for pronunciation training by sensing articulation through a smartphone |journal=Speech Communication |date=1 October 2022 |volume=144 |pages=42–56 |doi=10.1016/j.specom.2022.08.002 |url=https://www.sciencedirect.com/science/article/pii/S0167639322001066 |issn=0167-6393|url-access=subscription }}</ref>
Some promising areas for improvement which were being developed in 2024 include articulatory feature extraction<ref>{{cite arXiv |mode=cs2 |last1=Wu |first1=Peter |last2=Chen |first2=Li-Wei |display-authors=1 |title=Speaker-Independent Acoustic-to-Articulatory Speech Inversion |date=14 February 2023 |class=eess.AS |eprint=2302.06774 }}</ref><ref>{{cite arXiv |last1=Cho |first1=Cheol Jun |last2=Mohamed |first2=Abdelrahman |last3=Black |first3=Alan W. |last4=Anumanchipalli |first4=Gopala K. |display-authors=1 |title=Self-Supervised Models of Speech Infer Universal Articulatory Kinematics |date=January 2024 |class=eess.AS |eprint=2310.10788 |language=en}}</ref><ref>{{cite conference |last1=Mallela |first1=Jhansi |last2=Aluru |first2=Sai Harshitha |last3=Yarra |first3=Chiranjeevi |title=Exploring the Use of Self-Supervised Representations for Automatic Syllable Stress Detection |date=February 2024 |pages=1–6 |doi=10.1109/NCC60321.2024.10486028 |conference=National Conference on Communications |location=Chennai, India }}</ref> and transfer learning to suppress unnecessary corrections.<ref>{{cite book |last1=Sancinetti |first1=Marcelo |last2=Vidal |first2=Jazmin |title=ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) |chapter=A Transfer Learning Approach for Pronunciation Scoring |display-authors=1 |date=May 2022 |pages=6812–6816 |doi=10.1109/ICASSP43922.2022.9747727 |arxiv=2111.00976 |isbn=978-1-6654-0540-9 |s2cid=249437375 }}</ref> Other interesting advances under development include "augmented reality" interfaces for mobile devices using optical character recognition to provide pronunciation training on text found in user environments.<ref>{{cite journal |last1=Che Dalim |first1=Che Samihah |last2=Sunar |first2=Mohd Shahrizal |display-authors=1 |title=Using augmented reality with speech input for non-native children's language learning |journal=International Journal of Human-Computer Studies |date=February 2020 |volume=134 |pages=44–64 |doi=10.1016/j.ijhcs.2019.10.002 |s2cid=208098513 |url=https://core.ac.uk/download/pdf/233047951.pdf |access-date=28 February 2023}}</ref><ref>{{cite book |last1=Tolba |first1=Rahma M. |last2=Elarif |first2=Taha |display-authors=1 |title=Extended Reality and Metaverse |chapter=Mobile Augmented Reality for Learning Phonetics: A Review (2012–2022) |series=Springer Proceedings in Business and Economics |date=2023 |pages=87–98 |doi=10.1007/978-3-031-25390-4_7 |chapter-url=https://link.springer.com/chapter/10.1007/978-3-031-25390-4_7 |access-date=28 February 2023 |publisher=Springer International Publishing |isbn=978-3-031-25389-8 |language=en|chapter-url-access=subscription }}</ref>
In 2024, audio multimodal large language models were first described as assessing pronunciation.<ref>{{cite arXiv |last1=Fu |first1=Kaiqi |last2=Peng |first2=Linkai |last3=Yang |first3=Nan |last4=Zhou |first4=Shuran |display-authors=1 |title=Pronunciation Assessment with Multi-modal Large Language Models |date=July 2024 |class=cs.CL |eprint=2407.09209}} Note that [https://speak.com Speak.com] produced an earlier commercial system that they had not described in technical detail.</ref> That work has been carried forward by other researchers in 2025 who report positive results.<ref>{{cite arXiv |last1=Ma |first1=Rao |last2=Qian |first2=Mengjie |last3=Tang |first3=Siyuan |last4=Bannò |first4=Stefano |last5=Knill |first5=Kate M. |last6=Gales |first6=Mark J.F. |display-authors=1 |title=Assessment of L2 Oral Proficiency using Speech Large Language Models |eprint=2505.21148 |class=cs.CL |date=May 2025}}</ref> Subsequently, researchers demonstrated pronunciation scoring by providing a language model with textual descriptions of speech, including the speech-to-text transcript, phoneme sequences, pauses, and phoneme sequence matching; this approach can achieve performance similar to multimodal LLMs that analyze raw audio while avoiding their higher computational cost.<ref>{{cite arXiv |last1=Chen |first1=Hongjie |last2=Ma |first2=Wei |last3=Hirschberg |first3=Julia |display-authors=1 |title=TextPA: Pronunciation Assessment through Texts |eprint=2509.14187 |date=September 2025 |class=eess.AS }}</ref><ref>{{cite conference |title=Leveraging ASR and LLMs for Automated Scoring and Feedback in Children's Spoken Language Assessments |last1=Shankar |first1=Natarajan Balaji |last2=Zhang |first2=Kaiyuan |display-authors=1 |date=August 2025 |conference=10th Workshop on Speech and Language Technology in Education (SLaTE) |pages=1–5 |location=Nijmegen, Netherlands |publisher=ISCA |url=https://www.isca-archive.org/slate_2025/shankar25_slate.html}}</ref>
In 2025, the Duolingo English Test authors published a description of their pronunciation assessment method, purportedly built to measure intelligibility rather than accent imitation. While achieving a correlation of 0.82 with expert human ratings, very close to inter-rater agreement and outperforming alternative methods, the method is nonetheless based on experts' scores along the six-point CEFR common reference levels scale, instead of actual blinded listener transcriptions.<ref>{{cite journal |last1=Cai |first1=Danwei |last2=Naismith |first2=Ben |last3=Kostromitina |first3=Maria |last4=Teng |first4=Zhongwei |last5=Yancey |first5=Kevin P. |last6=LaFlair |first6=Geoffrey T. |display-authors=1 |date=July 2025 |title=Developing an Automatic Pronunciation Scorer: Aligning Speech Evaluation Models and Applied Linguistics Constructs |journal=Language Learning |volume=75 |pages=170–203 |doi=10.1111/lang.70000 |doi-access=free| quote="Proficiency [is] estimated by an ML classifier trained to predict the human CEFR rating of a speaking response"}}</ref>
Further promising work in 2025 includes assessment feedback aligning learner speech to synthetic utterances using interpretable features, identifying continuous spans of words for remediation feedback;<ref>{{cite conference |title=Comparative Pronunciation Assessment and Feedback with Interpretable Speech Features |last1=McGhee |first1=Charles |last2=Gales |first2=Mark J. F. |last3=Knill |first3=Kate M. |date=August 2025 |conference=10th Workshop on Speech and Language Technology in Education (SLaTE) |pages=36–40 |location=Nijmegen, Netherlands |publisher=ISCA |url=https://www.isca-archive.org/slate_2025/mcghee25_slate.html}}</ref> synthesizing corrected speech matching learners' self-perceived voices, which they prefer and imitate more accurately as corrections;<ref>{{cite conference |title=Synthesizing True Golden Voices to Enhance Pronunciation Training for Individual Language Learners |last1=Yamanaka |first1=Ryoga |last2=Osa |first2=Kento |last3=Fujiwara |first3=Akari |last4=Geng |first4=Haopeng |last5=Saito |first5=Daisuke |last6=Minematsu |first6=Nobuaki |last7=Inoue |first7=Yusuke |display-authors=1 |date=August 2025 |conference=10th Workshop on Speech and Language Technology in Education (SLaTE) |pages=209–213 |location=Nijmegen, Netherlands |publisher=ISCA |url=https://www.isca-archive.org/slate_2025/yamanaka25_slate.html}}</ref> and streaming such interactions.<ref>{{cite conference |title=Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement |last1=Nguyen |first1=Tuan-Nam |last2=Pham |first2=Ngoc-Quan |last3=Akti |first3=Seymanur |last4=Waibel |first4=Alexander |display-authors=1 |conference=Interspeech 2025 |pages=4163–4167 |location=Rotterdam, The Netherlands |publisher=ISCA |date=August 2025 |url=https://www.isca-archive.org/interspeech_2025/nguyen25c_interspeech.html }}</ref>
On January 21, 2026, Educational Testing Service's TOEFL iBT high-stakes English language test, required by US university admissions and employers from English as a foreign language applicants more often than all other internet-based tests combined, changed its speaking assessments.<ref>{{cite news |last1=Gupta |first1=Priyadarshini |title=TOEFL to ease student experience through adaptive testing and CEFR-aligned scoring scale |url=https://www.educationtimes.com/article/campus-beat-college-life/99739298/toefl-to-ease-student-experience-through-adaptive-testing-and-cefr-aligned-scoring-scale |access-date=22 November 2025 |work=www.educationtimes.com |agency=Education Times |publisher=The Times of India |date=August 11, 2025 |language=en}}</ref> While official rubrics claim that the new scoring will be based primarily on intelligibility,<ref>{{cite web |title=TOEFL iBT Speaking Scoring Guide |url=https://www.ets.org/pdfs/toefl/speaking-rubrics.pdf |website=ETS.org |publisher=Educational Testing Service |access-date=22 November 2025}}</ref> the new test's technical description indicates that it judges intelligibility by "correctness of pronunciation, naturalness of speech rhythm, and naturalness of prosody (e.g., syllable stress)" instead of agreement with blinded listener transcription.<ref>{{cite journal |last1=Manna |first1=Venessa |last2=Li |first2=Shuhong |last3=Papageorgiou |first3=Spiros |last4=Gu |first4=Lixiong |display-authors=1 |title=TOEFL iBT® Technical Manual |journal=ETS Research Report Series |date=October 2025 |volume=2025 |issue=1 |page=34 |doi=10.64634/eje8f497 |url=https://rr.ets.org/index.php/etsrr/article/view/28/17 |access-date=22 November 2025|doi-access=free }} See tables 5 and 6.</ref>
In April 2026, Google Search withdrew the assessment and remediation features of their pronunciation widget,<ref name="Snir2019" /> which now only plays audio and shows mouth position diagram animations without recording audio. And Google Translate added pronunciation assessment and remediation feedback features.<ref>{{cite news |last1=Mehta |first1=Ivan |title=Google Translate now lets you practice pronunciation |url=https://techcrunch.com/2026/04/29/google-translate-now-lets-you-practice-pronunciation/ |access-date=7 May 2026 |work=TechCrunch |date=29 April 2026}}</ref>
== In popular culture ==
The 2012 sci-fi horror film ''Prometheus'' shows android character David 8 learning to pronounce Proto-Indo-European phrases from a holographic virtual tutor.<ref>{{cite youtube |url=https://www.youtube.com/watch?v=ZTOcA_y1R_U&t=44s |title=Recitation of Schleicher's Fable in Proto-Indo-European from "Prometheus" [subtitled & translated] |time=0:44 |date=June 2012 |access-date=28 September 2025 }}</ref> Rosetta Stone, the first comprehensive language learning application with pronunciation assessment capabilities, was mentioned on ''Saturday Night Live''<ref>{{cite youtube |url=https://www.youtube.com/watch?v=ctDjnG8J9cY |title=Saturday Night Live: Rosetta Stone |publisher=NBCUniversal |date=February 2013 |access-date=28 September 2025}}</ref> and ''How I Met Your Mother''.<ref>{{cite web |title=How I Met Your Mother: 'The Rehearsal Dinner,' transcript of season 9, episode 12 |url=https://subslikescript.com/series/How_I_Met_Your_Mother-460649/season-9/episode-12-The_Rehearsal_Dinner |date=December 2013 |website=SubsLikeScript |access-date=28 September 2025}}</ref>
== See also == * Phonetics * Phonology * Speech segmentation — often called "forced alignment" (of audio to its expected phonemes) in this context<ref>{{cite conference |last1=Mathad |first1=Vikram C. |last2=Mahr |first2=Tristan J. |display-authors=1 |title=The Impact of Forced-Alignment Errors on Automatic Pronunciation Evaluation |conference=Interspeech 2021 |date=2021 |pages=176–180 |publisher=ISCA |doi=10.21437/interspeech.2021-1403 |url=https://www.isca-archive.org/interspeech_2021/mathad21_interspeech.html |access-date=10 March 2023}}</ref> * Statistical classification
== References == {{reflist|30em}}
== External links == * ISCA Special Interest Group on [https://sites.google.com/view/sigslate Speech and Language Technologies in Education (SLaTE)]
{{Natural language processing}}
Category:Educational technology Category:Language learning software Category:Natural language processing Category:Phonetics Category:Speech recognition Category:Statistical classification