Speech accessibility project (Hasegawa-Johnson et al., 2024)
Purpose: The Speech Accessibility Project (SAP) intends to facilitate research and development in automatic speech recognition (ASR) and other machine learning tasks for people with speech disabilities. The purpose of this article is to introduce this project as a resource for researchers, including baseline analysis of the first released data package.
Method: The project aims to facilitate ASR research by collecting, curating, and distributing transcribed U.S. English speech from people with speech and/or language disabilities. Participants record speech from their place of residence by connecting their personal computer, cell phone, and assistive devices, if needed, to the SAP web portal. All samples are manually transcribed, and 30 per participant are annotated using differential diagnostic pattern dimensions. For purposes of ASR experiments, the participants have been randomly assigned to a training set, a development set for controlled testing of a trained ASR, and a test set to evaluate ASR error rate.
Results: The SAP 2023-10-05 Data Package contains the speech of 211 people with dysarthria as a correlate of Parkinson’s disease, and the associated test set contains 42 additional speakers. A baseline ASR, with a word error rate of 3.4% for typical speakers, transcribes test speech with a word error rate of 36.3%. Fine-tuning reduces the word error rate to 23.7%.
Conclusions: Preliminary findings suggest that a large corpus of dysarthric and dysphonic speech has the potential to significantly improve speech technology for people with disabilities. By providing these data to researchers, the SAP intends to significantly accelerate research into accessible speech technology.
Supplemental Material S1. Examples of participant responses to the prompt "Please explain the steps to making breakfast for 4 people."
Hasegawa-Johnson, M., Zheng, X., Kim, H., Mendes, C., Dickinson, M., Hege, E., Zwilling, C., Channell, M. M., Mattie, L., Hodges, H., Ramig, L., Bellard, M., Shebanek, M., Sarι, L., Kalgaonkar, K., Frerichs, D., Bigham, J. P., Findlater, L., Lea, C., Herrlinger, S., Korn, P., Abou-Zahra, S., Heywood, R., Tomanek, K., & MacDonald, B. (2024). Community-supported shared infrastructure in support of speech accessibility. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2024_JSLHR-24-00122
Funding
This work was made possible by a grant to the University of Illinois from the AI Accessibility Coalition, whose members include Amazon, Apple, Google, Meta, and Microsoft. This article stems from the 2023 Research Symposium at ASHA Convention, which was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under Award R13DC003383. This work utilizes resources supported by the National Science Foundation’s Major Research Instrumentation program, Grant 1725729, as well as the University of Illinois at Urbana-Champaign. The content is solely the responsibility of the authors and does not represent the views of the AI Accessibility Coalition, the National Institutes of Health, the National Science Foundation, Amazon, Apple, Google, Meta, or Microsoft.
History
Usage metrics
