Reliability of short language samples (Wilder & Redmond, 2022)

Purpose: Language sample analysis (LSA) represents an ecologically valid method for diagnosing, identifying goals, and measuring progress in children with developmental language disorder (DLD). LSA is, however, time consuming. The purpose of this study was to determine the length of sample needed to obtain reliable LSA measures for children in kindergarten and first grade with typical language (TL) and DLD using automated analyses from the Systematic Analysis of Language Transcripts software.

Method: Play-based conversational language samples collected on kindergarten to first-grade children with TL (n = 21) and DLD (n = 21) from a community-based sample were analyzed. Eight LSA measures were calculated from 1-, 3-, 5-, 7-, and 10-min sample cuts and compared to 20-min samples for reliability.

Results: Reliability estimates were similar for the TL and DLD groups except for errors and omissions, which showed overall higher levels of reliability in the DLD group and reached acceptable levels at 3 min. Percent grammatical utterances were reliable at 7 min in the DLD group and not reliable in shorter samples in the TL group. The subordination index was reliable at 10 min for both groups. Number of different words reached acceptable reliability at the 3-min length for the DLD group and at the 10-min length for the TL group. Utterances and words per minute were reliable at 3 min and mean length of utterance at 7 min in both groups.

Conclusions: Speech-language pathologists can obtain reliable LSA measures from shorter, 7-min conversational language samples from kindergarten to first-grade children with DLD. Shorter language samples may encourage increased use of LSA.

Supplemental Material S1. Cronbach’s alpha comparing 1, 3, 7, and 10-minute samples to 20-minute samples by group (TL = typical language; DLD = developmental language disorder).

Supplemental Material S2. Pearson correlations between 1, 3, 7, and 10-minute samples and 20-minute samples by group (TL = typical language; DLD = developmental language disorder).

Supplemental Material S3. Coefficient of Variation (CV) for 1, 3, 7, 10, and 20-minute samples by group (TL = typical language; DLD = developmental language disorder).

Supplemental Material S4. Bland-Altman plots for eight LSA measures comparing 1, 3, 7, and 10-min sample lengths to 20-min sample length by group (TL = typical language; DLD = developmental language disorder).

Supplemental Material S5. Bland-Altman plots for MLU comparing 1, 3, 7, and 10-min sample lengths to 20-min sample length by group (TL, DLD) with limits of agreement (LOA) based on ±1.0 MLU (dashed horizontal lines).

Supplemental Material S6. Bland-Altman plots for MLU comparing 1, 3, 7, and 10-min sample lengths to 20-min sample length by group (TL, DLD) with limits of agreement (LOA) based on ±0.5 MLU (dashed horizontal lines).

Supplemental Material S7. Split-half reliability between the first 10 and last 10 minutes of 20-minute language samples calculated with Spearman-Brown coefficients, by group (TL = typical language; DLD = developmental language disorder).

Supplemental Material S8. STROBE checklist.

Wilder, A., & Redmond, S. M. (2022). The reliability of short conversational language sample measures in children with and without developmental language disorder. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2022_JSLHR-21-00628