Evaluation of a social communication assessment (Poll et al., 2024)

Purpose: This study evaluated the psychometric functioning of a new criterion-referenced assessment of adolescent social communication, the Transition Pragmatics Interview (TPI), based on the synthesis model of pragmatics. Two ways of interpreting item difficulty were explored: (a) as a function of the synthesis model elements of social communication ability that items were designed to assess, and (b) as a function of the developmental level required for a successful response based on an adapted situational-discourse-semantics (SDS) model (Norris & Hoffman, 1993).

Method: Thirty-seven participants aged 14–22 years completed the TPI. Responses were analyzed using Rasch analysis to evaluate the functioning of the scale and to determine item difficulty. Items were coded for the SDS developmental level required for an adequate response. The mean Rasch item difficulty for items at each SDS developmental level was analyzed for the five adapted SDS domains.

Results: Consistent with the first approach for interpreting item difficulty, TPI items varied in difficulty as a function of the element of social communication they were designed to assess (p < .001). Interpreting item difficulty based on the adapted SDS model was not supported: Items requiring higher SDS developmental levels were not more difficult than those requiring less (p = .55).

Conclusions: The TPI responses fit the Rasch model, supporting the TPI as a unidimensional measure and supporting the use of all items together to compute a single number that summarizes the level of social communication for each examinee. The item ordering from least to most difficult was consistent with prior findings on adolescent social communication development.

Supplemental Material S1. Category probability curves for the Transition Pragmatics Interview items.

Supplemental Material S2. Transition Pragmatics Interview items rated as requiring early, intermediate or advanced abilities to read situations by mean Rasch item difficulty in logit units.

Poll, G. H., Boone, W. J., & Petru, J. (2024). Evaluation of a new adolescent social communication assessment: The Transition Pragmatics Interview. Journal of Speech, Language, and Hearing Research, 67(7), 2353–2366. https://doi.org/10.1044/2024_JSLHR-23-00709