Roles of phonation types and decoders’ gender (Chang et al., 2023)

Purpose: Capturing phonation types such as breathy, modal, and pressed voices precisely can facilitate the recognition of human emotions. However, little is known about how exactly phonation types and decoders’ gender influence the perception of emotional speech. Based on the modified Brunswikian lens model, this article aims to examine the roles of phonation types and decoders’ gender in Mandarin emotional speech recognition by virtue of articulatory speech synthesis.

Method: Fifty-five participants (28 male and 27 female) completed a recognition task of Mandarin emotional speech, with 200 stimuli representing five emotional categories (happiness, anger, fear, sadness, and neutrality) and five types (original, copied, breathy, modal, and pressed). Repeated-measures analyses of variance were performed to analyze recognition accuracy and confusion data.

Results: For male and female decoders, the recognition accuracy of anger from pressed stimuli and fear from breathy stimuli was high; across all phonation-type stimuli, the recognition accuracy of sadness was also high, but that of happiness was low. The confusion data revealed that in recognizing fear from all phonation-type stimuli, female decoders chose fear responses more frequently and neutral responses less frequently than male decoders. In recognizing neutrality from breathy stimuli, female decoders significantly reduced their choice of neutral responses and misidentified neutrality as anger, while male decoders mistook neutrality from pressed stimuli for anger.

Conclusions: This study revealed that, in Mandarin, phonation types play crucial roles in recognizing anger, fear, and neutrality, while the recognition of sadness and happiness seems not to depend heavily on phonation types. Moreover, the decoders’ gender affects their recognition of neutrality and fear. These findings support the modified Brunswikian lens model and have significance for diagnosis and intervention among clinical populations with hearing impairment or gender-related psychiatric disorders.

Supplemental Material S1. Intensity ratings in the total sample and in male and female decoders, respectively.

Supplemental Material S2. Post-hoc comparisons between emotion pairs for each acoustic parameter (except intensity range).

Supplemental Material S3. Post-hoc comparisons between emotion pairs for naturalness and intelligibility, respectively.

Supplemental Material S4. Summary of repeated-measures ANOVAs for the proportions of emotion categorization responses.

Supplemental Material S5. Pairwise comparisons of the correct response proportions of the target emotions for different stimulus type pairs.

Supplemental Material S6. Pairwise comparisons of the proportions of the correct and incorrect response categories for each stimulus type.

Chang, Y., Chen, X., Chen, F., & Li, M. (2023). Roles of phonation types and decoders’ gender in recognizing Mandarin emotional speech. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2023_JSLHR-23-00356