Listener Background in L2 Speech Evaluation

*Mohammadreza Dalman and Okim Kang*

## **Abstract**

Listeners are integral parts of second language (L2) oral performance assessment. However, evaluation of listeners is susceptible to listener background variables and biases. These variables and preexisting biases distort native speaker (NS) listeners' perceptions of non-native speakers' (NNSs) speech performance and contribute errors into their oral performance assessment. Among listener background variables, listeners' first language status, the amount of exposure to different English varieties, listeners' educational background, prior language teaching experience, NNSs' linguistic stereotyping, and listener attitude have been investigated in the literature and assumed to exert sizable amount of variation in speakers' oral proficiency true scores. To minimize listeners' bias in the assessment context, listeners are provided with intensive training programs in which they are trained how to rate NNSs' speech more objectively utilizing scoring rubrics. To mediate listeners' bias in social contexts, the literature has provided strands of evidence in favor of structured intergroup contact programs, which are inoculations particularly devised to improve NSs' attitude, thereby making them more receptive to NNSs' English varieties. To enhance L2 listeners' self-efficacy and foster their autonomy, L2 instructors are encouraged to emphasize explicit instruction of listening strategies.

**Keywords:** listener background, listener bias, listener training, listening strategies, self-efficacy, assessment

#### **1. Introduction**

Despite recent advances made in the application of automatic speech recognition (ASR) technology in second language (L2) pronunciation, the evaluation of L2 speakers' oral performance is extensively carried out through the judgment of human raters. This is the case whether the evaluation is narrowly focused on pronunciation accuracy and speech intelligibility or more broadly on communication success. The human rater judgments of L2 speakers' oral performance have consequential impacts on L2 speakers due to the fact that they form the bases upon which critical decisions are made regarding L2 speakers' education and employment. However, human raters are vulnerable to be impacted by listeners' bias. Listeners who harbor negative expectations toward a certain group of speakers due to their social group affiliations, nationality, and un-prestigious accent have the proclivity to assess accented speech more negatively [1–3]. The bias formed mainly by listeners' background factors, such as listeners' predispositions, attitudes, expectations, and stereotypes, compromises the validity of oral proficiency assessment and unrightfully contributes to the

speakers' oral proficiency score variance. Thus, the results of the assessments may not make valid contributions to L2 speech research and teaching due to such inaccurate assessment. Given the fact that the insertion of the listener-related variables, also referred to as "trait-irrelevant" variables, would obscure the speakers' true speaking ability, the main objective of oral proficiency assessment and second language research is to mitigate the potency of the extraneous factors so that the obtained score rightfully reflects the true ability of the speakers [4].

The current chapter is a desktop review aimed at examining existing research that have investigated a wide range of listener background factors, which are at play in speakers' score variation and oral proficiency discrepancies. This review would inform future researchers of the presence of the listener-related variables and help them devise some effective strategies to lessen, if not to eradicate, their intervening effects. The factors specifically accentuated in this desktop review include listener first language status, effect of exposure to different varieties of English, listener educational background and linguistic knowledge, effects of prior language teaching and tutoring experience, linguistic stereotyping, and listener attitude. We will then discuss rater training in the assessment contexts and structured contact activities which are used as remedies for minimizing the effects of listeners' bias, as well as L2 listening strategy instruction and self-efficacy, followed by implications for L2 pronunciation research and pedagogy as well as recommendations for future directions.
