Individual, Family, and Community Education ETDs

Publication Date

7-12-2014

Abstract

English language proficiency assessments are currently in use by the 50 states (Wolf, Farnsworth, & Herman, 2008). Unfortunately, validations studies seldom address the effects of home languages on the use and interpretation of these assessments. If this step is overlooked, the assessment may include items that unfairly position one group of home language scores over another group. An omission of home languages was the case for the New Mexico English Language Proficiency Assessment (NMELPA) (Harcourt Assessment, Inc., 2006, 2007). For this reason, the NMELPA was investigated for group differences between those with first language origins in Spanish and Navajo, the largest groups in New Mexico. The investigation focused on the Reading and Listening subscales of the NMELPA. A Differential Item Functioning (DIF) analysis determined if items on the NMELPA exhibited different scores between the Navajo and Spanish groups. DIF occurs when two groups matched on ability have different probabilities of correctly answering an item (AERA, APA, & NCME, 1999; Camilli & Sheppard, 1994; Osterlind & Everson, 2009). For this study, a method based on Logistic Regression (LR) was used to examine the items for DIF. The DIF analysis determined if the two groups performed differently on any particular items. The results revealed DIF for 22 out of 24 items on the Reading subscale. Whereas, 15 out of 20 items on the Listening subscale indicated DIF. After correcting for effect size, five items on the Reading subscale indicated relatively large' DIF (Cole, Kawachi, Maller, & Berkman, 2000, p. 286). Conversely, no 'relatively large' DIF was detected for items on the Listening subscale. Based on these findings, the NMELPA contained certain items that clearly favored the Navajo group over the Spanish group. Overall, these findings illustrate how home languages could be a source of construct irrelevant variance in the internal structure of English proficiency assessments. Therefore, the validation argument for assessments of English proficiency should consider the effects of different home languages on score outcomes.'

Keywords

English Language Learners, Home Languages, English Proficiency, Logistic Regression, Differential Item Functioning

Document Type

Dissertation

Language

English

Degree Name

Educational Psychology

Level of Degree

Doctoral

Department Name

Individual, Family, and Community Education

First Committee Member (Chair)

Selig, James

Second Committee Member

Armstrong, Jan

Third Committee Member

Salisbury, Mark

Download

Share

COinS