Identification of nonuniform differential item functioning using a variation of the Mantel-Haenszel procedure and the iterative logit method

Abstract

The standard Mantel-Haenszel (MH) procedure, a simple modification of the MH procedure (reanalyzing the data separately for high- and low-performing groups), and the iterative logit method were compared with respect to their robustness and power to detect symmetric nonuniform diferential item functioning (DIF). Data for a 79-item test with 11% of DIF items were simulated using a three parameter logistic model for focal and reference groups. The ability distributions of both groups were normally distributed. The factors manipulated were sample size (200 and 1,000 examinees per group) and DIF-size (.25 and .50). As expected, results show that sample size and DIF size improve power, although the power is rather low at each of the factor levels. The MH variation yields higher power than the standard MH procedure and the iterative logit method for each cell of the design, but it was not robust in the N = 1,000 sample size condition.