Does Language Similarity Increase the Degree of Measurement Invariance for Items Measuring Instructional Quality?

Questionnaire data is often used to analyze variation in instructional quality as well as its impact on student performance across countries. Three dimensions of good teaching have been identified: classroom management, teacher support and cognitive activation. Yet, measurement equivalence has hardly been tested. Questionnaire scales are comparable if the identical construct (configural invariance) is measured with the same measurement unit (metric invariance) and origin (scalar invariance). So far, teaching quality scales have been shown to meet the first two stages of equivalence. He and Kubacka (2015) demonstrated that especially Chinese- and Spanish-speaking countries showed a lack of scalar invariance compared to an international reference group. Yet, the level of scale equivalence within these groups has not been tested. Therefore, the present contribution analyzes the degree of measurement invariance for PISA 2012 instructional quality items within as well as between Chinese-, German- and Spanish-speaking country groups, each comprising three countries. Maximum-likelihood based alignment is used to answer the research questions. Scales are invariant if 25% or less of the intercepts are non-invariant. For the Chinese-speaking group, no construct even reaches configural equivalence (RMSEA>0.08). For the German-and Spanish-speaking group, cognitive activation does not reach configural equivalence, while the intercepts for teacher support and classroom management are invariant. No construct is measurement invariant across all language groups. Thus, the construct itself as well as language similarity seem to influence the degree to which cross-cultural comparisons can be made.