Quick Screening of Response Patterns in Test and Survey Items

One call screens the item responses of a test or questionnaire for patterns that deserve a closer look before modelling: flat or long-string responding, low intra-individual variability, multivariate outliers, incoherent psychometric synonyms or even-odd halves, overuse of the midpoint or of the extremes, duplicated response vectors and identical patterns that arrive in series of consecutive submissions. Indices follow Curran (2016) ; person fit in item response theory (the standardized log-likelihood) is optional, and cut-offs can be taken from a reference group. Results are returned per person, with flags, frequency tables of response patterns and plots. Whole batches of data (a second collection phase, one classroom) can be compared with a reference group: the item correlation structure, with permutation tests that leave out flat responders and match the response level, and the Jennrich (1970) test; the similarity of consecutive rows; and near-duplicate respondents. Flagged respondents can be removed in one step and the same confirmatory or exploratory factor model can be refitted with and without them.