Automatic Toolkit for Construction, Optimization, Scoring and Simulation of Forced-Choice Tests

Forced-choice (FC) response has gained increasing popularity and interest for its resistance to faking when well-designed (Cao & Drasgow, 2019 ). To established well-designed FC scales, typically each item within a block should measure different trait and have similar level of social desirability (Zhang et al., 2020 ). Recent study also suggests the importance of high inter-item agreement of social desirability between items within a block (Pavlov et al., 2021 ). In addition to this, FC developers may also need to maximize factor loading differences (Brown & Maydeu-Olivares, 2011 ) or minimize item location differences (Cao & Drasgow, 2019 ) depending on scoring models. Decision of which items should be assigned to the same block, also called as item pairing, is thus critical to the quality of an FC test. Because such pairing process often requires researchers to meet multiple objectives, manual pairing becomes impractical or even not feasible once the number of latent traits and/or number of items per elevates. To address these problems, autoFC is developed as a automatic and efficient tool for facilitating the automatic construction of FC tests (Li et al., 2022 ), essentially exempting users from the burden of manual item pairing. Given characteristics of each item (and item responses), FC measures can be constructed either automatically based on user-defined pairing criteria and weights, or based on exact specifications of each block (i.e., blueprint; see Li et al., 2025 ). Users can also generate simulated responses based on the Thurstonian Item Response Theory model (Brown & Maydeu-Olivares, 2011 ) and predict trait scores of simulated/actual respondents based on an estimated model.

autoFC: Automatic tools for Forced-Choice Scale Assembly, Optimization, Scoring and Simulation

Overview

autoFC is a comprehensive, fully automatic R package designed to support the entire lifecycle of forced-choice (FC) scale development, supporting multiple mainstream FC formats including PICK, MOLE, RANK, Graded FC and Compositional FC. It provides robust tools for automated test assembly, advanced psychometric scoring (using R, Mplus, or Stan), and data simulation based on the Thurstonian IRT model (Brown & Maydeu-Olivares, 2011) for PICK, MOLE, and RANK, ordinal graded response IRT model (Brown & Maydeu-Olivares, 2018) for Graded FC, and compositional FC IRT model (Brown, 2016) for Compositional FC.

Augest 2026 Update Note

This major update introduces support for graded FC and compositional FC formats and implements all three scoring engines for both formats.

Generation of simulated FC responses on PICK, MOLE, RANK, graded FC and compositional FC formats are now fully supported using the simulate_fc_data() function.

In sum, most mainstream FC formats can now be easily simulated, scored and evaluated using the autoFC.

What's New in the May 2026 Major Update

This major update introduces critical performance, usability, and mathematical enhancements. These features are added to allow for much easier and convenient use of FC scales, than ever before.

• Thurstonian Factor Model (TFM) and Thurstonian IRT (IRT) Support: Native syntax generation and scoring for highly stable first-order and second-order models in lavaan and Mplus, completely bypassing tedious manual constraint declarations.

• Vectorized, C-Level Algorithmic Speedups: Restructured simulated annealing optimization engine to significantly improve performance.

• High-Performance Stan Integration: A unified Stan pipeline featuring within-chain parallel computing (reduce_sum)to speed up estimation even for larger blocks.

• Fast Scoring: Fast, vectorized MAP scoring in R using analytical gradients to score participants and calculate Standard Errors in all scoring engines.

Installation

You can install autoFC from CRAN:

install.packages("autoFC")

You can install the development version of autoFC from GitHub:

devtools::install_github("mtlpsych/autoFC")

Core Functions

autoFC divies the FC workflow into several logical stages:

1. Scale Assembly and Optimization

create_blueprint_template(): A starter function building up csv templates for users to fill in their own blueprint.

build_blueprint_blocks(): A block-by-block initializer that constructs starting scales matching the exact trait/sign blueprints that user specifies, while also allowing for optimizing item characteristics.

optimize_blocks() (Previously known as sa_pairing_generalized()): The CORE optimization engine based on the Simulated Annealing (SA) algorithm. It is capable of balancing within-block matching on 1 or more criteria (local fit) and overall trait pair distributions (global fit), while preventing within-block trait overlap.

2. Data Preparation and Diagnostics

convert_ranks_to_pairwise(), convert_mole_to_pairwise(): Converts raw fully ranked responses or "Most-Least Like Me" (MOLE) partial rankings into binary pairwise columns, handling unobserved comparisons using NAs.

summarize_trait_pairs(): Provides a tabular diagnostic summary of how evenly equally-keyed and mixed-keyed trait pairs are distributed across all trait combinations in your FC scale.

3. Syntax Generation

generate_tirt_lavaan_syntax(), generate_tirt_mplus_syntax(), prepare_tirt_stan_data(): For R and Mplus, generates syntax for either First-Order or Second-Order Thurstonian IRT models, also natively supporting character trait names and automated Heywood-case prevention (i.e., negative residual variances). For Stan, generates a unified, pre-compiled Stan model with multithread capabilities and logical dependency reduction.

4. Scoring and Estimation

All three functions below now allow score estimation for not only original calibration sample, BUT ALSO for new samples.

score_tirt_lavaan(): Fast MAP scoring and Standard Error (Yes, now available) estimation in R for lavaan models estimated from the model produced by generate_tirt_lavaan_syntax(). Also supports both first- and second-order parameterizations.

score_tirt_mplus(): Fast parameter extraction from Mplus .out files and participant scoring using R, bypassing Mplus's FSCORES engine.

score_tirt_stan(): Used in conjunction with prepare_tirt_stan_data(). Based on the modern features in cmdstan package, now it allows very fast scoring and prediction in Stan (at least 10x faster than stan prediction methods in earlier versions). In the case of Stan, use predict_tirt_stan() for predicting trait scores of new samples.

empirical_reliability(): Calculates the test-level empirical reliability of estimated trait scores (Brown & Maydeu-Olivares, 2018).

5. Parameter Extraction & Response Simulation

get_CFA_estimates(): Fits CFA model and/or extracts item parameters from raw Likert data or pre-fitted lavaan objects.

get_simulation_matrices(): Simulates latent traits, item parameters, and continuous utilities for Thurstonian IRT modeling, based on fitted CFA models.

plot_scores(): Visual diagnostics for comparing true vs. estimated scores, complete with reference lines.

RMSE_range(): Evaluates trait recovery accuracy across specified intervals of the latent continuum.

References

Brown, A. (2016). Thurstonian scaling of compositional questionnaire data. Multivariate Behavioral Research, 51(2-3), 345-356. https://doi.org/10.1080/00273171.2016.1150152

Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460-502. https://doi.org/10.1177/0013164410375112

Brown, A., & Maydeu-Olivares, A. (2018). Ordinal factor analysis of graded-preference questionnaire data. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 516-529. https://doi.org/10.1080/10705511.2017.1392247

Li, M., Zhang, B., Li, L., Sun, T., & Brown, A. (2025). Mixed-keying or desirability-matching in the construction of forced-choice measures? An empirical investigation and practical recommendations. Organizational Research Methods, 28(2), 296-329. https://doi.org/10.1177/10944281241229784