Detecting, Decomposing, and Stress-Testing Temporal Change in Repeated Decision Systems

Tools for detecting, decomposing, and testing temporal change in repeated binary decision systems. Decomposes each change in the decision rate into inflow (uptake) and outflow (reversal) through the marginal-transition identity, separates net change from gross turnover, and splits aggregate change into within-group and composition components with a symmetric midpoint decomposition. A unit-level cluster bootstrap gives intervals for every component and a global test of process stability with family-wise-adjusted contrasts and simultaneous confidence bands. Also provides prevalence and transition drift summaries, entropy trends, group-differential drift, change-point diagnostics, robustness and sensitivity analyses, an integrated audit, and a simulator for stable, marginal, masked, abrupt, group-masked, and composition-change scenarios. Complements the 'decisionpaths' package. Applications include algorithmic audit, AI governance, education, health, and organisational research.

DecisionDrift

Detecting, Decomposing, and Stress-Testing Temporal Change in Repeated Decision Systems

Overview

DecisionDrift is an R package for auditing temporal drift in repeated binary decision systems — such as algorithmic screening tools, institutional review boards, or any AI system that issues repeated yes/no decisions over time.

It complements the decisionpaths package by shifting focus from path construction to system-level change:

Installation

From GitHub (recommended)

# install.packages("remotes")
remotes::install_github("subirhait/DecisionDrift")

From CRAN (once accepted)

install.packages("DecisionDrift")

Quick start

library(DecisionDrift)

# --- 1. Build a drift panel object -----------------------------------------
dat <- data.frame(
  id       = rep(1:100, each = 8),
  time     = rep(1:8, times = 100),
  decision = rbinom(800, 1, rep(seq(0.25, 0.55, length.out = 8), 100)),
  group    = rep(c("A", "B"), 50, each = 1)
)

dp <- dd_build(dat, id, time, decision,
               group      = group,
               event_time = 5L)   # known policy change at wave 5

# --- 2. Run the full audit in one call --------------------------------------
aud <- dd_audit(dp, R = 499, seed = 1)
print(aud)
plot(aud)

# --- 3. Or call individual modules ------------------------------------------
dd_prevalence(dp)      # Was the overall decision rate stable?
dd_transition(dp)      # Did switching/persistence dynamics change?
dd_entropy_trend(dp)   # Did path complexity evolve?
dd_group_drift(dp)     # Did drift differ across groups?
dd_changepoint(dp)     # Where did the system break?

# --- 4. Summary drift indices -----------------------------------------------
dd_indices(dp)
# DDI = Decision Drift Index
# TDI = Transition Drift Index
# GDD = Group Differential Drift
# CDB = Cumulative Drift Burden

# --- 5. Decompose and test (new in 0.2.0) ----------------------------------
dd_decompose(dp, R = 499)        # change = inflow - outflow, with turnover
dd_group_decompose(dp, R = 499)  # within-group vs. composition change
dd_global_test(dp, R = 999)      # are uptake and reversal constant?

# --- 6. Stress-test conclusions ---------------------------------------------
dd_robustness(dp)      # Are conclusions stable across analytic choices?
dd_sensitivity(dp)     # How vulnerable are conclusions to data problems?

# --- 7. Simulate panels with known drift -------------------------------------
masked <- dd_simulate(n = 600, waves = 6, scenario = "masked", seed = 1)

Package architecture

DecisionDrift ├── dd_build() Core: build drift_panel object │ ├── Module 1: dd_prevalence() Drift in decision rate → DDI ├── Module 2: dd_transition() Drift in transition structure → TDI ├── Module 3: dd_entropy_trend() Drift in path complexity / stability ├── Module 4: dd_group_drift() Group-differential drift → GDD ├── Module 5: dd_changepoint() Change-point & regime detection │ ├── dd_indices() All four indices in one call (DDI, TDI, GDD, CDB) │ ├── dd_decompose() Flow decomposition: inflow, outflow, turnover, │ NEC, CMB, CTB ├── dd_group_decompose() Within-group vs. composition change ├── dd_cluster_bootstrap() Unit-level cluster bootstrap (shared draws) ├── dd_global_test() Global stability test, adjusted contrasts, │ simultaneous bands ├── dd_simulate() Panels with known drift scenarios │ ├── dd_robustness() Stability across: balanced panel, LOPO, │ LOGO, min_waves grid, bootstrap CIs │ ├── dd_sensitivity() Vulnerability to: miscoding, missingness, │ threshold shifts, composition shifts │ └── dd_audit() Flagship: 3-layer audit → test-based conclusion

The four drift indices

The three-layer audit

dd_audit() follows a deliberate three-layer logic:

1. Detection — Did the process change? dd_global_test(), dd_prevalence(), dd_transition()

2. Decomposition — Was change in inflow or outflow, within groups or in composition, gradual or abrupt? dd_decompose(), dd_group_decompose(), dd_entropy_trend(), dd_group_drift(), dd_changepoint()

3. Stress-testing — Are conclusions robust to noise, coding choices, and missing waves? dd_robustness(), dd_sensitivity()

The audit closes with a conclusion that states the global test result (bootstrap p-value at a chosen alpha). It does not assign labels such as "strong drift". Practical-importance cut-offs depend on the application and are supplied by the user, for example thresholds = c(TDI = 0.03, NEC = 0.05).

Why prevalence is not enough

The positive rate can stay flat while many units move in and out of the positive state. The marginal-transition identity

pi(t+1) - pi(t) = (1 - pi(t)) * uptake(t) - pi(t) * reversal(t)

shows that change is inflow minus outflow. If both rise together, prevalence does not move. dd_decompose() reports both flows and their sum (turnover), and dd_global_test() tests whether uptake and reversal stayed constant.

Relationship to decisionpaths

decisionpaths answers: What paths occurred, how stable, how entropic, how equitable?

DecisionDrift answers: Did the system itself change, when, how much, for whom, and is that conclusion robust?

They are designed to be used together: build paths with decisionpaths::dp_build(), audit drift with DecisionDrift::dd_audit().

A third package in the ecosystem, AIBias, will ask: Did inequality accumulate or amplify over repeated decisions?

Dependencies

Imports: cli, rlang, stats, tibble, utils

Suggests (for plotting): ggplot2, patchwork

Citation

Hait, S. (2026). DecisionDrift: Detecting, decomposing, and stress-testing temporal change in repeated decision systems. R package version 0.2.0. https://github.com/subirhait/DecisionDrift

License

MIT © Subir Hait