Bayesian Prior Elicitation, Diagnostics, and Regulatory Reporting

A toolkit for constructing, validating, and justifying Bayesian priors in clinical trial settings. Implements expert elicitation via quantile matching, the roulette method, and moment matching across six distribution families, linear and logarithmic expert pooling, meta-analytic-predictive (MAP) priors derived from historical trials, prior-data conflict diagnostics including the Box p-value (computed from the exact prior predictive distribution where one has a closed form), an exploratory surprisal (S-value) companion to it, surprise index, information divergence, and Mahalanobis distance, sensitivity analyses with tornado and influence heatmap plots, sceptical, robust, and power priors, and automated prior justification reports. Includes a fully modular 'Shiny' application for interactive use. Methods based on O'Hagan et al. (2006, ISBN:9780470029886), Box (1980) , Greenland (2023) , Oakley and O'Hagan (2010) < https://tonyohagan.co.uk/shelf/>, Schmidli et al. (2014) , Neuenschwander et al. (2010) , Roever et al. (2021) , Ibrahim and Chen (2000) , Spiegelhalter et al. (1994) .

bayprior

Structured Bayesian prior elicitation, conflict diagnostics, and regulatory reporting.

Overview

bayprior is an advanced interactive R package and Shiny application designed for biostatisticians and clinical researchers to implement Bayesian Prior Elicitation, Conflict Diagnostics, and Sensitivity Analysis for clinical trials.

bayprior focuses on a specific stage of the Bayesian trial workflow: how do you construct, validate, and justify your prior to a regulator? It brings structured elicitation, prior-data conflict diagnostics, and regulatory report generation together in one integrated workflow. The FDA's 2026 draft guidance on Bayesian methods makes this a growing area of focus.

bayprior enables users to:

• Elicit structured priors — SHELF-style quantile matching, moment matching, and the interactive roulette method across Beta, Normal, Gamma, Log-Normal, Exponential, and Weibull families.
• Aggregate expert opinions — Linear or logarithmic pooling of multiple expert priors with pairwise Bhattacharyya agreement diagnostics and cross-family compatibility validation.
• Derive priors from historical trials — Meta-analytic-predictive (MAP) priors from a random-effects meta-analysis, with an explicit prior on between-trial heterogeneity.
• Diagnose prior-data conflict — Box's p-value (Normal approximation or exact prior predictive), S-value, surprise index, KL divergence, Bhattacharyya overlap, and multivariate Mahalanobis distance, supporting binary, continuous, Poisson/count, and survival data types.
• Quantify sensitivity — Posterior conclusions evaluated across hyperparameter grids via tornado plots and influence heatmaps, with independent data entry.
• Build robust priors — Sceptical, robust mixture, and calibrated power priors for regulatory sensitivity analyses.
• Generate regulatory reports — Self-contained HTML, PDF, or Word (.docx) prior justification reports rendered via Quarto.

Features and Modules

Core Methodology

Prior Elicitation

Three structured elicitation approaches are implemented. Quantile matching fits a parametric distribution to expert-specified probability-value pairs via numerical optimisation. Moment matching derives hyperparameters analytically from an expert-supplied mean and SD. The SHELF roulette method (Oakley & O'Hagan, 2010) lets the expert allocate chips across histogram bins, fitting a parametric family to the chip allocation in real time.

All three methods support six distribution families:

MAP Prior (Historical Data)

The meta-analytic-predictive (MAP) approach (Schmidli et al., 2014) derives an informative prior directly from multiple historical trials via a random-effects meta-analysis, with an explicit prior on the between-trial heterogeneity parameter tau -- how much heterogeneity is assumed among historical trials directly affects how much borrowing is justified, so map_prior() requires this to be stated rather than left as an unstated default (see ?resolve_tau_prior for outcome-specific defaults). It returns an ordinary prior object that can be used directly, or passed into robust_prior() for an additional vague-mixture component.

y/se per historical trial can be supplied directly, or derived from raw trial-level summary statistics via the optional convenience function historical_effect_sizes(), a thin wrapper around metafor::escalc() (metafor is Suggests-only and not required to use map_prior() itself).

Prior-Data Conflict Diagnostics

Conflict detection follows Box (1980). Complementary metrics are computed:

• Prior predictive p-value — tests whether observed data is plausible under the prior predictive distribution.
• S-value (surprisal, bits) — -log2(p): an exploratory companion to the p-value above, not a thresholded decision rule, since prior-data conflict checks are typically not powered for a binary accept/reject call.
• Surprise index — standardised distance between prior mean and observed data.
• Bhattacharyya overlap — distributional overlap between prior and normalised likelihood.
• KL divergence — information-theoretic distance from prior to likelihood.

The Box p-value uses the exact prior predictive distribution wherever one has a closed form: Beta-Binomial (Beta prior, binary data) and Gamma-Poisson / Negative Binomial (Gamma prior, Poisson or survival data). Other combinations use a Normal approximation to the prior and the likelihood, and $box_pvalue_method reports which was used. exact = FALSE forces the approximation, which can flag conflict too often for informative priors with small samples.

Four data types are supported:

Sensitivity Analysis

The sensitivity module is fully independent of conflict diagnostics — users can enter observed data directly without running conflict diagnostics first. Results are visualised as tornado plots and influence heatmaps. A dedicated sensitivity_cri() function tracks credible interval width specifically — a key regulatory quantity.

Robust and Power Priors

The robust mixture prior (Schmidli et al., 2014) mixes the informative prior with a vague Normal component. The sceptical prior (Spiegelhalter & Freedman, 1994) is centred at the null treatment effect. The power prior (Ibrahim & Chen, 2000) down-weights historical data by delta in (0, 1], calibrated to achieve a target Bayes Factor.

Shiny application

run_app() launches the full workflow as an interactive app (also available as a live demo). The Welcome page shows the analytical workflow, tracks which steps you have completed, and can load a synthetic Phase II example (TRIAL-001) so you can try the conflict check straight away.

Installation

PDF reports require Quarto CLI and a LaTeX installation:

install.packages("tinytex")
tinytex::install_tinytex()

Quick Start

library(bayprior)

# Elicit a Beta prior
prior <- elicit_beta(mean = 0.35, sd = 0.10, method = "moments",
                     label = "Response rate", expert_id = "Expert_1")
plot(prior)

# Pool two experts
e1  <- elicit_beta(mean = 0.30, sd = 0.10, method = "moments", expert_id = "E1")
e2  <- elicit_beta(mean = 0.42, sd = 0.12, method = "moments", expert_id = "E2")
agg <- aggregate_experts(list(E1 = e1, E2 = e2), weights = c(0.6, 0.4))

# Conflict diagnostics (exact prior predictive p-value where available; exact = FALSE forces the approximation)
cd <- prior_conflict(prior, list(type = "binary", x = 18, n = 40))
print(cd)

# Derive a MAP prior from historical trials (log-odds scale)
hist_prior <- map_prior(
  y  = qlogis(c(0.28, 0.35, 0.31)),   # historical response rates, as log-odds
  se = c(0.30, 0.26, 0.34),           # their standard errors
  outcome_type = "single_arm_log_odds"
)
plot(hist_prior)

# Sensitivity analysis
sa <- sensitivity_grid(
  prior,
  data_summary = list(type = "binary", x = 18, n = 40),
  param_grid   = list(alpha = seq(1, 8, 0.5), beta = seq(2, 20, 1)),
  target       = c("posterior_mean", "prob_efficacy"),
  threshold    = 0.30
)
plot_tornado(sa)

# Robust and sceptical priors
rob  <- robust_prior(prior, vague_weight = 0.20)
scep <- sceptical_prior(null_value = 0.20, family = "beta",
                        strength = "moderate")

# Generate regulatory report
prior_report(
  prior           = prior,
  conflict        = cd,
  sensitivity     = sa,
  robust_prior    = rob,
  sceptical_prior = scep,
  output_format   = "html",
  trial_name      = "TRIAL-001",
  sponsor         = "Example Pharma Ltd",
  author          = "N.P., Biostatistician"
)

# Launch the Shiny app
run_app()

Vignettes

browseVignettes("bayprior")

Documentation

Full documentation — vignettes, changelog, and cheat sheet. Function help is available in R via help(package = "bayprior") or ?bayprior.

Code of Conduct

Please note that the bayprior project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

References

• O'Hagan, A. et al. (2006). Uncertain Judgements: Eliciting Experts' Probabilities. Wiley.
• Box, G. E. P. (1980). Sampling and Bayes' inference in scientific modelling and robustness. JRSS-A, 143, 383-430.
• Greenland, S. (2023). Divergence versus decision P-values: A distinction worth making in theory and keeping in practice. Scandinavian Journal of Statistics, 50(1), 54-88.
• Oakley, J. E. & O'Hagan, A. (2010). SHELF: the Sheffield Elicitation Framework. University of Sheffield.
• Schmidli, H. et al. (2014). Robust meta-analytic-predictive priors in clinical trials with historical control information. Biometrics, 70, 1023-1032.
• Neuenschwander, B., Capkun-Niggli, G., Branson, M. & Spiegelhalter, D. J. (2010). Summarizing historical information on controls in clinical trials. Clinical Trials, 7(1), 5-18.
• Roever, C., Bender, R., Dias, S., Schmid, C. H., Schmidli, H., Sturtz, S., Weber, S. & Friede, T. (2021). On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis. Research Synthesis Methods, 12(4), 448-474.
• Ibrahim, J. G. & Chen, M.-H. (2000). Power prior distributions for regression models. Statistical Science, 15, 46-60.
• Spiegelhalter, D. J., Freedman, L. S. & Parmar, M. K. B. (1994). Bayesian approaches to randomized trials. JRSS-A, 157, 357-416.
• U.S. Food and Drug Administration (2026). Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products (Draft Guidance for Industry).