Routines for Performing Empirical Calibration of Observational Study Estimates

Routines for performing empirical calibration of observational study estimates. By using a set of negative control hypotheses we can estimate the empirical null distribution of a particular observational study setup. This empirical null distribution can be used to compute a calibrated p-value, which reflects the probability of observing an estimated effect size when the null hypothesis is true taking both random and systematic error into account.


EmpiricalCalibration

Introduction

This R package contains routines for performing empirical calibration of observational study estimates. By using a set of negative control hypotheses we can estimate the empirical null distribution of a particular observational study setup. This empirical null distribution can be used to compute a calibrated p-value, which reflects the probability of observing an estimated effect size when the null hypothesis is true taking both random and systematic error into account, as described in the paper [Interpreting observational studies: why empirical calibration is needed to correct p-values.] (http://dx.doi.org/10.1002/sim.5925).

Features

  • Estimate the empirical null distribution given the effect estimates of a set of negative controls
  • Estimate the calibrated p-value of a given hypothesis given the estimated empirical null distribution
  • Produce various plots for evaluating the empirical calibration
  • Contains the data sets from the paper for illustration

Screenshots and examples

data(sccs) #Load one of the included data sets
negatives <- sccs[sccs$groundTruth == 0,] #Select the negative controls
null <- fitNull(negatives$logRr,negatives$seLogRr) #Fit the null distribution
positive <- sccs[sccs$groundTruth == 1,]  #Select the positive control plotCalibrationEffect(negatives$logRr,negatives$seLogRr,positive$logRr,positive$seLogRr,null)
 
#Compute the calibrated p-value:
calibrateP(positive$logRr,positive$seLogRr, null) #Compute calibrated p-value
[1] 0.8390598

Technology

This is a pure R package.

System requirements

Requires R (version 3.1.0 or newer).

Getting Started

In R, use the following commands to install the latest stable version from CRAN:

install.packages("EmpiricalCalibration")

To install the latest development version directly from GitHub, use:

install.packages("devtools")
library(devtools)
install_github("ohdsi/EmpiricalCalibration")

Getting Involved

License

EmpiricalCalibration is licensed under Apache License 2.0

Development

This package has been developed in RStudio. ###Development status

This package is ready for use.

Acknowledgements

Martijn Schuemie is the author of this package.

News

EmpiricalCalibration v1.2.0 (Release date: 2016-08-15)

NEW FEATURES

  • Ability to add credible intervals to calibration effect plot

  • Plot CI calibration (using leave-one-out cross-validation)

BUG FIXES

  • Fixed vignette name in index

  • Removed coverage plot (moved to MethodEvaluation package)

EmpiricalCalibration v1.1.0 (Release date: 2016-02-15)

Changes: initial submission to CRAN

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("EmpiricalCalibration")

1.3.6 by Martijn Schuemie, 18 days ago


https://github.com/OHDSI/EmpiricalCalibration


Report a bug at https://github.com/OHDSI/EmpiricalCalibration/issues


Browse source code at https://github.com/cran/EmpiricalCalibration


Authors: Martijn Schuemie, Marc Suchard


Documentation:   PDF Manual  


Apache License 2.0 license


Imports ggplot2, gridExtra, methods

Suggests knitr, rmarkdown


See at CRAN