Simulation and Analysis of Contextual Multi-Armed Bandit Policies

Facilitates the simulation and evaluation of context-free and contextual multi-Armed Bandit policies or algorithms to ease the implementation, evaluation, and dissemination of both existing and new bandit algorithms and policies.


Contextual: Multi-Armed Bandits in R

lifecycle AppVeyor Build Status Build Status codecovLicense: GPL v3 DOI CRAN_Status_Badge

Overview

R package facilitating the simulation and evaluation of context-free and contextual Multi-Armed Bandit policies.

The package has been developed to:

  • Introduce a wider audience to contextual bandit policies' advanced sequential decision strategies.
  • Ease the implementation, evaluation and dissemination of both existing and new contextual Multi-Armed Bandit policies.

Installation

To install contextual from CRAN:

install.packages('contextual')

To install the development version (requires the devtools package):

install.packages("devtools")
devtools::install_github('Nth-iteration-labs/contextual')

When working on or extending the package, clone its GitHub repository, then do:

install.packages("devtools")
devtools::install_deps(dependencies = TRUE)
devtools::build()
devtools::reload()

clean and rebuild...

Documentation

See the demo directory for practical examples and replications of both synthetic and offline (contextual) bandit policy evaluations.

How to replicate figures from two books, both offering a first introduction to context-free Multi-Armed Bandits:

Basic, context-free multi-armed bandit examples:

Examples of both synthetic and offline contextual multi-armed bandit evaluations:

Some more extensive vignettes to get you started with the package:

Paper offering a general overview of the package's structure & API:

Overview of core classes

Policies and Bandits

Overview of contextual's growing library of contextual and context-free bandit policies:

General Context-free Contextual Other
Random
Oracle
Fixed




Epsilon-Greedy
Epsilon-First
UCB1, UCB2
Thompson Sampling
BootstrapTS
Softmax
Gradient
Gittins
CMAB Naive Epsilon-Greedy
Epoch-Greedy
LinUCB (General, Disjoint, Hybrid)
Linear Thompson Sampling
ProbitTS
LogitBTS
GLMUCB

Lock-in Feedback (LiF)







Overview of contextual's bandit library:

Basic Synthetic Contextual Synthetic Offline Continuous
Basic Bernoulli Bandit
Basic Gaussian Bandit



Contextual Bernoulli
Contextual Logit
Contextual Hybrid
Contextual Linear
Contextual Wheel
Replay Evaluator
Bootstrap Replay
Propensity Weighting
Direct Method
Doubly Robust
Continuum




Alternative parallel backends

By default, "contextual" uses R's built-in parallel package to facilitate parallel evaluation of multiple agents over repeated simulation. See the demo/alternative_parallel_backends directory for several alternative parallel backends:

Maintainers

Robin van Emden: author, maintainer* Maurits Kaptein: supervisor*

* Tilburg University / Jheronimus Academy of Data Science.

If you encounter a clear bug, please file a minimal reproducible example on GitHub.

News

contextual 0.9.0

  • First release

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("contextual")

0.9.8 by Robin van Emden, 11 days ago


https://github.com/Nth-iteration-labs/contextual


Report a bug at https://github.com/Nth-iteration-labs/contextual/issues


Browse source code at https://github.com/cran/contextual


Authors: Robin van Emden [aut, cre] , Maurits Kaptein [ctb]


Documentation:   PDF Manual  


GPL-3 license


Imports R6, data.table, R.devices, foreach, doParallel, itertools, iterators, Formula

Suggests testthat, RCurl, splitstackshape, covr, knitr, here, rmarkdown, devtools, ggplot2, vdiffr


See at CRAN