RenyiExtropy
Entropy and Extropy Measures for Probability Distributions
Provides functions to compute Shannon entropy, Renyi entropy, Tsallis entropy, and related extropy measures for discrete probability distributions. Includes joint and conditional entropy, KL divergence, Jensen-Shannon divergence, cross-entropy, normalized entropy, and Renyi extropy (including the conditional and maximum forms). All measures use the natural logarithm (nats). Useful for information theory, statistics, and machine learning applications.
RenyiExtropy 
RenyiExtropy provides a comprehensive collection of entropy and extropy measures for discrete probability distributions. All measures use the natural logarithm (results in nats).
Functions
| Function | Description |
|---|---|
shannon_entropy() |
Shannon entropy H(P) |
renyi_entropy() |
Rényi entropy H_q(P) of order q |
tsallis_entropy() |
Tsallis entropy S_q(P) of order q |
normalized_entropy() |
Shannon entropy normalised to [0, 1] |
extropy() / shannon_extropy() |
Classical (Shannon) extropy J(P) |
renyi_extropy() |
Rényi extropy J_q(P) of order q |
max_renyi_extropy() |
Maximum Rényi extropy for n outcomes |
conditional_renyi_extropy() |
Conditional Rényi extropy J_q(Y|X) |
joint_entropy() |
Joint entropy H(X, Y) |
conditional_entropy() |
Conditional entropy H(Y|X) |
kl_divergence() |
Kullback–Leibler divergence D_KL(P ‖ Q) |
js_divergence() |
Jensen–Shannon divergence |
cross_entropy() |
Cross-entropy H(P, Q) |
Installation
Install the released version from CRAN:
install.packages("RenyiExtropy")
Or install the development version from GitHub:
# install.packages("remotes")
remotes::install_github("itsmdivakaran/RenyiExtropy")
Quick start
library(RenyiExtropy)
p <- c(0.2, 0.5, 0.3)
# Entropy measures
shannon_entropy(p) # 1.030 nats
renyi_entropy(p, q = 2) # 0.968 nats
tsallis_entropy(p, q = 2) # 0.620
normalized_entropy(p) # 0.937
# Extropy measures
extropy(p) # 0.775 nats
renyi_extropy(p, q = 2) # 0.742 nats
max_renyi_extropy(3) # maximum over uniform 3-outcome distribution
# Divergences
kl_divergence(p, c(0.3, 0.4, 0.3)) # KL(P || Q)
js_divergence(p, c(0.3, 0.4, 0.3)) # symmetric, bounded by log(2)
cross_entropy(p, c(0.3, 0.4, 0.3)) # H(P) + KL(P || Q)
# Joint / conditional measures
Pxy <- matrix(c(0.2, 0.3, 0.1, 0.4), nrow = 2, byrow = TRUE)
joint_entropy(Pxy) # H(X, Y)
conditional_entropy(Pxy) # H(Y | X)
conditional_renyi_extropy(Pxy, q = 2) # J_q(Y | X)
Limit / continuity behaviour
Functions with order parameter q automatically return their
q → 1 limit (Shannon entropy or classical extropy) when
|q − 1| < 1e-8:
renyi_entropy(p, 1) # equals shannon_entropy(p)
renyi_extropy(p, 1) # equals extropy(p)
tsallis_entropy(p, 1) # equals shannon_entropy(p)
Citation
If you use RenyiExtropy in your research, please cite:
Mahesh Divakaran, G Rajesh, Sreekumar Jayalekshmi (2024). RenyiExtropy: Entropy and Extropy Measures for Probability Distributions. R package version 0.4.0. https://CRAN.R-project.org/package=RenyiExtropy
License
MIT © Divakaran Mahesh, G Rajesh, Sreekumar Jayalekshmi