Neural Networks Made Algebraic

Build, compose, stack, sum, and realize feed-forward neural networks as algebraic objects. Implements the calculus and constructive approximations described by Rafi, Padgett, and Nakarmi (2024) , Grohs, Hornung, Jentzen, et al. (2023) , and Jentzen, Kuckuck, and von Wurstemberger (2023) . Includes neural network polynomials, transcendental-function approximations, multidimensional maximum convolution, and vectorized batch realization.

nnR

Neural networks made algebraic.

nnR represents a feed-forward neural network as an ordered list of weight matrices and bias vectors. It implements composition, stacking, sums, scalar multiplication, and constructive ReLU approximations without a training step. The calculus follows Rafi, Padgett, and Nakarmi (2024), building on Grohs, Hornung, Jentzen, and Zimmermann (2023) and Jentzen, Kuckuck, and von Wurstemberger (2023).

Highlights

• Algebraic operations: comp(), stk(), stk_many(), nn_sum(), nn_sum_many(), slm(), and srm().
• Constructive approximations: Sqr(), Prd(), Pwr(), and the general neural-network polynomial Pnm().
• Taylor families: Xpn(), Csn(), and Sne().
• Norm, maximum, multidimensional maximum-convolution, and trapezoidal-rule networks.
• Fast realization of one point or a column batch, plus reusable realized functions with realize_nn().
• Structural validation with is_nn() and validate_nn().

The approximation theorems cited above use ReLU activation. Sigmoid() and Tanh() are also available for exploration, but do not inherit those ReLU guarantees.

Quick start

library(nnR)

# c0 + c1*x + c2*x^2, with coefficients in ascending power order
polynomial <- Pnm(c(1, -2, 0.5), q = 3, eps = 0.2)

# A matrix is evaluated as a batch with one sample per column.
x <- matrix(seq(-1, 1, length.out = 9), nrow = 1)
observed <- inst(polynomial, ReLU, x)
expected <- 1 - 2 * x + 0.5 * x^2
cbind(observed = c(observed), expected = c(expected))

# Validate once and reuse the resulting function.
f <- realize_nn(polynomial, ReLU)
f(x)

Composition respects realization:

inner <- Aff(2, 1)
outer <- Aff(-3, 4)
z <- 0.5

inst(comp(outer, inner), ReLU, z)
inst(outer, ReLU, inst(inner, ReLU, z))

Maximum convolution accepts one sample per column in any dimension:

X <- matrix(c(0, 0, 1, 0, 0, 1, 1, 1), nrow = 2)
y <- colSums(X)
approximant <- MC(X, y, L = 1)
inst(approximant, ReLU, X)

See the package vignette and function reference for the full calculus and its mathematical definitions.