Penalized Principal Machine for Sufficient Dimension Reduction

A unified, computation-friendly framework for penalized principal machines (P2M), a class of sparse sufficient dimension reduction (SDR) estimators for regression and binary classification. Principal machines (PM) estimate the central subspace by solving a family of convex-loss problems over several cutoffs; their penalized counterparts (P2M) add a row-group sparsity penalty so that dimension reduction and variable selection are performed simultaneously. All estimators are fitted by a single group coordinate descent (GCD) algorithm that accommodates least squares, logistic, asymmetric least squares, L2-hinge, hinge (support vector machine, SVM) and quantile losses, together with the least absolute shrinkage and selection operator (LASSO), the smoothly clipped absolute deviation (SCAD) penalty and the minimax concave penalty (MCP). Methods are described in Li, Artemiou and Li (2011) , Shin and Artemiou (2017) , Artemiou, Dong and Shin (2021) and Breheny and Huang (2015) .

ppmSDR: Penalized Principal Machine for Sparse Sufficient Dimension Reduction

A unified and computationally efficient R package for sparse sufficient dimension reduction (SDR) using Penalized Principal Machines (P²M) and Group Coordinate Descent (GCD).

Overview

The ppmSDR package provides a unified interface and efficient algorithms for sparse sufficient dimension reduction (SDR) in regression and classification. It implements the Penalized Principal Machine (P²M) family, which generalizes the principal support vector machine (PSVM) by allowing a wide range of convex loss functions together with modern sparsity-inducing penalties. Efficient computation is achieved via the Group Coordinate Descent (GCD) and MM-GCD algorithms, making the package scalable to large and high-dimensional data.

Key Features

• A single front-end ppm() that fits any of ten penalized principal machine estimators, selected through the loss argument, for both regression and binary classification.
• State-of-the-art sparse SDR methods: P²LSM / P²WLSM (least squares, weighted), P²LR / P²WLR (logistic, weighted), P²L2M / P²WL2M (L2-hinge, weighted), P²SVM / P²WSVM (hinge, weighted), P²QR (quantile), and P²AR (asymmetric least squares).
• Group SCAD, MCP and LASSO penalties for row-wise sparsity and variable selection.
• Fast, scalable optimization (GCD, MM-GCD), with an optional step-halving line search (line.search) that guarantees a monotonically non-increasing penalized objective for the iterative GCD solvers.
• ppm_tune() for cross-validation of the sparsity parameter.
• Bundled example datasets (boston, wdbc) and summary()/print() methods for inspecting the estimated basis and selected variables.

Installation

# Install from GitHub (requires devtools)
devtools::install_github("c16267/ppmSDR")

Repository Structure

ppmSDR/
├── R/
│   ├── ppm.R              # ppm(): unified front-end + input validation + dispatch
│   ├── ppm_tune.R         # ppm_tune(): dCor-based K-fold cross-validation
│   ├── estimators.R       # ten internal P2M solvers (selected via loss=)
│   ├── methods.R          # S3 methods: print.ppm, summary.ppm, print.ppm_tune
│   ├── utils-internal.R   # internal GCD helpers (thresholding, block-diagonal algebra,
│   │                      #   penalized objective Q(theta) and step-halving line search)
│   ├── data.R             # documentation of the bundled datasets
│   └── ppmSDR-package.R   # package-level documentation and imports
├── data/                  # boston.rda, wdbc.rda
├── man/                   # generated by roxygen2
├── vignettes/             # ppmSDR.Rmd
└── tests/                 # unit tests (testthat)

The loss-specific solvers are internal; users access them through the loss argument of ppm() rather than calling them directly.

Main Functions

Supported losses (the loss argument of ppm())

Each method supports the "grSCAD", "grMCP" and "grLasso" penalties.

Step-halving line search

The iterative GCD solvers replace the loss at each iteration by a local quadratic approximation, which does not by itself guarantee descent of the penalized P²M objective. With line.search = TRUE, ppm() accepts the candidate update θ̂ produced by a GCD step only after halving it, θ ← θ + 2^(−m)(θ̂ − θ), with the smallest m ≥ 0 (up to max.halving = 10) for which the original penalized objective does not increase, and stops at the current iterate if no such m exists. The default line.search = NULL turns the safeguard on for "logit", "wlogit", "l2svm" and "wl2svm" and off otherwise; line.search = FALSE gives the plain updates. The fitted object records iter, status, the objective path objective and the number of halvings n.halving at each iteration.

Example Usage

library(ppmSDR)

## Generate data: the first two predictors form the central subspace
set.seed(1)
n <- 1000; p <- 10
B <- matrix(0, p, 2); B[1, 1] <- B[2, 2] <- 1
x <- MASS::mvrnorm(n, rep(0, p), diag(1, p))
y <- (x %*% B[, 1] / (0.5 + (x %*% B[, 2] + 1)^2)) + 0.2 * rnorm(n)

## Penalized principal least-squares SVM (P2LSM) via the unified interface
fit <- ppm(x, y, loss = "lssvm", penalty = "grSCAD", lambda = 0.01)
round(fit$evectors[, 1:2], 3)
print(fit)
summary(fit, d = 2)

## Penalized principal asymmetric least squares (P2AR) on the same regression data
fit_ar <- ppm(x, y, loss = "asls", penalty = "grSCAD", lambda = 0.05)
round(fit_ar$evectors[, 1:2], 3)

## Binary response with a two-dimensional central subspace: P2AR and the
## penalized principal weighted logistic regression (P2WLR)
y.binary <- sign(x[, 1] + x[, 2]^3 / 3 + 0.2 * rnorm(n))
fit_arb <- ppm(x, y.binary, loss = "asls", penalty = "grSCAD", lambda = 0.2)
summary(fit_arb, d = 2)
fit_wlr <- ppm(x, y.binary, loss = "wlogit", penalty = "grSCAD", lambda = 0.005)
summary(fit_wlr, d = 2)

## Penalized principal logistic regression (P2LR): the step-halving line search
## is on by default; set line.search = FALSE for the plain iterative GCD updates
fit_lr <- ppm(x, y, loss = "logit", penalty = "grSCAD", lambda = 0.01)
fit_lr$status                        # "converged"
fit_lr$n.halving                     # number of halvings m_t at each iteration
all(diff(fit_lr$objective) <= 0)     # the penalized objective never increases

## Select lambda by cross-validation (set the seed for reproducible folds)
set.seed(1)
cv <- ppm_tune(x, y, loss = "lssvm", d = 2, n.fold = 5,
               nlambda = 10, lambda.max = 0.02)
cv$opt.lambda
summary(cv$fit, d = 2)

Bundled datasets

data(boston)   # Boston housing (continuous response: medv)
data(wdbc)     # Wisconsin Diagnostic Breast Cancer (binary: diagnosis B/M)

See the package vignette for a worked walk-through:

browseVignettes("ppmSDR")

References

• Artemiou, A. and Dong, Y. (2016). Sufficient dimension reduction via principal lq support vector machine, Electronic Journal of Statistics, 10: 783–805.
• Artemiou, A., Dong, Y. and Shin, S. J. (2021). Real-time sufficient dimension reduction through principal least squares support vector machines, Pattern Recognition, 112: 107768.
• Breheny, P. and Huang, J. (2015). Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors, Statistics and Computing, 25: 173–187.
• Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96: 1348–1360.
• Hunter, D. R. and Lange, K. (2004). A tutorial on MM algorithms, The American Statistician, 58(1): 30–37.
• Jang, H. J., Shin, S. J. and Artemiou, A. (2023). Principal weighted least square support vector machine: An online dimension-reduction tool for binary classification, Computational Statistics & Data Analysis, 187: 107818.
• Kim, B. and Shin, S. J. (2019). Principal weighted logistic regression for sufficient dimension reduction in binary classification, Journal of the Korean Statistical Society, 48(2): 194–206.
• Li, B., Artemiou, A. and Li, L. (2011). Principal support vector machines for linear and nonlinear sufficient dimension reduction, Annals of Statistics, 39(6): 3182–3210.
• Shin, J. and Shin, S. J. (2024). A concise overview of principal support vector machines and its generalization, Communications for Statistical Applications and Methods, 31(2): 235–246.
• Shin, J., Shin, S. J. and Artemiou, A. (2024). The R package psvmSDR: A unified algorithm for sufficient dimension reduction via principal machines, arXiv preprint arXiv:2409.01547.
• Shin, S. J. and Artemiou, A. (2017). Penalized principal logistic regression for sparse sufficient dimension reduction, Computational Statistics & Data Analysis, 111: 48–58.