GPU Drop-Ins for Statistics on Wide Data

GPU versions of base R statistics for data with many variables, using the NVIDIA 'CUDA' toolkit: correlation matrices (Pearson, Spearman and Kendall) and their tests, the strongest correlated pairs among tens of thousands of variables without forming the matrix, association scans with covariates as in expression quantitative trait loci studies (Shabalin 2012 ), permutation tests with Westfall-Young maxT adjustment, distance matrices, hierarchical and k-means clustering, Granger causality tests, B-spline mutual information, principal components, and least-squares fits. Results agree with the base R functions they replace. Without 'CUDA' the package installs, and its functions explain how to get a 'CUDA' build. Described in Buckner et al. (2010) .

gputools

GPU drop-ins for base-R statistics on wide data. gputools computes the statistics that cost the most when there are many variables -- every pairwise correlation, distance or association test, and the clustering built on them -- on an NVIDIA GPU, and returns the numbers base R would: every function is tested against its base-R counterpart, to float accuracy in single precision and to 1e-8 or better in double.

It was written for, and mostly used in, genomics: correlating thousands of genes or regulatory regions across samples, screening pairs of variables, clustering expression profiles. The GPU pays off most there; measured against base R on 8 CPU cores, Kendall's tau ran about 2,900 times faster, Pearson correlation 16 times, distance matrices 7 times.

What it computes

Pairwise statistics over many variables -- where the GPU wins:

Models and linear algebra, with the same interface as base R:

These are there for convenience: R's own lm, glm and %*% were as fast or faster at every size we measured; for general GPU array work in R, torch is the better tool. The benchmarks vignette has the measurements.

Installing

You need an NVIDIA CUDA toolkit, 11.7 or later, to build the package, and an NVIDIA GPU and driver to run the GPU functions. Install from r-universe:

install.packages("gputools",
  repos = c("https://nullsatz.r-universe.dev", "https://cloud.r-project.org"))

or from GitHub:

# install.packages("remotes")
remotes::install_github("nullsatz/gputools")

Either way, on Linux gputools is compiled on your machine against your CUDA toolkit. The prebuilt packages r-universe offers (Windows, macOS, and its opt-in Linux binaries) are built without CUDA: they install and load, but their GPU functions cannot run.

Windows

gputools runs on Windows under WSL2 (the Windows Subsystem for Linux), which supports CUDA:

1. Install the ordinary NVIDIA driver for Windows; it also serves WSL2. Do not install a separate Linux driver inside WSL.
2. In a WSL2 Linux distribution (Ubuntu, say), install R and the CUDA toolkit for WSL, following NVIDIA's CUDA on WSL guide.
3. Install gputools inside WSL as above.

The native Windows package installs and loads, but its GPU functions cannot run.

Finding CUDA

configure finds CUDA through CUDA_HOME, CUDA_PATH, nvcc on the PATH, or /usr/local/cuda; in a conda environment set CUDA_HOME=$CONDA_PREFIX. See INSTALL for the details and the conda packages.

The package also installs and loads on a machine with no GPU (a login node, CI); gpuAvailable() says whether the GPU functions can run there.

Using it

library(gputools)

# expression of 2,000 genes (columns) in 300 samples (rows)
expr <- matrix(rnorm(300 * 2000), 300, 2000)

r <- gpuCor(expr)                       # all 2,000 x 2,000 correlations
r$coefficients[1:5, 1:5]                # as cor(expr)
r$ts[1:5, 1:5]                          # their t-statistics

tau <- gpuCor(expr, method = "kendall")$coefficients   # as cor(..., "kendall")

tree <- gpuDistClust(t(expr), clustmethod = "average") # hclust(dist(t(expr)))
groups <- cutree(tree, k = 10)

fit <- gpuLm(mpg ~ ., mtcars)           # an "lm": summary(), predict(), ...

Each function computes in single precision by default, which is many times faster than double on most consumer GPUs. Ask for double per call, or for the whole session:

gpuCor(x, precision = "double")
options(gputools.precision = "double")

(gpuLm, gpuLsfit and gpuGlm take useSingle = FALSE instead.)

Kernels are compiled for the GPU in use the first time each is needed, so the first call of a function takes a fraction of a second longer.

Whether the GPU is faster depends on the function, the data size and the hardware. On an RTX A5500, single precision, the GPU started to win at about these sizes:

For finding the strongly correlated pairs among many variables, gpuCorPairs never builds the correlation matrix: every pair with |r| >= 0.8 among 10,000 variables took 0.02 s against 6.5 s for cor() and which(), and among 50,000 variables, 0.09 s. An eQTL scan with gpuAssoc (20,000 genes, 50,000 SNPs, 500 samples, 5 covariates) took 0.37 s against 11 s for Matrix eQTL, reporting the same associations. gpuKmeans put 100,000 points into 1,000 clusters in 0.25 s, against 14.5 s for kmeans(). Permutation tests gain most: a thousand permutations of every correlation among 1,000 x 1,000 variables, maxT included, took 0.13 s against 20 s in R, and of t tests of 20,000 genes, 0.06 s against 23 s.

Below these sizes base R is faster, and each call carries a fixed cost of about 50 µs, so don't loop over small pieces: put all the variables in one matrix and make one call. vignette("benchmarks", package = "gputools") has the full measurements and the script to repeat them on your hardware.

Testing on a GPU

R CMD check runs the full test suite, but GitHub's runners have no GPU, so there the GPU tests skip. To run them on a Slurm cluster:

sbatch -p <gpu partition> ci/slurm-gpu-test.sh <conda env> [<conda env> ...]

Used in

• The FANTOM5 atlas of active enhancers across human cell types and tissues (Andersson et al., Nature 2014) clustered enhancers by their expression with gputools' hierarchical clustering on an NVIDIA GPU.

Using gputools in published work? Let us know in an issue and we will list it.

Citing

Buckner J, Wilson J, Seligman M, Athey B, Watson S, Meng F (2010). The gputools package enables GPU computing in R. Bioinformatics 26(1), 134-135. doi:10.1093/bioinformatics/btp608

Acknowledgements

Thanks to Craig Stark at UC Irvine for donating time on his lab's Mac to the original package.

License

GPL-3