Crowd Sourced System Benchmarks

Benchmark your CPU and compare against other CPUs. Also provides functions for obtaining system specifications, such as RAM, CPU type, and R version.


BuildStatus codecov.io Downloads CRAN_Status_Badge

R benchmarking made easy. The package contains a number of benchmarks, heavily based on the benchmarks at http://r.research.att.com/benchmarks/R-benchmark-25.R, for assessing the speed of your system.

Overview

A straightforward way of speeding up your analysis is to buy a better computer. Modern desktops are relatively cheap, especially compared to user time. However, it isn’t clear if upgrading your computing is worth the cost. The benchmarkme package provides a set of benchmarks to help quantify your system. More importantly, it allows you to compare your timings with other systems.

Overview

The package is on CRAN and can be installed in the usual way

install.packages("benchmarkme")

There are two groups of benchmarks:

  • benchmark_std(): this benchmarks numerical operations such as loops and matrix operations. The benchmark comprises of three separate benchmarks: prog, matrix_fun, and matrix_cal.
  • benchmark_io(): this benchmarks reading and writing a 5 / 50, MB csv file.

The benchmark_std() function

This benchmarks numerical operations such as loops and matrix operations. This benchmark comprises of three separate benchmarks: prog, matrix_fun, and matrix_cal. If you have less than 3GB of RAM (run get_ram() to find out how much is available on your system), then you should kill any memory hungry applications, e.g. firefox, and set runs = 1 as an argument.

To benchmark your system, use

library("benchmarkme")
## Increase runs if you have a higher spec machine
res = benchmark_std(runs = 3)

and upload your results

## You can control exactly what is uploaded. See details below.
upload_results(res)

You can compare your results to other users via

plot(res)

The benchmark_io() function

This function benchmarks reading and writing a 5MB or 50MB (if you have less than 4GB of RAM, reduce the number of runs to 1). Run the benchmark using

res_io = benchmark_io(runs = 3)
upload_results(res_io)
plot(res_io)

By default the files are written to a temporary directory generated

tempdir()

which depends on the value of

Sys.getenv("TMPDIR")

You can alter this to via the tmpdir argument. This is useful for comparing hard drive access to a network drive.

res_io = benchmark_io(tmpdir = "some_other_directory")

Parallel benchmarks

The benchmark functions above have a parallel option - just simply specify the number of cores you want to test. For example to test using four cores

res_io = benchmark_std(runs = 3, cores = 4)

Previous versions of the package

This package was started around 2015. However, multiple changes in the byte compiler over the last few years, has made it very difficult to use previous results. So we have to start from scratch.

The previous data can be obtained via

data(past_results, package = "benchmarkmeData")

Machine specs

The package has a few useful functions for extracting system specs:

  • RAM: get_ram()
  • CPUs: get_cpu()
  • BLAS library: get_linear_algebra()
  • Is byte compiling enabled: get_byte_compiler()
  • General platform info: get_platform_info()
  • R version: get_r_version()

The above functions have been tested on a number of systems. If they don’t work on your system, please raise GitHub issue.

Uploaded data sets

A summary of the uploaded data sets is available in the benchmarkmeData package

data(past_results_v2, package = "benchmarkmeData")

A column of this data set, contains the unique identifier returned by the upload_results() function.

What’s uploaded

Two objects are uploaded:

  1. Your benchmarks from benchmark_std or benchmark_io;
  2. A summary of your system information (get_sys_details()).

The get_sys_details() returns:

  • Sys.info();
  • get_platform_info();
  • get_r_version();
  • get_ram();
  • get_cpu();
  • get_byte_compiler();
  • get_linear_algebra();
  • installed.packages();
  • Sys.getlocale();
  • The benchmarkme version number;
  • Unique ID - used to extract results;
  • The current date.

The function Sys.info() does include the user and nodenames. In the public release of the data, this information will be removed. If you don’t wish to upload certain information, just set the corresponding argument, i.e.

upload_results(res, args = list(sys_info = FALSE))

Development of this package was supported by Jumping Rivers

News

Version 1.0.0

  • Update version focused on R 3.5 & above. Start anew. Sorry everyone

Version 0.6.1

  • Improved BLAS detection (suggested by @ck37 #15)

Version 0.6.0

  • Adding parallel benchmarks (thanks to @jknowles)
  • Since JIT has been introduced, just byte compile the package for ease of comparison.

Version 0.5.1

  • Add id_prefix to the upload function
  • Can now run benchmark_std if the package is not attached (thanks to @YvesCR)
  • Nicer version of print.bytes (thanks to @richierocks)
  • Adding parallel benchmarks (thanks to @jknowles)

Version 0.5.0

  • Bug fix in get_byte_compiler when cmpfun was used.

Version 0.4.0

  • Update to shinyapps.io example
  • Moved benchmark description to shinyapps.io
  • Additional checks on get_ram()

Version 0.3.0

  • New vignette describing benchmarks.
  • Used Sys.getpid() to try and determine the BLAS/LAPACK library (suggested by Ashley Ford).

Version 0.2.3

  • Return NA for get_cpu()/get_ram() when it isn't possible to determine CPU/RAM.

Version 0.2.2

  • First CRAN version

Version 0.2.0

  • More flexibility in plot and datatable functions - you can now specify the test you want to compare.
  • The number of cores returned by get_cpu().
  • Adding io benchmarks.
  • New shiny interface.

Version 0.1.9

  • Default log scale on y-axis (suggested by @eddelbuettel). Fixes #5.
  • Moved data sets to benchmarkmeData package.
  • New ranking function to compare results with past.

Version 0.1.8

  • Added introduction to benchmarkme vignette.
  • Adjust placement of "You" in the S3 plot.
  • Add .Machine to get_sys_details.

Version 0.1.7

  • Add locale to get_sys_details.

Version 0.1.6

  • Further RAM and Mac issues.

Version 0.1.4

  • Bug fix: Remove white space from apple RAM output (thanks to @vzemlys). Fixes #2.

Version 0.1.3

  • Add a fall-back when getting RAM - grab everything.
  • Minor: Added a horizontal line automatically generated plots.
  • Deprecated benchmark_all (use benchmark_std).

Version 0.1.2

  • First public release.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("benchmarkme")

1.0.3 by Colin Gillespie, 2 days ago


https://github.com/csgillespie/benchmarkme


Report a bug at https://github.com/csgillespie/benchmarkme/issues


Browse source code at https://github.com/cran/benchmarkme


Authors: Colin Gillespie [aut, cre]


Documentation:   PDF Manual  


GPL-2 | GPL-3 license


Imports benchmarkmeData, compiler, doParallel, dplyr, foreach, graphics, httr, Matrix, methods, parallel, tibble, utils

Suggests covr, DT, ggplot2, knitr, RcppZiggurat, rmarkdown, testthat


Imported by disk.frame.

Suggested by benchmarkmeData, mkin.


See at CRAN