Test Coverage for Packages

Track and report code coverage for your package and (optionally) upload the results to a coverage service like 'Codecov' < http://codecov.io> or 'Coveralls' < http://coveralls.io>. Code coverage is a measure of the amount of code being exercised by a set of tests. It is an indirect measure of test quality and completeness. This package is compatible with any testing methodology or framework and tracks coverage of both R code and compiled C/C++/FORTRAN code.


Build Status AppVeyor Build Status codecov.io CRAN version

Track test coverage for your R package and view reports locally or (optionally) upload the results to codecov or coveralls.

Installation

install.packages("covr")
 
# For devel version
devtools::install_github("r-lib/covr")

The easiest way to setup covr on Travis-CI is with usethis.

usethis::use_coverage()

Usage

A coverage report can be used to inspect coverage for each line in your package. Using report() requires shiny.

library(covr)
 
# If run with no arguments implicitly calls `package_coverage()`
report()

covr also defines an RStudio Addin, which runs report() on the active project. This can be used via the addin menu or by binding the action to a shortcut, e.g. Ctrl-Shift-C.

Interactively

# if `getwd()` is the package's directory.
package_coverage()
 
# or a package in another directory
cov <- package_coverage("/dir/lintr")
 
# view results as a data.frame
as.data.frame(cov)
 
# zero_coverage() shows only uncovered lines.
# If run within RStudio, `zero_coverage()` will open a marker pane with the
# uncovered lines.
zero_coverage(cov)

Manual Installation

Codecov

If you are already using Travis-CI or Appveyor CI add the following to your project's .travis.yml to track your coverage results over time with Codecov.

r_github_packages:
  - r-lib/covr
 
after_success:
  - Rscript -e 'covr::codecov()'

To use a different CI service or call codecov() locally you can set the environment variable CODECOV_TOKEN to the token generated on codecov.io.

Codecov currently has support for the following CI systems (* denotes support without needing CODECOV_TOKEN).

You will also need to enable the repository on Codecov.

Coveralls

Alternatively you can upload your results to Coveralls using coveralls().

r_github_packages:
  - r-lib/covr
 
after_success:
  - Rscript -e 'covr::coveralls()'

For CI systems not supported by coveralls you need to set the COVERALLS_TOKEN environment variable. It is wise to use a Secure Variable so that it is not revealed publicly.

Also you will need to turn on coveralls for your project at https://coveralls.io/repos.

Exclusions

covr supports a couple of different ways of excluding some or all of a file.

Function Exclusions

The function_exclusions argument to package_coverage() can be used to exclude functions by name. This argument takes a vector of regular expressions matching functions to exclude.

# exclude print functions
package_coverage(function_exclusions = "print\\.")
 
# exclude `.onLoad` function
package_coverage(function_exclusions = "\\.onLoad")

Line Exclusions

The line_exclusions argument to package_coverage() can be used to exclude some or all of a file. This argument takes a list of filenames or named ranges to exclude.

# exclude whole file of R/test.R
package_coverage(line_exclusions = "R/test.R")
 
# exclude lines 1 to 10 and 15 from R/test.R
package_coverage(line_exclusions = list("R/test.R" = c(1:10, 15)))
 
# exclude lines 1 to 10 from R/test.R, all of R/test2.R
package_coverage(line_exclusions = list("R/test.R" = c(1, 10), "R/test2.R"))

Exclusion Comments

In addition you can exclude lines from the coverage by putting special comments in your source code.

This can be done per line.

f1 <- function(x) {
  x + 1 # nocov
}

Or by specifying a range with a start and end.

f2 <- function(x) { # nocov start
  x + 2
} # nocov end

The patterns used can be specified by setting the global options covr.exclude_pattern, covr.exclude_start, covr.exclude_end.

FAQ

Will covr work with testthat, RUnit, etc...

Covr should be compatible with any testing package, it uses tools::testInstalledPackage() to run your packages tests.

Will covr work with alternative compilers such as ICC

Covr now supports Intel's icc compiler, thanks to work contributed by Qin Wang at Oracle.

Covr is known to work with clang versions 3.5+ and gcc version 4.2+.

If the appropriate gcov version is not on your path you can set the appropriate location with the covr.gcov options. If you set this path to "" it will turn off coverage of compiled code.

options(covr.gcov = "path/to/gcov")

How does covr work?

covr tracks test coverage by modifying a package's code to add tracking calls to each call.

The vignette vignettes/how_it_works.Rmd contains a detailed explanation of the technique and the rationale behind it.

You can view the vignette from within R using

vignette("how_it_works", package = "covr")

Why can't covr run during R CMD check

Because covr modifies the package code it is possible there are unknown edge cases where that modification affects the output. In addition when tracking coverage for compiled code covr compiles the package without optimization, which can modify behavior (usually due to package bugs which are masked with higher optimization levels).

Alternative Coverage Tools

News

3.0.1

  • Add an RStudio Addin for running a coverage report.

  • Never use mcexit fix on windows (#223).

  • Fix for a performance regression in parsing and reading parse data (#274).

  • Fix switch support for packages, which was broken due to a bug in how parse data is stored in packages.

  • Improve behavior of switch coverage, it now supports default values and fall through properly.

  • Add -p flag to gcov command to preserve file paths. Fixes a bug where gcov output didn't get reported when multiple compiled source files had the same name (#271, @patperry)

3.0.0

  • The covr license has been changed to GPL-3.
  • Set environment variable R_COVR=true when covr is running (#236, #268).
  • Made the gather-and-merge-results step at the end of package_coverage() more memory efficient (#226, @HenrikBengtsson).
  • Support code coverage with icc (#247, @QinWang).

2.2.2

  • filter_not_package_files() now works if a source reference does not have a filename (#254, @hughjonesd).
  • Fix test broken with xml2 v1.1.0
  • Filter out non-local filenames from results (#237).
  • Vignette rewrite / improvements (#229, @CSJCampbell).
  • Fix code that returns structure(NULL, *) which is deprecated in R 3.4.0 (#260, #261, @renkun-ken).

2.2.1

  • Fix test broken with DT 0.2

2.2.0

  • Fix tests broken with updated htmlwidgets
  • Change report tab title based on filename (Chen Liang).
  • Add support for cobertura XML output (@wligtenberg).
  • Add mcparallel support by patching mcparallel:::mcexit() automatically for packages using parallel (#195, @kforner).

2.1.0

  • Add support for GitLab CI (#190, @enbrown).
  • Update exclusion documentation to include line_exclusions and function exclusions (#191).
  • Support coverage of R6 methods (#174).
  • Explicitly set default packages (including methods) (#183, #180)
  • Set R_LIBS and R_LIBS_SITE as well as R_LIBS_USER (#188).
  • Automatically exclude RcppExport files (#170).
  • Memoised and Vectorized functions now able to be tracked.

2.0.1

  • Support for filtering by function as well as line.
  • Now tracks coverage for RC methods
  • Rewrote loading and saving to support parallel code and tests including quit() calls.
  • Made passing code to function_coverage() and package_coverage() not use non-standard evaluation.
  • NULL statements are analyzed for coverage (#156, @krlmlr).
  • Finer coverage analysis for brace-less if, while and for statements (#154, @krlmlr).
  • Run any combination of coverage types (#104, #133)
  • Remove inconsistencies in line counts between shiny app and services (#129)
  • Include header files in gcov output (#112)
  • Add support for C++11 (#131)
  • Always clean gcov files even on failure (#108)
  • zero_coverage works with RStudio markers (#119)
  • Remove the devtools dependency

1.3.0

  • Set .libPaths() in subprocess to match those in calling process (#140, #147).
  • Move devtools dependency to suggests, only needed on windows
  • move htmltools to suggests

Initial Release

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("covr")

3.1.0 by Jim Hester, 2 months ago


https://github.com/r-lib/covr


Report a bug at https://github.com/r-lib/covr/issues


Browse source code at https://github.com/cran/covr


Authors: Jim Hester [aut, cre], Willem Ligtenberg [ctb], Kirill Müller [ctb], Henrik Bengtsson [ctb], Steve Peak [ctb], Kirill Sevastyanenko [ctb], Jon Clayden [ctb], Robert Flight [ctb], Eric Brown [ctb], Brodie Gaslam [ctb], Will Beasley [ctb], Robert Krzyzanowski [ctb], Markus Wamser [ctb], Karl Forner [ctb], Gergely Daróczi [ctb], Jouni Helske [ctb], Kun Ren [ctb], Jeroen Ooms [ctb], Ken Williams [ctb], Chris Campbell [ctb], David Hugh-Jones [ctb], Qin Wang [ctb]


Documentation:   PDF Manual  


GPL-3 license


Imports digest, stats, utils, jsonlite, rex, httr, crayon, withr

Depends on methods

Suggests R6, knitr, rmarkdown, shiny, htmltools, htmlwidgets, DT, testthat, rstudioapi, xml2, parallel, memoise, mockery


Imported by GRANBase, goodpractice, pkgnet.

Suggested by ABCoptim, ALA4R, AMR, BTYDplus, CholWishart, ClusterR, CompGLM, CoordinateCleaner, DBI, DataExplorer, DepthProc, DiagrammeR, EML, EcoGenetics, EcoNetGen, EmissV, EpiModel, FLightR, FRK, FSelectorRcpp, FTRLProximal, FedData, GGIR, GSODR, GeoMongo, HARtools, Inflation, KernelKnn, LendingClub, MFPCA, MODISTools, MTSYS, Momocs, NLMR, ODEsensitivity, OpenImageR, PROscorer, PROscorerTools, PlackettLuce, RGF, RInno, RNeXML, RSauceLabs, RcppThread, RcppXPtrUtils, RepoGenerator, Rilostat, Rpolyhedra, RxODE, SciViews, SmartEDA, SpaDES.core, TrajDataMining, USAboundaries, actogrammr, aemo, aire.zmvm, alfred, angstroms, anomalize, antaresProcessing, antaresViz, ashr, auk, autothresholdr, awsjavasdk, ballr, behavr, belg, bench, betalink, bgmfiles, bigKRLS, bigrquery, bikedata, binomen, biofiles, biolink, bizdays, blastula, blob, blorr, bnclassify, bomrang, breathtestcore, breathteststan, brglm2, broom, bsplus, cRegulome, callr, canvasXpress, cattonum, ccafs, cdcfluview, cellranger, checkr, chunkR, cifti, cleanNLP, cli, clipr, clusteredinterference, clustree, coalitions, codemetar, coefplot, colorednoise, colorplaner, comorbidity, comparer, comperank, comperes, conflicted, congressbr, covTestR, covequal, cowplot, cpr, cptcity, cranlike, cranly, crossword.r, crplyr, crunch, crunchy, cvequality, damr, darksky, data.world, dataonderivatives, datapasta, datasauRus, datastepr, datr, daymetr, dbplot, dbplyr, debugme, decido, dendextend, desc, descriptr, detrendr, devtools, diceR, disposables, dkanr, dockerfiler, docuSignr, document, docxtools, dparser, dplyr, dplyr.teradata, dqrng, dragulaR, driftR, dsrTest, dtplyr, dub, duckduckr, dynprog, ecotox, edgarWebR, eixport, elmNNRcpp, epicontacts, epubr, errorist, errorlocate, essurvey, eulerr, evaluator, exampletestr, exuber, fakemake, fastDummies, fedreporter, fiery, filelock, filesstrings, fixerapi, fmbasics, fmdates, fmriqa, foghorn, foolbox, forcats, formattable, formulize, fs, fslr, funData, fuzzyjoin, fuzzywuzzyR, gastempt, gcite, geex, gender, geofacet, geogrid, geojsonR, geojsonlint, geojsonsf, getCRUCLdata, getTBinR, ggedit, ggetho, gghalfnorm, ggplot2, ggridges, ggspatial, gh, gibble, gifti, glassdoor, glue, goldi, googleAnalyticsR, googleAuthR, googleCloudStorageR, googleComputeEngineR, googlePolylines, googlePublicData, googledrive, googlesheets, graphframes, groupedstats, gtable, hansard, haven, healthcareai, heatmaply, hesim, httptest, httr, huxtable, iheatmapr, ijtiff, iml, incidence, infer, inferr, inplace, ipeaData, ipumsr, isdparser, itunesr, joineR, jpndistrict, kdensity, kdtools, kendallRandomWalks, kerasR, kernelboot, keyholder, keyring, knitrProgressBar, komadown, landscapetools, languagelayeR, lans2r, lazyeval, leaflet.minicharts, lexRankr, librarysnapshot, likelihoodExplore, lime, lingtypology, listWithDefaults, liteq, live, loder, lolR, lubridate, lutz, lwgeom, manipulateWidget, mapview, mason, mathpix, matlabr, mcmcr, mcparallelDo, memery, memoise, messaging, mfGARCH, mgsub, miniCRAN, mleap, mljar, mlrMBO, mmapcharr, mockr, modeldb, modelr, moderndive, modifiedmk, mscstts, mudata2, n1qn1, nandb, naniar, naptime, natserv, ncdump, ncmeta, net.security, netdiffuseR, neurobase, neurohcp, neverhpfilter, nitrcbot, nmslibR, normalr, oai, oasis, odbc, olsrr, openadds, opensensmapr, optiRum, optim.functions, oshka, outbreaks, owmr, pacotest, paletteer, palr, parlitools, parsedate, passport, pastecs, pavo, pccc, pcr, pds3, petro.One, phenocamr, pivot, pivotaltrackR, pkgbuild, pkgconfig, pkgdown, pkgload, pltesim, plyr, pmatch, pmc, pollen, popEpi, postlightmercury, ppitables, prettycode, primefactr, processx, prodigenr, productplots, prof.tree, projector, proto, proustr, psycho, ptstem, purrr, purrrlyr, quickPlot, qwraps2, r6extended, rAltmetric, rODE, rPackedBar, rWishart, ratelimitr, rbgm, rcartocolor, rclimateca, rdflib, readr, readxl, rebird, recipes, refuge, reinforcelearn, rematch, rematch2, reprex, reproducible, reqres, reshape2, rex, rfm, rgeopat2, rhmmer, rhub, riingo, rintrojs, rjsonapi, rlang, rmapzen, rmonad, rnpn, rnr, roadoi, robotstxt, rodham, routr, roxygen2, rpdo, rpicosat, rppo, rrd, rsimsum, rsinaica, rslp, rsnps, rstack, rstantools, rtika, rtrek, rtweet, ruler, rvest, rvg, sGMRFmix, sabre, sankey, scales, scopr, sealr, searcher, secret, securitytxt, segregation, seleniumPipes, semver, sergeant, sessioninfo, sf, sfdct, shades, shinyShortcut, shinyWidgets, showimage, sicegar, sigmoid, skeletor, skimr, skynet, smoothr, snakecase, spMaps, spaero, spark, sparseinv, spatialwarnings, spbabel, spdplyr, spduration, spex, spiderbar, spind, splashr, sss, sstModel, stapler, statesRcontiguous, statnet.common, stemmatology, stopwords, strider, stringr, sugrrants, surveydata, svDialogs, svGUI, svMisc, svglite, swatches, tRophicPosition, tabr, tabularaster, tactile, tailr, tauturri, tcgsaseq, telegram.bot, templates, tensorr, testthat, texPreview, text2vec, textTinyR, textreuse, thankr, themetagenomics, theseus, thinkr, tibble, tibbletime, tidyLPA, tidygraph, tidyhydat, tidyposterior, tidyr, tidyselect, tidystringdist, tiler, tinyProject, tmap, tokenizers, toxboot, tracer, trip, tsibble, uncmbb, unitizer, unjoin, updog, useful, usethis, utilsIPEA, validatetools, valr, vein, vembedr, vimp, vinereg, viridis, viridisLite, voteogram, vqtl, wdm, webdriver, whereport, whoami, widyr, withr, workflowr, xml2, xmlparsedata, yardstick, yesno, zFactor, zeitgebr, zip.


See at CRAN