Functions that Apply to Rows and Columns of Matrices (and to Vectors)

High-performing functions operating on rows and columns of matrices, e.g. col / rowMedians(), col / rowRanks(), and col / rowSds(). Functions optimized per data type and for subsetted calculations such that both memory usage and processing time is minimized. There are also optimized vector-based methods, e.g. binMeans(), madDiff() and weightedMedian().


News

Package: matrixStats

Version: 0.53.0 [2018-01-23]

NEW FEATURES:

o Functions sum2(x) and means2(x) now accept also logical input 'x', which corresponds to using as.integer(x) but without the need for neither coercion nor internal extra copies. With sum2(x, mode = "double") it is possible to count number of TRUE elements beyond 2^31-1, which base::sum() does not support.

o Functions col-/rowSums2() and col-/rowMeans2() now accept also logical input 'x'.

o Function binMeans(y, x, bx) now accepts logical 'y', which corresponds to to using as.integer(y) but without the need for coercion to integer.

o Functions col-/rowTabulates(x) now support logical input 'x'.

o Now count() can count beyond 2^31-1.

o allocVector() can now allocate long vectors (longer than 2^31-1).

o Now sum2(x, mode = "integer") generates a warning if typeof(x) == "double" asking if as.integer(sum2(x)) was intended.

o Inspired by Hmisc::wtd.var(), when sum(w) <= 1, now weightedVar(x, w) produces an informative warning that the estimate is invalid.

CODE REFACTORING:

o Harmonized the ordering of the arguments of colAvgsPerColSet() with that of rowAvgsPerColSet().

BUG FIXES:

o col-/rowLogSumExp() could core dump R for "large" number of columns/rows. Thanks Brandon Stewart at Princeton University for reporting on this.

o count() beyond 2^31-1 would return invalid results.

o Functions col-/rowTabulates(x) did not count missing values.

o indexByRow(dim, idxs) would give nonsense results if 'idxs' had indices greater than prod(dim) or non-positive indices; now it gives an error.

o indexByRow(dim) would give nonsense results when prod(dim) >= 2^31; now it gives an informative error.

o col-/rowAvgsPerColSet() would return vector rather than matrix if nrow(X) <= 1. Thanks to Peter Hickey (Johns Hopkins University) for troubleshooting and providing a fix.

DEPRECATED AND DEFUNCT:

o Previously deprecated meanOver() and sumOver() are defunct. Use mean2() and sum2() instead.

o Previously deprecated weightedVar(x, w, method = "0.14.2") is defunct.

o Dropped previously defunct weightedMedian(..., ties = "both").

o Dropped previously defunct argument 'centers' for col-/rowMads(). Use 'center' instead.

o Dropped previously defunct argument 'flavor' of colRanks() and rowRanks().

Version: 0.52.2 [2017-04-13]

BUG FIXES:

o Several of the row- and column-based functions would core dump R if the matrix was of a data type other than logical, integer, or numeric, e.g. character or complex. This is now detected and an informative error is produced instead. Similarly, some vector-based functions could potentially core dump R or silently return a nonsense result. Thank you Hervé Pagès, Bioconductor Core, for the report.

DEPRECATED AND DEFUNCT:

o rowVars(..., method = "0.14.2") that was added for very unlikely needs of backward compatibility of an invalid degree-of-freedom term is deprecated.

Version: 0.52.1 [2017-04-04]

BUG FIXES:

o The package test on matrixStats:::benchmark() tried to run even if not all suggested packages were available.

Version: 0.52.0 [2017-04-03]

SIGNIFICANT CHANGES:

o Since anyNA() is a built-in function since R (>= 3.1.0), please use that instead of anyMissing() part of this package. The latter will eventually be deprecated. For consistency with the anyNA() name, colAnyNAs() and rowAnyNAs() are now also available replacing the identically colAnyMissings() and rowAnyMissings() functions, which will also be deprecated in a future release.

o meanOver() was renamed to mean2() and sumOver() was renamed to sum2().

NEW FEATURES:

o Added colSums2() and rowSums2() which work like colSums() and rowSums() of the base package but also supports efficient subsetting via optional arguments 'rows' and 'cols'.

o Added colMeans2() and rowMeans2() which work like colMeans() and rowMeans() of the base package but also supports efficient subsetting via optional arguments 'rows' and 'cols'.

o Functions colDiffs() and rowDiffs() gained argument 'dim.'.

o Functions colWeightedMads() and rowWeightedMads() gained arguments 'constant' and 'center'. The current implementation only support scalars for these arguments, which means that the same values are applied to all columns and rows, respectively. In previous version a hard-to-understand error would be produced if 'center' was of length greater than one; now an more informative error message is given.

o Package is now silent when loaded; it no longer displays a startup message.

SOFTWARE QUALITY:

o Continuous-integration testing is now also done on macOS, in addition to Linux and Windows.

o ROBUSTNESS: Package now registers the native API using also R_useDynamicSymbols().

CODE REFACTORING:

o Cleaned up native low-level API and renamed native source code files to make it easier to navigate the native API.

o Now using roxygen for help and NAMESPACE (was R.oo::Rdoc).

BUG FIXES:

o rowAnys(x) on numeric matrices 'x' would return rowAnys(x == 1) and not rowAnys(x != 0). Same for colAnys(), rowAlls(), and colAlls(). Thanks Richard Cotton for reporting on this.

o sumOver(x) and meanOver(x) would incorrectly return -Inf or +Inf if the intermediate sum would have that value, even if one of the following elements would turn the intermediate sum into NaN or NA, e.g. with 'x' as c(-Inf, NaN), c(-Inf, +Inf), or c(+Inf, NA).

o WORKAROUND: Benchmark reports generated by matrixStats:::benchmark() would use any custom R prompt that is currently set in the R session, which may not render very well. Now it forces the prompt to be the built-in "> " one.

DEPRECATED AND DEFUNCT:

o The package API is only intended for matrices and vectors of type numeric, integer and logical. However, a few functions would still return if called with a data.frame. This was never intended to work and is now an error. Specifically, functions colAlls(), colAnys(), colProds(), colQuantiles(), colIQRs(), colWeightedMeans(), colWeightedMedians(), and colCollapse() now produce warnings if called with a data.frame. Same for the corresponding row- functions. The use of a data.frame will be produce an error in future releases.

o meanOver() and sumOver() are deprecated because they were renamed to mean2() and sum2(), respectively.

o Previously deprecated (and ignored) argument 'flavor' of colRanks() and rowRanks() is now defunct.

o Previously deprecated support for passing non-vector, non-matrix objects to rowAlls(), rowAnys(), rowCollapse(), and the corresponding column-based versions are now defunct. Likewise, rowProds(), rowQuantiles(), rowWeightedMeans(), rowWeightedMedians(), and the corresponding column-based versions are also defunct. The rationale for this is to tighten up the identity of the matrixStats package and what types of input it accepts. This will also help optimize the code further.

Version: 0.51.0 [2016-10-08]

PERFORMANCE AND MEMORY:

o SPEEDUP / CLEANUP: rowMedians() and colMedians() are now plain functions. They were previously S4 methods (due to a Bioconductor legacy). The package no longer imports the methods package.

o SPEEDUP: Now native API is formally registered allowing for faster lookup of routines from R.

Version: 0.50.2 [2016-04-24]

BUG FIXES:

o Package now installs on R (>= 2.12.0) as claimed. Thanks to Mikko Korpela at Aalto University School of Science, Finland, for troubleshooting and providing a fix.

o logSumExp(c(-Inf, -Inf, ...)) would return NaN rather than -Inf. Thanks to Jason Xu (University of Washington) for reporting and Brennan Vincent for troubleshooting and contributing a fix.

Version: 0.50.1 [2015-12-14]

BUG FIXES:

o The Undefined Behavior Sanitizer (UBsan) reported on a memcall(src, dest, 0) call when dest == null. Thanks to Brian Ripley and the CRAN check tools for catching this. We could reproduce this with gcc 5.1.1 but not with gcc 4.9.2.

Version: 0.50.0 [2015-12-13]

NEW FEATURES:

o MAJOR FEATURE UPDATE: Subsetting arguments 'idxs', 'rows' and 'cols' were added to all functions such that the calculations are performed on the requested subset while avoiding creating a subsetted copy, i.e. rowVars(x, cols = 4:6) is a much faster and more memory efficient version than rowVars(x[, 4:6]) and even yet more efficient than apply(x, MARGIN = 1L, FUN = var). These features were added by Dongcan Jiang, Peking University, with support from the Google Summer of Code program. A great thank you to Dongcan and to Google for making this possible.

Version: 0.15.0 [2015-10-26]

NEW FEATURES:

o CONSISTENCY: Now all weight arguments ('w' and 'W') default to NULL, which corresponds to uniform weights.

CODE REFACTORING:

o ROBUSTNESS: Importing 'stats' functions in namespace.

BUG FIXES:

o weightedVar(x, w) used the wrong bias correction factor resulting in an estimate that was tau too large, where tau = ((sum(w) - 1) / sum(w)) / ((length(w) - 1) / length(w)). Thanks to Wolfgang Abele for reporting and troubleshooting on this.

o weightedVar(x) with length(x) = 1 returned 0 no NA. Same for weightedSd().

o weightedMedian(x, w = NA_real_) returned 'x' rather than NA_real_. This only happened for length(w) = 1.

o allocArray(dim) failed for prod(dim) >= .Machine$integer.max.

DEPRECATED AND DEFUNCT:

o CLEANUP: Defunct argument 'centers' for col-/rowMads(); use 'center'.

o weightedVar(x, w, method = "0.14.2") is deprecated.

Version: 0.14.2 [2015-06-23]

BUG FIXES:

o x_OP_y() and t_tx_OP_y() would return garbage on Solaris SPARC (and possibly other architectures as well) when input was integer and had missing values.

Version: 0.14.1 [2015-06-17]

BUG FIXES:

o product(x, na.rm = FALSE) for integer 'x' with both zeros and NAs returned zero rather than NA.

o weightedMean(x, w, na.rm = TRUE) did not handle missing values in 'x' properly, if it was an integer. It would also return NaN if there were weights 'w' with missing values, whereas stats::weighted.mean() would skip such data points. Now weightedMean() does the same.

o (col|row)WeightedMedians() did not handle infinite weights as weightedMedian() does.

o x_OP_y(x, y, OP, na.rm = FALSE) returned garbage iff 'x' or 'y' had missing values of type integer.

o rowQuantiles() and rowIQRs() did not work for single-row matrices. Analogously for the corresponding column functions.

o rowCumsums(), rowCumprods() rowCummins(), and rowCummaxs(), accessed out-of-bound elements for Nx0 matrices where N > 0. The corresponding column methods has similar memory errors for 0xK matrices where K > 0.

o anyMissing(list(NULL)) returned NULL; now FALSE.

o rowCounts() resulted in garbage if a previous column had NAs (because it forgot to update index kk in such cases).

o rowCumprods(x) handled missing values and zeros incorrectly for integer 'x (not double); a zero would trump an existing missing value causing the following cumulative products to become zero. It was only a zero that trumped NAs; any other integer would work as expected. Note, this bug was not in colCumprods().

o rowAnys(x, value, na.rm = FALSE) did not handle missing values in a numeric 'x' properly. Similarly, for non-numeric and non-logical 'x', row- and colAnys(), row- and colAlls(), anyValue() and allValue() did not handle when 'value' was a missing value.

o All of the above bugs were identified and fixed by Dongcan Jiang (Peking University, China), who also added corresponding unit tests.

Version: 0.14.0 [2015-02-13]

SIGNIFICANT CHANGES:

o CLEANUP: anyMissing() is no longer an S4 generic. This was done as part of the migration of making all functions of matrixStats plain R functions, which minimizes calling overhead and it will also allow us to drop 'methods' from the package dependencies. I've scanned all CRAN and Bioconductor packages depending on matrixStats and none of them relied on anyMissing() dispatching on class, so hopefully this move has little impact. The only remaining S4 methods are now colMedians() and rowMedians().

NEW FEATURES:

o CONSISTENCY: Renamed argument 'centers' of col-/rowMads() to 'center'. This is consistent with col-/rowVars().

o CONSISTENCY: col-/rowVars() now use na.rm = FALSE as the default (na.rm = TRUE was mistakenly introduced as the default in v0.9.7).

PERFORMANCE AND MEMORY:

o SPEEDUP: The check for user interrupts at the C level is now done less frequently of the functions. It does every k:th iteration, where k = 2^20, which is tested for using (iter % k == 0). It turns out, at least with the default compiler optimization settings that I use, that this test is 3 times faster if k = 2^n where n is an integer. The following functions checks for user interrupts: logSumExp(), (col|row)LogSumExps(), (col|row)Medians(),, (col|row)Mads(), (col|row)Vars(), and (col|row)Cum(Min|Max|prod|sum)s().

o SPEEDUP: logSumExp(x) is now faster if 'x' does not contain any missing values. It is also faster if all values are missing or the maximum value is +Inf - in both cases it can skip the actual summation step.

SOFTWARE QUALITY:

o ROBUSTNESS/TESTS: Package tests cover 96% of the code (was 91%).

CODE REFACTORING:

o CLEANUP: Package no longer depends on R.methodsS3.

BUG FIXES:

o all() and any() flavored methods on non-numeric and non-logical (e.g. character) vectors and matrices with na.rm = FALSE did not give results consistent with all() and any() if there were missing values. For example, with x <- c("a", NA, "b") we have all(x == "a") == FALSE and any(x == "a") == TRUE whereas our corresponding methods would return NA in those cases. The methods fixed are allValue(), anyValue(), col-/rowAlls(), and col-/rowAnys(). Added more package tests to cover these cases.

o logSumExp(x, na.rm = TRUE) would return NA if all values were NA and length(x) > 1. Now it returns -Inf for all length(x):s.

Version: 0.13.1 [2015-01-21]

BUG FIXES:

o diff2() with differences >= 3 would read spurious values beyond the allocated memory. This error, introduced in 0.13.0, was harmless in the sense that the returned value was unaffected and still correct. Thanks to Brian Ripley and the CRAN check tools for catching this. I could reproduce it locally with 'valgrind'.

Version: 0.13.0 [2015-01-20]

SIGNIFICANT CHANGES:

o SPEEDUP/CLEANUP: Turned several S3 and S4 methods into plain R functions, which decreases the overhead of calling the functions. After this there are no longer any S3 methods. Remaining S4 methods are anyMissing() and rowMedians().

NEW FEATURES:

o Added weightedMean(), which is ~10 times faster than stats::weighted.mean().

o Added count(x, value) which is a notably faster than sum(x == value). This can also be used to count missing values etc.

o Added allValue() and anyValue() for all(x == value) and any(x == value).

o Added diff2(), which is notably faster than base::diff() for vectors, which it is designed for.

o Added iqrDiff() and (col|row)IqrDiffs().

o CONSISTENCY: Now rowQuantiles(x, na.rm = TRUE) returns all NAs for rows with missing values. Analogously for colQuantiles(), colIQRs(), rowIQRs() and iqr(). Previously, all these functions gave an error saying missing values are not allowed.

o COMPLETENESS: Added corresponding "missing" vector functions for already existing column and row functions. Similarly, added "missing" column and row functions for already existing vector functions, e.g. added iqr() and count() to complement already existing (col|row)IQRs() and (col|row)Counts() functions.

o ROBUSTNESS: Now column and row methods give slightly more informative error messages if a data.frame is passed instead of a matrix.

DOCUMENTATION:

o DOCUMENTATION: Added vignette summarizing available functions.

PERFORMANCE AND MEMORY:

o SPEEDUP: (col|row)Diffs() are now implemented in native code and notably faster than diff() for matrices.

o SPEEDUP: Made binCounts() and binMeans() a bit faster.

o SPEEDUP: Implemented weightedMedian() in native code, which made it ~3-10 times faster. Dropped support for ties = "both", because it would have to return two values in case of ties, which made the API unnecessarily complicated. If really needed, then call the function twice with ties = "min" and ties = "max".

o SPEEDUP: (col|row)Anys() and (col|row)Alls() is now notably faster compared to previous versions.

CODE REFACTORING:

o CLEANUP: In the effort of migrating anyMissing() into a plain R function, the specific anyMissing() implementations for data.frame:s and and list:s were dropped and is now handled by anyMissing() for "ANY", which is the only S4 method remaining now. In a near future release, this remaining "ANY" method will turned into a plain R function and the current S4 generic will be dropped. We know of know CRAN and Bioconductor packages that relies on it being a generic function. Note also that since R (>= 3.1.0) there is a base::anyNA() function that does the exact same thing making anyMissing() obsolete.

BUG FIXES:

o weightedMedian(..., ties = "both") would give an error if there was a tie. Added package test for this case.

DEPRECATED AND DEFUNCT:

o weightedMedian(..., ties = "both") is now defunct.

Version: 0.12.2 [2014-12-07]

BUG FIXES:

o CODE FIX: The native code for product() on integer vector incorrectly used C-level abs() on intermediate values despite those being doubles requiring fabs(). Despite this, the calculated product would still be correct (at least when validated on several local setups as well as on the CRAN servers). Again, thanks to Brian Ripley for pointing out another invalid integer-double coersion at the C level.

DEPRECATED AND DEFUNCT:

o weightedMedian(..., interpolate = FALSE, ties = "both") is defunct.

Version: 0.12.1 [2014-12-06]

SOFTWARE QUALITY:

o ROBUSTNESS: Updated package tests to check methods in more scenarios, especially with both integer and numeric input data.

BUG FIXES:

o (col|row)Cumsums(x) where 'x' is integer would return garbage for columns (rows) containing missing values.

o rowMads(x) where 'x' is numeric (not integer) would give incorrect results for rows that had an odd number of values (no ties). Analogously issues with colMads(). Added package tests for such cases too. Thanks to Brian Ripley and the CRAN check tools for (yet again) catching another coding mistake. Details: This was because the C-level calculation of the absolute value of residuals toward the median would use integer-based abs() rather than double-based fabs(). Now it fabs() is used when the values are double and abs() when they are integers.

Version: 0.12.0 [2014-12-05]

o Submitted to CRAN.

Version: 0.11.9 [2014-11-26]

NEW FEATURES:

o Added (col|row)Cumsums(), (col|row)Cumprods(), (col|row)Cummins(), and (col|row)Cummaxs().

BUG FIXES:

o (col|row)WeightedMeans() with all zero weights gave mean estimates with values 0 instead of NaN.

Version: 0.11.8 [2014-11-25]

PERFORMANCE AND MEMORY:

o SPEEDUP: Implemented (col|row)Mads(), (col|row)Sds() and (col|row)Vars() in native code.

o SPEEDUP: Made (col|row)Quantiles(x) faster for 'x' without missing values (and default type = 7L quantiles). It should still be implemented in native code though.

o SPEEDUP: Made rowWeightedMeans() faster.

BUG FIXES:

o (col|row)Medians(x) when 'x' is integer would give invalid median values in case (a) it was calculated as the mean of two values ("ties"), and (b) the sum of those values where greater than .Machine$integer.max. Now such ties are calculated using floating point precision. Add lots of package tests.

Version: 0.11.6 [2014-11-16]

PERFORMANCE AND MEMORY:

o SPEEDUP: Now (col|row)Mins(), (col|row)Maxs() and (col|row)Ranges() are implemented in native code providing a significant speedup.

o SPEEDUP: Now colOrderStats() also is implemented in native code, which indirectly makes colMins(), colMaxs() and colRanges() faster.

o SPEEDUP: colTabulates(x) no longer uses rowTabulates(t(x)).

o SPEEDUP: colQuantiles(x) no longer uses rowQuantiles(t(x)).

DEPRECATED AND DEFUNCT:

o CLEANUP: Argument 'flavor' of (col|row)Ranks() is now ignored.

Version: 0.11.5 [2014-11-15]

SIGNIFICANT CHANGES:

o (col|row)Prods() now uses default method = "direct" (was "expSumLog").

PERFORMANCE AND MEMORY:

o SPEEDUP: Now colCollapse(x) no longer utilizes rowCollapse(t(x)). Added package tests for (col|row)Collapse().

o SPEEDUP: Now colDiffs(x) no longer uses rowDiffs(t(x)). Added package tests for (col|row)Diffs().

o SPEEDUP: Package no longer utilizes match.arg() due to its overhead; methods sumOver(), (col|row)Prods() and (col|row)Ranks() were updated.

Version: 0.11.4 [2014-11-14]

NEW FEATURES:

o Added support for vector input to several of the row- and column methods as long as the "intended" matrix dimension is specified via argument 'dim'. For instance, rowCounts(x, dim = c(nrow, ncol)) is the same as rowCounts(matrix(x, nrow, ncol)), but more efficient since it avoids creating/allocating a temporary matrix.

PERFORMANCE AND MEMORY:

o SPEEDUP: Now colCounts() is implemented in native code. Moreover, (col|row)Counts() are now also implemented in native code for logical input (previously only for integer and double input). Added more package tests and benchmarks for these functions.

Version: 0.11.3 [2014-11-11]

SIGNIFICANT CHANGES:

o Turned sdDiff(), madDiff(), varDiff(), weightedSd(), weightedVar() and weightedMad() into plain functions (were generic functions).

CODE REFACTORING:

o Removed unnecessary usage of '::'.

Version: 0.11.2 [2014-11-09]

SIGNIFICANT CHANGES:

o SPEEDUP: Implemented indexByRow() in native code and it is no longer a generic function, but a regular function, which is also faster to call. The first argument of indexByRow() has been changed to 'dim' such that one should use indexByRow(dim(X)) instead of indexByRow(X) as in the past. The latter form is still supported, but deprecated.

NEW FEATURES:

o Added allocVector(), allocMatrix() and allocArray() for faster allocation numeric vectors, matrices and arrays, particularly when filled with non-missing values.

DEPRECATED AND DEFUNCT:

o Calling indexByRow(X) with a matrix 'X' is deprectated. Instead call it with indexByRow(dim(X)).

Version: 0.11.1 [2014-11-07]

NEW FEATURES:

o Better support for long vectors.

o PRECISION: Using greater floating-point precision in more internal intermediate calculations, where possible.

SOFTWARE QUALITY:

o ROBUSTNESS: Although unlikely, with long vectors support for binCounts() and binMeans() it is possible that a bin gets a higher count than what can be represented by an R integer (.Machine$integer.max = 2^31-1). If that happens, an informative warning is generated and the bin count is set to .Machine$integer.max. If this happens for binMeans(), the corresponding mean is still properly calculated and valid.

CODE REFACTORING:

o CLEANUP: Cleanup and harmonized the internal C API such there are two well defined API levels. The high-level API is called by R via .Call() and takes care of most of the argument validation and construction of the return value. This function dispatch to functions in the low-level API based on data type(s) and other arguments. The low-level API is written to work with basic C data types only.

BUG FIXES:

o Package incorrectly redefined R_xlen_t on R (>= 3.0.0) systems where LONG_VECTOR_SUPPORT is not supported.

Version: 0.11.0 [2014-11-02]

NEW FEATURES:

o Added sumOver() and meanOver(), which are notably faster versions of sum(x[idxs]) and mean(x[idxs]). Moreover, instead of having to do sum(as.numeric(x)) to avoid integer overflow when 'x' is an integer vector, one can do sumOver(x, mode = "numeric"), which avoids the extra copy created when coercing to numeric (this numeric copy is also twice as large as the integer vector). Added package tests and benchmark reports for these functions.

Version: 0.10.4 [2014-11-01]

PERFORMANCE AND MEMORY:

o SPEEDUP: Made anyMissing(), logSumExp(), (col|row)Medians(), (col|row)Counts() slightly faster by making the native code assign the results directly to the native vector instead of to the R vector, e.g. ansp[i] = v where ansp = REAL(ans) instead of REAL(ans)[i] = v.

o Added benchmark reports for anyMissing() and logSumExp().

Version: 0.10.3 [2014-10-01]

BUG FIXES:

o binMeans() returned 0.0 instead of NA_real_ for empty bins.

Version: 0.10.2 [2014-09-01]

BUG FIXES:

o On some systems, the package failed to build on R (<= 2.15.3) with compilation error: "redefinition of typedef 'R_xlen_t'".

Version: 0.10.1 [2014-06-09]

PERFORMANCE AND MEMORY:

o Added benchmark reports for also non-matrixStats functions col-/rowSums() and col-/rowMeans().

o Now all colNnn() and rowNnn() methods are benchmarked in a combined report making it possible to also compare colNnn(x) with rowNnn(t(x)).

Version: 0.10.0 [2014-06-07]

SOFTWARE QUALITY:

o Relaxed some packages tests such that they assert numerical correctness via all.equal() rather than identical().

o Submitted to CRAN.

BUG FIXES:

o The package tests for product() incorrectly assumed that the value of prod(c(NaN, NA)) is uniquely defined. However, as documented in help("is.nan"), it may be NA or NaN depending on R system/platform.

Version: 0.9.7 [2014-06-05]

BUG FIXES:

o Introduced a bug in v0.9.5 causing col-/rowVars() and hence also col-/rowSds() to return garbage. Add package tests for these now.

o Submitted to CRAN.

Version: 0.9.6 [2014-06-04]

NEW FEATURES:

o Added signTabulate() for tabulating the number of negatives, zeros, positives and missing values. For doubles, the number of negative and positive infinite values are also counted.

PERFORMANCE AND MEMORY:

o SPEEDUP: Now col-/rowProds() utilizes new product() function.

o SPEEDUP: Added product() for calculating the product of a numeric vector via the logarithm.

Version: 0.9.5 [2014-06-04]

SIGNIFICANT CHANGES:

o SPEEDUP: Made weightedMedian() a plain function (was an S3 method).

o CLEANUP: Now only exporting plain functions and generic functions.

o SPEEDUP: Turned more S4 methods into S3 methods, e.g. rowCounts(), rowAlls(), rowAnys(), rowTabulates() and rowCollapse().

NEW FEATURES:

o Added argument 'method' to col-/rowProds() for controlling how the product is calculated.

PERFORMANCE AND MEMORY:

o SPEEDUP: Package is now byte compiled.

o SPEEDUP: Made rowProds() and rowTabulates() notably faster.

o SPEEDUP: Now rowCounts(), rowAnys(), rowAlls() and corresponding column methods can search for any value in addition to the default TRUE. The search for a matching integer or double value is done in native code, which is notably faster (and more memory efficient because it avoids creating any new objects).

o SPEEDUP: Made colVars() and colSds() notably faster and rowVars() and rowSds() a slightly bit faster.

o Added benchmark reports, e.g. matrixStats:::benchmark('colMins').

Version: 0.9.4 [2014-05-23]

SIGNIFICANT CHANGES:

o SPEEDUP: Turned several S4 methods into S3 methods, e.g. indexByRow(), madDiff(), sdDiff() and varDiff().

Version: 0.9.3 [2014-04-26]

NEW FEATURES:

o Added argument 'trim' to madDiff(), sdDiff() and varDiff().

Version: 0.9.2 [2014-04-04]

BUG FIXES:

o The native code of binMeans(x, bx) would try to access an out-of-bounds value of argument 'y' iff 'x' contained elements that are left of all bins in 'bx'. This bug had no impact on the results and since no assignment was done it should also not crash/core dump R. This was discovered thanks to new memtests (ASAN and valgrind) provided by CRAN.

Version: 0.9.1 [2014-03-31]

BUG FIXES:

o rowProds() would throw "Error in rowSums(isNeg) : 'x' must be an array of at least two dimensions" on matrices where all rows contained at least one zero. Thanks to Roel Verbelen at KU Leuven for the report.

Version: 0.9.0 [2014-03-26]

NEW FEATURES:

o Added weighedVar() and weightedSd().

Version: 0.8.14 [2013-11-23]

PERFORMANCE AND MEMORY:

o MEMORY: Updated all functions to do a better job of cleaning out temporarily allocated objects as soon as possible such that the garbage collector can remove them sooner, iff wanted. This increase the chance for a smaller memory footprint.

o Submitted to CRAN.

Version: 0.8.13 [2013-10-08]

NEW FEATURES:

o Added argument 'right' to binCounts() and binMeans() to specify whether binning should be done by (u,v] or [u,v). Added system tests validating the correctness of the two cases.

CODE REFACTORING:

o Bumped up package dependencies.

Version: 0.8.12 [2013-09-26]

PERFORMANCE AND MEMORY:

o SPEEDUP: Now utilizing anyMissing() everywhere possible.

Version: 0.8.11 [2013-09-21]

SOFTWARE QUALITY:

o ROBUSTNESS: Now importing 'loadMethod' from 'methods' package such that 'matrixStats' S4-based methods also work when 'methods' is not loaded, e.g. when 'Rscript' is used, cf. Section 'Default packages' in 'R Installation and Administration'.

o ROBUSTNESS: Updates package system tests such that the can run with only the 'base' package loaded.

Version: 0.8.10 [2013-09-15]

CODE REFACTORING:

o CLEANUP: Now only importing two functions from the 'methods' package.

o Bumped up package dependencies.

Version: 0.8.9 [2013-08-29]

NEW FEATURES:

o CLEANUP: Now the package startup message acknowledges argument 'quietly' of library()/require().

Version: 0.8.8 [2013-07-29]

DOCUMENTATION:

o The dimension of the return value was swapped in help("rowQuantiles").

Version: 0.8.7 [2013-07-28]

PERFORMANCE AND MEMORY:

o SPEEDUP: Made (col|row)Mins() and (col|row)Maxs() much faster.

BUG FIXES:

o rowRanges(x) on an Nx0 matrix would give an error. Same for colRanges(x) on an 0xN matrix. Added system tests for these and other special cases.

Version: 0.8.6 [2013-07-20]

CODE REFACTORING:

o Bumped up package dependencies.

BUG FIXES:

o Forgot to declare S3 methods (col|row)WeightedMedians().

Version: 0.8.5 [2013-05-25]

PERFORMANCE AND MEMORY:

o Minor speedup of (col|row)Tabulates() by replacing rm() calls with NULL assignments.

Version: 0.8.4 [2013-05-20]

DOCUMENTATION:

o CRAN POLICY: Now all Rd \usage{} lines are at most 90 characters long.

Version: 0.8.3 [2013-05-10]

PERFORMANCE AND MEMORY:

o SPEEDUP: binCounts() and binMeans() now uses Hoare's Quicksort for presorting 'x' before counting/averaging. They also no longer test in every iteration (== for every data point) whether the last bin has been reached or not, but only after completing a bin.

Version: 0.8.2 [2013-05-02]

DOCUMENTATION:

o Minor corrections and updates to help pages.

Version: 0.8.1 [2013-05-02]

BUG FIXES:

o Native code of logSumExp() used an invalid check for missing value of an integer argument. Detected by Brian Ripley upon CRAN submission.

Version: 0.8.0 [2013-05-01]

NEW FEATURES:

o Added logSumExp(lx) and (col|row)LogSumExps(lx) for accurately computing of log(sum(exp(lx))) for standalone vectors, and row and column vectors of matrices. Thanks to Nakayama (Japan) for the suggestion and contributing a draft in R.

Version: 0.7.1 [2013-04-23]

NEW FEATURES:

o Added argument 'preserveShape' to colRanks(). For backwardcompatibility the default is preserveShape = FALSE, but it may change in the future.

BUG FIXES:

o Since v0.6.4, (col|row)Ranks() gave the incorrect results for integer matrices with missing values.

o Since v0.6.4, (col|row)Medians() for integers would calculate ties as floor(tieAvg).

Version: 0.7.0 [2013-01-14]

NEW FEATURES:

o Now (col|row)Ranks() support "max" (default), "min" and "average" for argument 'ties.method'. Added system tests validation these cases. Thanks Peter Langfelder (UCLA) for contributing this.

Version: 0.6.4 [2013-01-13]

NEW FEATURES:

o Added argument 'ties.method' to rowRanks() and colRanks(), but still only support for "max" (as before).

CODE REFACTORING:

o ROBUSTNESS: Lots of cleanup of the internal/native code. Native code for integer and double cases have been harmonized and are now generated from a common code template. This was inspired by code contributions from Peter Langfelder (UCLA).

Version: 0.6.3 [2013-01-13]

NEW FEATURES:

o Added anyMissing() for data type 'raw', which always returns FALSE.

SOFTWARE QUALITY:

o ROBUSTNESS: Added system test for anyMissing().

o ROBUSTNESS: Now S3 methods are declared in the namespace.

Version: 0.6.2 [2012-11-15]

SOFTWARE QUALITY:

o CRAN POLICY: Made example(weightedMedian) faster.

Version: 0.6.1 [2012-10-10]

BUG FIXES:

o In some cases binCounts() and binMeans() could try to go past the last bin resulting a core dump.

o binCounts() and binMeans() would return random/garbage values for bins that were beyond the last data point.

Version: 0.6.0 [2012-10-04]

NEW FEATURES:

o Added binMeans() for fast sample-mean calculation in bins. Thanks to Martin Morgan at the Fred Hutchinson Cancer Research Center, Seattle, for contributing the core code for this.

o Added binCounts() for fast element counting in bins.

Version: 0.5.3 [2012-09-10]

SOFTWARE QUALITY:

o CRAN POLICY: Replaced the .Internal(psort(...)) call with a call to a new internal partial sorting function, which utilizes the native rPsort() part of the R internals.

Version: 0.5.2 [2012-07-02]

CODE REFACTORING:

o Updated package dependencies to match CRAN.

Version: 0.5.1 [2012-06-25]

NEW FEATURES:

o GENERALIZATION: Now (col|row)Prods() handle missing values.

CODE REFACTORING:

o Package now only imports the 'methods' package.

BUG FIXES:

o In certain cases, (col|row)Prods() would return NA instead of 0 for some elements. Added a redundancy test for the case. Thanks Brenton Kenkel at University of Rochester for reporting on this.

Version: 0.5.0 [2012-04-16]

NEW FEATURES:

o Added weightedMad() from aroma.core v2.5.0.

o Added weightedMedian() from aroma.light v1.25.2.

CODE REFACTORING:

o This package no longer depends on the aroma.light package for any of its functions.

o Now this package only imports R.methodsS3, meaning it no longer loads R.methodsS3 when it is loaded.

Version: 0.4.5 [2012-03-19]

NEW FEATURES:

o Updated the default argument 'centers' of rowMads()/colMads() to explicitly be (col|row)Medians(x,...). The default behavior has not changed.

Version: 0.4.4 [2012-03-05]

SOFTWARE QUALITY:

o ROBUSTNESS: Added system/redundancy tests for rowMads()/colMads().

o CRAN: Made the system tests "lighter" by default, but full tests can still be run, cf. tests/*.R scripts.

BUG FIXES:

o colMads() would return the incorrect estimates. This bug was introduced in matrixStats v0.4.0 (2011-11-11).

Version: 0.4.3 [2011-12-11]

BUG FIXES:

o rowMedians(..., na.rm = TRUE) did not handle NaN (only NA). The reason for this was the the native code used ISNA() to test for NA and NaN, but it should have been ISNAN(), which is opposite to how is.na() and is.nan() at the R level work. Added system tests for this case.

Version: 0.4.2 [2011-11-29]

NEW FEATURES:

o Added rowAvgsPerColSet() and colAvgsPerRowSet().

Version: 0.4.1 [2011-11-25]

DOCUMENTATION:

o Added help pages with an example to rowIQRs() and colIQRs().

o Added example to rowQuantiles().

BUG FIXES:

o rowIQRs() and colIQRs() would return the 25% and the 75% quantiles, not the difference between them. Thanks Pierre Neuvial at CNRS, Evry, France for the report.

Version: 0.4.0 [2011-11-11]

SIGNIFICANT CHANGES:

o Dropped the previously introduced expansion of 'center' in rowMads() and colMads(). It added unnecessary overhead if not needed.

NEW FEATURES:

o Added rowRanks() and colRanks(). Thanks Hector Corrada Bravo (University of Maryland) and Harris Jaffee (John Hopkins).

Version: 0.3.0 [2011-10-13]

PERFORMANCE AND MEMORY:

o SPEEDUP/LESS MEMORY: colMedians(x) no longer uses rowMedians(t(x)); instead there is now an optimized native-code implementation. Also, colMads() utilizes the new colMedians() directly. This improvement was kindly contributed by Harris Jaffee at Biostatistics of John Hopkins, USA.

SOFTWARE QUALITY:

o Added additional unit tests for colMedians() and rowMedians().

Version: 0.2.2 [2010-10-06]

NEW FEATURES:

o Now the result of (col|row)Quantiles() contains column names.

Version: 0.2.1 [2010-04-05]

NEW FEATURES:

o Added a startup message when package is loaded.

CODE REFACTORING:

o CLEANUP: Removed obsolete internal .First.lib() and .Last.lib().

Version: 0.2.0 [2010-03-30]

o DOCUMENTATION: Fixed some incorrect cross references.

Version: 0.1.9 [2010-02-03]

BUG FIXES:

o (col|row)WeightedMeans(..., na.rm = TRUE) would incorrectly treat missing values as zeros. Added corresponding redundancy tests (also for the median case). Thanks Pierre Neuvial for reporting this.

Version: 0.1.8 [2009-11-13]

BUG FIXES:

o colRanges(x) would return a matrix of wrong dimension if 'x' did not have any missing values. This would affect all functions relying on colRanges(), e.g. colMins() and colMaxs(). Added a redundancy test for this case. Thanks Pierre Neuvial at UC Berkeley for reporting this.

o (col|row)Ranges() return a matrix with dimension names.

Version: 0.1.7 [2009-06-20]

BUG FIXES:

o WORKAROUND: Cannot use "%#x" in rowTabulates() when creating the column names of the result matrix. It gave an error OSX with R v2.9.0 devel (2009-01-13 r47593b) current the OSX server at R-forge.

Version: 0.1.6 [2009-06-17]

DOCUMENTATION:

o Updated the help example for rowWeightedMedians() to run conditionally on aroma.light, which is only a suggested package - not a required one. This in order to prevent R CMD check to fail on CRAN, which prevents it for building binaries (as it currently happens on their OSX servers).

Version: 0.1.5 [2009-02-04]

BUG FIXES:

o For some errors in rowOrderStats(), the stack would not become UNPROTECTED before calling error.

Version: 0.1.4 [2009-02-02]

NEW FEATURES:

o Added methods (col|row)Weighted(Mean|Median)s() for weighted averaging.

DOCUMENTATION:

o Added help to more functions.

SOFTWARE QUALITY:

o Package passes R CMD check flawlessly.

Version: 0.1.3 [2008-07-30]

NEW FEATURES:

o Added (col|row)Tabulates() for integer and raw matrices.

BUG FIXES:

o rowCollapse(x) was broken and returned the wrong elements.

Version: 0.1.2 [2008-04-13]

NEW FEATURES:

o Added (col|row)Collapse().

o Added varDiff(), sdDiff() and madDiff().

o Added indexByRow().

Version: 0.1.1 [2008-03-25]

NEW FEATURES:

o Added (col|row)OrderStats().

o Added (col|row)Ranges() and (col|row)(Min|Max)s().

o Added colMedians().

o Now anyMissing() support most data types as structures.

Version: 0.1.0 [2007-11-26]

NEW FEATURES:

o Imported the rowNnn() methods from Biobase.

o Created.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("matrixStats")

0.53.1 by Henrik Bengtsson, 4 months ago


https://github.com/HenrikBengtsson/matrixStats


Report a bug at https://github.com/HenrikBengtsson/matrixStats/issues


Browse source code at https://github.com/cran/matrixStats


Authors: Henrik Bengtsson [aut, cre, cph], Hector Corrada Bravo [ctb], Robert Gentleman [ctb], Ola Hossjer [ctb], Harris Jaffee [ctb], Dongcan Jiang [ctb], Peter Langfelder [ctb], Peter Hickey [ctb]


Documentation:   PDF Manual  


Artistic-2.0 license


Suggests base64enc, ggplot2, knitr, microbenchmark, R.devices, R.rsp


Imported by ACNE, CpGFilter, DGCA, DHS.rates, FADA, GJRM, GPrank, IMIFA, LFDREmpiricalBayes, Luminescence, MFHD, MHTcop, MoEClust, NetLogoR, PSCBS, SGP, SemiParSampleSel, WGCNA, abcrf, adjclust, aroma.affymetrix, aroma.cn, aroma.core, baystability, bdynsys, bigstep, bingat, bmrm, bnclassify, brms, calmate, carx, cellWise, childhoodmortality, cna, cointReg, cointmonitoR, dplR, eseis, estudy2, expss, fergm, filesstrings, fslr, haploReconstruct, jointseg, kernDeepStackNet, kgschart, kpmt, liger, loo, lspartition, ltmle, matrixTests, microsamplingDesign, mixOmics, mmtfa, mrfDepth, neurobase, nparMD, peakPick, randomizationInference, sizeMat, smerc, staRdom, stability, stapler, statar, stm, summarytools, support, visualFields.

Depended on by BayesTwin, DAMOCLES, DPWeibull, DisHet, FastHCS, FastPCS, FastRCS, GAD, GUniFrac, InfiniumPurify, LS2Wstat, NSA, POMaSPU, RAC, StructFDR, aSPU, cosinor2, localgauss, r2dRue, samr, splitFeas, ttScreening.

Suggested by DatabionicSwarm, GeneralizedUmatrix, LSAmitR, MetaQC, MultiBD, cifti, detrendr, dtree, tmlenet.


See at CRAN