Diagnostic Tools for Hierarchical (Multilevel) Linear Models

A suite of diagnostic tools for hierarchical (multilevel) linear models. The tools include not only leverage and traditional deletion diagnostics (Cook's distance, covratio, covtrace, and MDFFITS) but also convenience functions and graphics for residual analysis. Models can be fit using either lmer in the 'lme4' package or lme in the 'nlme' package.

Up to now diagnostics for mixed and hierarchical models have required much programming by the analyst, especially if one desires influence diagnostics. To help fill this need, HLMdiag:

  • Provides convenience functions for residual analysis.

    • Allows the analyst to obtain residuals estimated by least squares (LS) or empirical Bayes (EB).
    • Allows the analyst to obtain different residual quantities (e.g. marginal, conditional, BLUPs for the two-level model).
  • Implements influence analysis.

    • Leverage
    • Deletion diagnostics -- Cook's distance, MDFFITS, covariance ratio & trace

HLMdiag strives to provide an easy to use interface for models fit using lmer from the package lme4 that is draws from the ideas of influence.measures for regression diagnostics.

Development may be quite slow for a while, as I am finishing my degree, so please bear with me!

Development version

If you would like to download the development version of HLMdiag, I would recommend using Hadley Wickham's devtools package:

# install.packages("devtools")


Version 0.3.1


  • .extractV.lme (and thus .lme_matrices) was fixed to work with more complex covariance structures fit via nlme.
  • Updated package to work with the most recent version of ggplot2

Version 0.3.0


  • Influence diagnostics in HLMdiag 0.3.0 are available for two-level models fit using the lmer function in lme4or thelmefunction innlme. I am still working to implement these methods for higher-level models usinglme`.
  • HLMdiag no longer loads lme4 automatically (see above for the reason).


  • The rotate_ranef function has been added. This function rotates the random effects in an effort to find the least confounded residuals for distributional assessment.


  • LSresids was fixed for an issue with the order of the resulting data frame.
  • case_delete was fixed so that numeric group labels work properly, which fixes an issue with rvc.
  • Fixed an issue with group_qqnorm, by using ppoints rather than .SampleQuantiles.
  • Fixed an issue with case_delete.lmerMod, to use the getME() function to extract n.
  • A bug in the calculation of the Cholesky residuals was fixed (thanks to Harry Hiemstra for reporting the bug and the fix)

Version 0.2.5

  • Fixed a compatibility issue with Rcpp

Version 0.2.4

  • Added citation for the JSS paper
  • Fixed a bug with the calculation of Cook's distance

Version 0.2.3

  • Added a function to calculate rotated random effects
  • Added new data sets
  • Maintenance for compatibility with lme4 1.0 and R 3.0.2

Version 0.2.2

  • Changed the standardization of the EB level-1 residuals in HLMresid to a more appropriate definition: e / var(\hat{e}).
  • Added functions to add compatibility with the development version of lme4.

Version 0.2.1

  • Fixed a bug in group_qqnorm
  • Checked compatibility with R 2.15.3

Version 0.2.0


  • Influence diagnostics in HLMdiag 0.2.0 are compatible with hierarchical (multilevel) linear models of any size and with models with crossed factors.
  • HLMdiag 0.2.0 offers significantly faster computation of the deletion diagnostics for fixed effects that are based on one-step approximations.
  • S3 methods have been created for cooks.distance, mdffits, covratio, covtrace, rvc, and leverage for objects of class mer.
  • Full deletions are still available using the case_delete function, with corresponding S3 methods for objects of class case_delete.


  • A leverage function has been added.
  • case_delete and other deletion functions now allow for the user to manually specify a subset to delete.


  • changes to the arguments of dotplot_diag to accomodate a more general usage.
  • diagnostics no longer requires a model parameter to be specified.
  • A delete parameter has been added to case_delete to allow for manual specification of a subset that should be deleted.
  • For observation-level deletion using case_delete the user should specify group = NULL rather than group = FALSE.

Version 0.1.6

  • Updates for compatibility with ggplot2 >= 0.9.2
  • Updates to NAMESPACE

Version 0.1.5

  • Updates to NAMESPACE to fix compatibility issues

Version 0.1.4

  • Updates for compatibility with ggplot2 0.9.0

Version 0.1.3

  • Added "marginal" residuals to the type argument for HLMresid.

Version 0.1.2


  • Removed the formula argument from LSresids. The formula is now obtained automatically from the mer object. Note: we are still working on automatic recognition of math operators such as log(), but anything in I() is recognized.
  • Added the function HLMresid, a wrapper that will extract both the LS and EB residuals given an mer object.
  • Added a level argument to LSresids, so the function can extract LS residuals from either level of the model.


  • Fixed the ordering of output from 'LSresids' to match the mode frame obtained from the mer object.

Version 0.1.1

  • Created 'adjust_lmList' class to handle fitting separate linear models when a factor is constant across the group.
  • 'random_ls_coef' was removed and replaced by the 'coef' method for 'adjust_lmList' objects.
  • Added 'rvc' diagnostic.
  • Extended 'case_delete' from only handling deletion for fixed effects to also handling deletion for variance components.
  • Improved 'dotplot_diag' to handle modified dotplots.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.


0.5.0 by Adam Loy, 17 days ago


Report a bug at https://github.com/aloy/HLMdiag/issues

Browse source code at https://github.com/cran/HLMdiag

Authors: Adam Loy [cre, aut] , Jaylin Lowe [aut] , Jack Moran [aut]

Documentation:   PDF Manual  

GPL-2 license

Imports ggplot2, stats, methods, plyr, reshape2, MASS, Matrix, mgcv, dplyr, magrittr, stringr, purrr, tibble, tidyselect, janitor, Rcpp, rlang, ggrepel, diagonals

Suggests mlmRev, WWGbook, lme4, nlme, testthat, knitr, rmarkdown, car, gridExtra, qqplotr

Linking to Rcpp, RcppArmadillo

Imported by emdi, lmeresampler.

See at CRAN