Retrieval Functions for USGS and EPA Hydrologic and Water Quality Data

Collection of functions to help retrieve U.S. Geological Survey (USGS) and U.S. Environmental Protection Agency (EPA) water quality and hydrology data from web services. USGS web services are discovered from National Water Information System (NWIS) < https://waterservices.usgs.gov/> and < https://waterdata.usgs.gov/nwis>. Both EPA and USGS water quality data are obtained from the Water Quality Portal < https://www.waterqualitydata.us/>.


dataRetrieval

Retrieval functions for USGS and EPA hydrologic and water quality data. For tutorial information, see:

usgs-r.github.io/dataRetrieval To install the dataRetrieval package, you must be using R 3.0 or greater and run the following command:

install.packages("dataRetrieval")

To get inter-CRAN release updates, use the command:

install.packages("dataRetrieval",repos="http://owi.usgs.gov/R")

To get cutting-edge changes, install from GitHub using the devtools packages:

library(devtools)
install_github("USGS-R/dataRetrieval")

##Reporting bugs

Please consider reporting bugs and asking questions on the Issues page:

https://github.com/USGS-R/dataRetrieval/issues

Follow @USGS-R on Twitter for updates on all USGS R packages: https://twitter.com/USGS_R

LinuxWindowsTest Coverage
VersionMonthly DownloadsTotal Downloads

##Disclaimer This software is in the public domain because it contains materials that originally came from the U.S. Geological Survey, an agency of the United States Department of Interior. For more information, see the official USGS copyright policy

Although this software program has been used by the U.S. Geological Survey (USGS), no warranty, expressed or implied, is made by the USGS or the U.S. Government as to the accuracy and functioning of the program and related program material nor shall the fact of distribution constitute any such warranty, and no responsibility is assumed by the USGS in connection therewith.

This software is provided "AS IS."

News

dataRetrieval 2.6.3

  • All USGS web services are being flipped from http to https. This version of dataRetrieval will attempt to access https, if that fails, will attempt http.

dataRetrieval 2.5.12

  • Response to readr 1.0.0 updates
  • Added NWIS Water Use data
  • Added NWIS Statistical Service data
  • Added message on updates to NWIS data on startup message
  • Automatically retries web service calls when status code is greater than 400
  • More integrated use of pkg.env to set base URLs

dataRetrieval 2.5.7

  • HEAD requests can be returned in readWQP* functions. This can give some indication on amount of data available.

dataRetrieval 2.5.0

  • Package httr is now used for web service retrievals

dataRetrieval 2.4.0

  • Package readr now used for tab delimited parsing
  • readr functions used to determine column types. Mostly, this produces the same results.
  • In the case where there is text in a numeric column (specified by the RDB header), these now remain characters (previously was converted to numeric)
  • Columns that come back from web services as integers remain integers (previously was converted to numeric)
  • Added reported time zone code information. dateTime columns by default get converted to UTC, but the original time zone code (tz_cd for instance) is appended to the data frame.

dataRetrieval 2.3.0

  • Converted all Water Quality Portal queries to sorted=no to greatly improve retrieval times
  • Converted gwlevels in readNWISdata to rdb

dataRetrieval 2.2.0

  • Changed the output of tz_cd (in all *NWIS functions) to the timezone that is reported in the data frame. Before this version, it was the time zone reported by NWIS. This was confusing however since the dateTime columns default to UTC.
  • Minor bug fixes
  • Help file updates
  • Improved query options in readNWISdata and readWQPdata functions.

dataRetrieval 2.1.4

  • Expanded readNWISdata to qw data
  • Improved citation file

dataRetrieval 2.1.1

  • Removed tests that call external web services, server upgrades have been causing false negative results
  • Added functionality to readWQPdata to allow R objects as arguments rather than just character strings
  • Added parameterCd='all' option for qw calls.
  • Exposed sysdata constants

dataRetrieval 2.0.2

  • Moved data files (parameterCdFile and pCodeToName) to hidden sysdata.rda
  • Improved error handling.
  • Fixed WaterML2 bug.
  • Switched readNWISgwl to read the RDB output.

dataRetrieval 2.0.1

  • Removed deprecated functions.
  • Added siteInfo and variableInfo dataframe attributes to all data retrival functions
  • Improved help files
  • Added data file (pCodeToName) that shows the association of USGS parameter code to characteristicName

dataRetrieval 2.0.0

  • Overhaul of function names based on initial user feedback
  • Metadata attributes added to returned dataframes
  • EGRET specific functions moved to EGRET

dataRetrieval 1.4.0

  • Changed naming convention:
  • Changed WaterML2 rbind fill from plyr function to dplyr. Removed plyr import, added dplyr.

dataRetrieval 1.3.3

  • Updated getNWISSiteInfo to retrieve multiple site file datasets at once using a vector of siteNumbers as input argument.
  • Updated error-handling for Web service calls. More information is returned when errors happen
  • Added some basic processing to Water Quality Portal raw data retrievals. Date columns are returned as Date objects, value columns are numeric, and a column is created from the date/time/timezone columns that is POSIXct.
  • Added very generalized NWIS and WQP retrieval functions (getNWISData, getNWISSites, getGeneralWQPData, and whatWQPsites) which allow the user to use any argument available on the Web service platform.

dataRetrieval 1.3.2

  • Deprecated getQWData, updated readWQPdata to take either parameter code or characteristic name.
  • Changed the name of raw data retrievals to: readNWISqw, getNWISunitData, getNWISdvData, and getWQPqwData (from: readNWISqw, retrieveUnitNWISData, retrieveNWISData, getRawQWData)
  • Added NA warning to getDVData function
  • Updated mergeReport to allow for Sample data with different measurements taken on the same day

dataRetrieval 1.2.2

  • Added getNWISDataAvailability function to find measured parameters and period of record information for a requested station.

  • Added constructNWISURL function to get the URL that is used to retrieve the data.

  • Added getSampleSTORET function to get STORET data directly in Sample dataframe form.

  • Fixed a small leap year bug by changing day of year by making Feb. 29 always 59, and March 1st always 60 (even in non-leap years).

dataRetrieval 1.2.1

  • Improved documentation, especially example functions.

  • Improved vignette for a more complete walk-through.

  • Expanded the capabilities to retrieve raw data from the web services.

  • Added Storet data retrievals in readWQPdata function

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("dataRetrieval")

2.7.3 by Laura DeCicco, 2 months ago


https://pubs.usgs.gov/tm/04/a10/


Report a bug at https://github.com/USGS-R/dataRetrieval/issues


Browse source code at https://github.com/cran/dataRetrieval


Authors: Robert Hirsch [aut], Laura DeCicco [aut, cre], David Lorenz [aut], Jordan Read [ctb], Jordan Walker [ctb], Lindsay Carr [ctb], David Watkins [aut]


Documentation:   PDF Manual  


Task views:


CC0 license


Imports httr, curl, reshape2, lubridate, stats, utils, dplyr, xml2, readr, jsonlite

Suggests htmlTable, knitr, testthat


Imported by CityWaterBalance, EGRET, countyfloods, waterData.


See at CRAN