An API Wrapper for the Bureau of Labor Statistics (BLS)

Scrapes various data from < https://www.bls.gov/>. The U.S. Bureau of Labor Statistics is the statistical branch of the United States Department of Labor. The package has additional functions to help parse, analyze and visualize the data.


Build Status CRAN_Status_Badge Project Status: Active - The project has reached a stable, usable state and is being actively developed.

Designed to be a tidy API wrapper for the Bureau of Labor Statistics (BLS.) The package has additional functions to help parse, analyze and visualize the data. The package utalizes "tidyverse" concepts for internal functionality and encourages the use of those concepts with the output data.

Install

  • Stable version from CRAN:
install.packages("blscrapeR")
  • The latest development version from GitHub:
devtools::install_github("keberwein/blscrapeR")

Before getting started, you’ll probably want to head over to the BLS and get set up with an API key. While an API key is not required to use the package, the query limits are much higher if you have a key and you’ll have access to more data. Plus, it’s free (as in beer), so why not?

Basic Usage

For “quick and dirty” type of analysis, the package has some quick functions that will pull metrics from the API without series numbers. These quick functions include unemployment, employment, and civilian labor force on a national level.

library(blscrapeR)
df <- quick_unemp_rate()
head(df, 5)
#> # A tibble: 5 x 6
#>    year    period periodName value footnotes  seriesID
#>   <dbl>    <list>     <list> <dbl>    <list>    <list>
#> 1  2017 <chr [1]>  <chr [1]>   4.1 <chr [1]> <chr [1]>
#> 2  2017 <chr [1]>  <chr [1]>   4.2 <chr [1]> <chr [1]>
#> 3  2017 <chr [1]>  <chr [1]>   4.4 <chr [1]> <chr [1]>
#> 4  2017 <chr [1]>  <chr [1]>   4.3 <chr [1]> <chr [1]>
#> 5  2017 <chr [1]>  <chr [1]>   4.4 <chr [1]> <chr [1]>

Search BLS IDs

Some knowledge of BLS ids are needed to query the API. The package includes a "fuzzy search" function to help find these ids. There are currenlty more than 75,000 series ids in the package's internal dataset, series_ids. While these aren't all the series ids the BLS offers, it contains the most popular. The BLS Data Finder is another good resource for finding series ids, that may not be in the internal dataset.

library(blscrapeR)
# Find series ids relating to the total labor force in LA.
ids <- search_ids(keyword = c("Labor Force", "Los Angeles"))
head(ids)
#> # A tibble: 6 x 4
#>                                                                  series_title
#>                                                                         <chr>
#> 1 Labor Force: Balance Of California, State Less Los Angeles-Long Beach-Glend
#> 2  Labor Force: Los Angeles-Long Beach-Glendale, Ca Metropolitan Division (S)
#> 3 Labor Force: Balance Of California, State Less Los Angeles-Long Beach-Glend
#> 4       Labor Force: Los Angeles-Long Beach, Ca Combined Statistical Area (U)
#> 5                                     Labor Force: Los Angeles County, Ca (U)
#> 6                                       Labor Force: Los Angeles City, Ca (U)
#> # ... with 3 more variables: series_id <chr>, seasonal <chr>,
#> #   periodicity_code <chr>
library(blscrapeR)
# Find series ids relating to median weekly earnings of women software developers.
ids <- search_ids(keyword = c("Earnings", "Software", "Women"))
head(ids)
#> # A tibble: 1 x 4
#>                                                                  series_title
#>                                                                         <chr>
#> 1 (Unadj)- Median Usual Weekly Earnings (Second Quartile), Employed Full Time
#> # ... with 3 more variables: series_id <chr>, seasonal <chr>,
#> #   periodicity_code <chr>

API Keys

You should consider getting an API key form the BLS. The package has a function to install your key in your .Renviron so you’ll only have to worry about it once. Plus, it will add extra security by not having your key hard-coded in your scripts for all the world to see.

From the BLS:

Service Version 2.0 (Registered) Version 1.0 (Unregistered)
Daily query limit 500 25
Series per query limit 50 25
Years per query limit 20 10
Net/Percent Changes Yes No
Optional annual averages Yes No
Series descriptions Yes No

Key Install

library(blscrapeR)
set_bls_key("YOUR_KEY_IN_QUOTATIONS")
# First time, reload your enviornment so you can use the key without restarting R.
readRenviron("~/.Renviron")
# You can check it with:
Sys.getenv("BLS_KEY")

Note: The above script will add a line to your .Renviron file to be re-used when ever you are in the package. If you are not comfortable with that, you can add the following line to your .Renviron file manually to produce the same result.

BLS_KEY='YOUR_KEY_IN_SINGLE_QUOTES'

Advanced Usage

Now that you have an API key installed, you can call your key in the package’s function arguments with "BLS_KEY". Don't forget the quotes! If you just HAVE to have your key hard-coded in your scripts, you can also pass they key as a string.

Download Multiple BLS Series at Once

library(blscrapeR)
 
# Grab several data sets from the BLS at onece.
# NOTE on series IDs: 
# EMPLOYMENT LEVEL - Civilian labor force - LNS12000000
# UNEMPLOYMENT LEVEL - Civilian labor force - LNS13000000
# UNEMPLOYMENT RATE - Civilian labor force - LNS14000000
df <- bls_api(c("LNS12000000", "LNS13000000", "LNS14000000"),
              startyear = 2008, endyear = 2017, Sys.getenv("BLS_KEY")) %>%
    # Add time-series dates
    dateCast()
# Plot employment level
library(ggplot2)
gg1200 <- subset(df, seriesID=="LNS12000000")
library(ggplot2)
ggplot(gg1200, aes(x=date, y=value)) +
    geom_line() +
    labs(title = "Employment Level - Civ. Labor Force")

Median Weekly Earnings

library(blscrapeR)
library(tidyverse)
# Median Usual Weekly Earnings by Occupation, Unadjusted Second Quartile.
# In current dollars
df <- bls_api(c("LEU0254530800", "LEU0254530600"), startyear = 2000, endyear = 2016, registrationKey = Sys.getenv("BLS_KEY")) %>%
    spread(seriesID, value) %>% dateCast()
# A little help from ggplot2!
library(ggplot2)
ggplot(data = df, aes(x = date)) + 
    geom_line(aes(y = LEU0254530800, color = "Database Admins.")) +
    geom_line(aes(y = LEU0254530600, color = "Software Devs.")) + 
    labs(title = "Median Weekly Earnings by Occupation") + ylab("value") +
    theme(legend.position="top", plot.title = element_text(hjust = 0.5)) 

For more advanced usage, please see the package vignettes.

Basic Mapping

Like the the “quick functions” for requesting API data, there are two "quick" map functions, bls_map_county() and bls_map_state(). These functions are designed to work with two helper functions get_bls_county() and get get_bls_state(). Each helper function downloads recent data for unemployment rate, unemployment level, employment rate, employment level and civilian labor force. These functions do not pull data from the API, rather they pull data from text files and do not count against daily query limits.

Note: Even though the get_bls functions return data in the correct formats, the bls_map functions can be used with any data set that includes FIPS codes.

The example below maps the current unemployment rate by county. Alaska and Hawaii have to re-located to save space.

library(blscrapeR)
# Grap the data in a pre-formatted data frame.
# If no argument is passed to the function it will load the most recent month's data.
df <- get_bls_county()
 
#Use map function with arguments.
map_bls(map_data = df, fill_rate = "unemployed_rate", 
               labtitle = "Unemployment Rate by County")

Advanced Mapping

What's R mapping without some interactivity? Below we’re using two additional packages, leaflet, which is popular for creating interactive maps and tigris, which allows us to download the exact shape files we need for these data and includes a few other nice tools.

# Leaflet map
library(blscrapeR)
library(tigris)
library(leaflet)
map.shape <- counties(cb = TRUE, year = 2015)
df <- get_bls_county()
 
# Slice the df down to only the variables we need and rename "fips" colunm
# so I can get a cleaner merge later.
df <- df[, c("unemployed_rate", "fips")]
colnames(df) <- c("unemployed_rate", "GEOID")
 
# Merge df with spatial object.
leafmap <- geo_join(map.shape, df, by="GEOID")
 
# Format popup data for leaflet map.
popup_dat <- paste0("<strong>County: </strong>", 
                    leafmap$NAME, 
                    "<br><strong>Value: </strong>", 
                    leafmap$unemployed_rate)
 
pal <- colorQuantile("YlOrRd", NULL, n = 20)
# Render final map in leaflet.
leaflet(data = leafmap) %>% addTiles() %>%
    addPolygons(fillColor = ~pal(unemployed_rate), 
                fillOpacity = 0.8, 
                color = "#BDBDC3", 
                weight = 1,
                popup = popup_dat)

Note: This is just a static image since the full map would not be as portable for the purpose of documentation.

News

blscrapeR 3.1.1

Bug fixes

  • The bls_api() function now returns the catalog data with a catalog=TRUE argument.

Enhancements

  • The bls_api() function now returns verbose error message if the api request fails.

  • Added BLS series_ids internal dataset.

  • Added search_ids() function to search the internal series_ids data set.

blscrapeR 3.0.1

Bug Fixes

  • Added an functionality and documentation for an "annual" argument for qcew_api().

Vignettes

  • Added new feature examples to QCEW_API.Rmd.

  • Removed qcew.Rmd, since it was duplicated.

blscrapeR 3.0

New Features

  • Added the map_bls() function. This function replaces the deprecated bls_map_county() bls_map_state() functions and is not backward compatible.

  • Updated internal data to 2016 FIPS standards.

Enhancements

  • Updated internal functions to use tidyverse methods. This includes the addition of a few dependencies such as purrr, tibble, and dplyr.

  • All data are now returned as tibbles.

Documentation

  • Updated all existing vignettes to use tidyverse functions.

  • Edited mapping vignette to include new map_bls() function.

  • Added tidyverse style code to the readme and vignettes.

blscrapeR 2.1.5

  • Document and vignette fixes

blscrapeR 2.1.4

New Features

  • Added total non-farm employment as a "quick function."

  • Added a dt_format argument to the dateCast() function.

blscrapeR 2.1.3

New Features

  • Added the compound pipe operator from the magrittr package to exported functions.

Enhancements

  • Added urlExists as a utility function for more robust error handling.

  • Added try/catch to url downloads.

blscrapeR 2.1.2

New Features

  • The ability to select custom choropleth colors with all map functions.

  • Automatic end dates set for bls_api() function.

  • Removed dependencies for xts and zoo packages. Changed to base R.

Bug fixes

  • Fixed error when rendering national maps with county-wise data.

  • Alaska and Hawaii moved further apart in map_data.

  • Date column changed to optional for bls_api() function.

blscrapeR 2.1.1

New Features

  • Added a map projection argument to the bls_map_county() function. Choices are either Lambert or Mercator. Mercator is the default for single states, and Lambert the default for national views.

  • Enhanced error handling for the get_bls_state() function.

Data

  • Added Mercator projection data set.

Bug Fixes

  • Fixed date parsing with the get_bls_state() function.

blscrapeR 2.0.0

Major Release

  • The BLS changed their servers from http to https. API protocols changed in accordance.

blscrapeR 1.0.2.9000

Bug Fixes

  • FIPS fix for the get_bls_state() function.

blscrapeR 1.0.2

Bug Fixes

  • Updated URLs for the get_bls_state() function.

Tets

  • Updated tests for the get_bls_state() function.

blscrapeR 1.0.1

Major Release

  • The BLS changed the location of their county level employment data on the server side. Changes were made in the get_bls_county() function that were not backwards compatible.

New Features

  • Added the qcew_api() function to gather data from the BLS Quarterly Census of Employment and Wages. This API is owned by the QCEW and is separate from the main BLS API.

  • Added 404 error handling to the main bls_api() function.

Data

  • Added three data sets; niacs, size_titles, and area_titles. These data sets act as helpers for the QCEW API and provide industry and area codes that the API regularly uses.

Bug Fixes

  • Updated URLs for the get_bls_county() function.

Documentation

  • Added documentation for the qcew_api() function.

  • Added documentation for the three new data sets; niacs, size_titles, and area_titles.

  • Added a vignette for the qcew_api() function.

Tets

  • Added a test for the qcew_api() function.

blscrapeR 0.4.2

New Features

  • Added stateName argument to get_bls_county() that allows user to specify a state or list of states.

  • Added stateName argument to map_bls_county() that allows user to specify state(s) to map.

Documentation

  • Added mapping vignette.

  • Added manual pages for the county_map_data and state_map_data internal data sets.

Data

  • Added tigris and broom packages to prep_maps.R in data-raw folder.

Bug Fixes

  • Fixed date argument in get_bls_county() and get_bls_state() to return the most recent date if argument is NULL.

  • Added error handling to map_bls_state() and map_bls_county().

  • Added tests for map_bls_state() and map_bls_county() to the /testthat directory.

blscrapeR 0.4.1

New Features

  • Added quick functions to pull popular data sets from the API without the need of the user inputting a series id.

  • Added data argument to get_bls_county. If NULL the function will return the most recent month.

  • Added the inflation_adjust() function to help users with calculating inflation from the CPI. Since the API will only allow twenty years of data, the inflation function pulls data from a text file instead that allows the user to get CPI data back to 1947.

  • Added more error handling to bls_api() function.

Documentation

  • Added documentation to get_bls_county() to explain the new date argument.

  • Added package vignettes.

blscrapeR 0.4.0

Major Changes

  • Added the set_bls_key() function to be used with the bls_api() function. The new function writes the user's api key to the Renviron. The change is backward compatible since the user is sill able to enter their api key as a string. However, for security purposes, the stored key method is preferred and should be promoted.

  • Added testthat directory and added test_bls_api.R. The tests won't affect anything in the package functionality, but will be useful for future testing.

Deprecated Features

  • Truncated the bls_state_data() function and added those features to the get_bls_state() function.

  • Removed dplyr from imports since it's not necessary anymore. Added leaflet and cron to package recommends.

Documentation

  • Renamed blscrape.R to bls_api.R since the bls_api() function was the only function in that file.

  • Added testthat to recommends.

blscrapeR 0.3.2

New Features

  • Revised the map prep in data-raw to render smaller data frames, thereby making the total package size much smaller.

  • Added a label title argument to map functions bls_map_state() and bls_map_county()

  • Standardized colnames in all returned data frames throughout the package.

Deprecated Features

  • Truncated the inflation() function for now. The API seems to return adjusted dollar-values.

Documentation

  • Made documentation for helper functions invisible in the manual.

  • Added News.Rd

blscrapeR 0.3.1

New Features

  • Changed name of county_map() to avoid conflicts with other packages. I didn't mark this as a major release because no other functions in the package are dependent on this.

  • Various fixes in map functions including removing unused themes and general code tidyness.

Documentation

  • Various fixes to examples.

  • Made descriptions more robust.

blscrapeR 0.3.0

Major Changes

  • Changed name of get_data() to bls_api() to avoid any problems in the global environment. Bumped the version to 0.3.0 because this is the primary workhorse function of the package.

blscrapeR 0.2.1

New Features

  • Added custom maps rendered from shape files

Major Changes

  • Removed shape files from package data to cut down on size. This fundamentally changes the way the mapping functions work. Opted instead, for spacial poly data frames rendered from the shapefiles.

blscrapeR 0.2.0

Major Changes

  • Complete re-write of old get_data() function to pull from API only.

  • Added arguments to get_data() to pass the BLS API perams.

blscrapeR 0.1.0

New features

  • Added bls_state_data() function to pull most recent sate-level employment data for mapping.

  • Added arguments to get_data() to pass the BLS API perams.

  • Added documentation for data sets.

Bug Fixes

  • Fixed zoo::index() error.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("blscrapeR")

3.1.1 by Kris Eberwein, 3 months ago


https://github.com/keberwein/blscrapeR


Report a bug at https://github.com/keberwein/blscrapeR/issues


Browse source code at https://github.com/cran/blscrapeR


Authors: Kris Eberwein [aut, cre]


Documentation:   PDF Manual  


MIT + file LICENSE license


Imports httr, jsonlite, ggplot2, magrittr, utils, stats, grDevices, dplyr, purrr, tibble, stringr

Suggests testthat, knitr, rmarkdown, tidyverse, tigris, leaflet


See at CRAN