Provides a set of functions for data manipulation with list objects, including mapping, filtering, grouping, sorting, updating, searching, and other useful functions. Most functions are designed to be pipeline friendly so that data processing with lists can be chained.
rlist is a set of tools for working with list objects. Its goal is to make it easier to work with lists by providing a wide range of functions that operate on non-tabular data stored in them.
This package supports list mapping, filtering, grouping, sorting, updating, searching, file input/output, and many other functions. Most functions in the package are designed to be pipeline friendly so that data processing with lists can be chained.
rlist Tutorial is a highly recommended complete guide to rlist.
Install the latest version from GitHub:
Install from CRAN:
In R, there are numerous powerful tools to deal with structured data stored in tabular form such as data frame. However, a variety of data is non-tabular: different records may have different fields; for each field they may have different number of values.
It is hard or no longer straightforward to store such data in data frame, but the
list object in R is flexible enough to represent such records of diversity. rlist is a toolbox to deal with non-structured data stored in
list objects, providing a collection of high-level functions which are pipeline friendly.
Suppose we have a list of developers, each of whom has a name, age, a few interests, a list of programming languages they use and the number of years they have been using them.
This type of data is non-relational since it does not well fit the shape of a data frame, yet it can be easily stored in JSON or YAML format. In R, list objects are flexible enough to represent a wide range of non-relational datasets like this. This package provides a wide range of functions to query and manipulate this type of data.
The following examples use
str() to show the structure of the output.
Filter those who like music and has been using R for more than 3 years.
str( list.filter(devs, "music" %in% interest & lang$r >= 3) )
List of 1$ p2:List of 4..$ name : chr "James"..$ age : num 25..$ interest: chr [1:2] "sports" "music"..$ lang :List of 3.. ..$ r : num 3.. ..$ java: num 2.. ..$ cpp : num 5
Select their names and ages.
str( list.select(devs, name, age) )
List of 3$ p1:List of 2..$ name: chr "Ken"..$ age : num 24$ p2:List of 2..$ name: chr "James"..$ age : num 25$ p3:List of 2..$ name: chr "Penny"..$ age : num 24
Map each of them to the number of interests.
str( list.map(devs, length(interest)) )
List of 3$ p1: int 3$ p2: int 2$ p3: int 2
In addition to these basic functions, rlist also supports various types of grouping, joining, searching, sorting, updating, etc. For the introduction to more functionality, please go through the rlist Tutorial.
In this package, almost all functions that work with expressions accept the following forms of lambda expressions:
x ~ expression
f(x) ~ expression
f(x,i) ~ expression
f(x,i,name) ~ expression
x refers to the list member itself,
i denotes the index,
name denotes the name. If the symbols are not explicitly declared,
.name will by default be used to represent them, respectively.
nums <- list(a=c(1,2,3),b=c(2,3,4),c=c(3,4,5))list.map(nums, c(min=min(.),max=max(.)))list.filter(nums, x ~ mean(x)>=3)list.map(nums, f(x,i) ~ sum(x,i))
Query the name of each developer who likes music and uses R, and put the results in a data frame.
library(pipeR)devs %>>%list.filter("music" %in% interest & "r" %in% names(lang)) %>>%list.select(name,age) %>>%list.stack
name age1 Ken 242 James 25
The example above uses
pipeR(http://renkun.me/pipeR/) package for pipeline operator
%>>% that chains commands in a fluent style.
List() function wraps a list within an environment where almost all list functions are defined. Here is the List-environment version of the previous example.
ldevs <- List(devs)ldevs$filter("music" %in% interest & "r" %in% names(lang))$select(name,age)$stack()$data
name age1 Ken 242 James 25
help(package = rlist)
or view the documentation on CRAN
This package is under MIT License.
unlista nested list recursively. (#102)
classesto filter list element recursively by class name.
list.expandimplements a list version of
is.nullto clean the results.
group.namesto indicate whether to preserve group names.
list.tablewill directly call
tableupon input data if
list.skipreturns the original data when asked to skip 0 elements.
list.skiptakes the first
nelements when asked to skip a negative number of elements.
nyweatherscraped from OpenWeatherMap (#2)
list.loadnow supports text-based progress bar when
progress = TRUEwhich is by default enabled if over 5 files are to be loaded. (#92)
list.namesgives a list or vector names by mapping.
list.lastfind the first or last list element that meets a given condition.
list.unzipto transform a list of elements with similar structure into a list of decoupled fields.
list.groupnow supports grouping by multi-key which produces multi-level list. (#69)
list.loadnow supports loading from multiple filenames given in character vector. (#74)
list.loadis now able to guess the file format even if the file type is not specified. (#76)
list.mapsnow allows the usage of
..2, etc. to refer to unnamed arguments. (#80)
list.loadnow supports merging and ungrouping as means to aggregating loaded results. (#82)
data.table = FALSE, which is done by reference and thus has higher performance.
nas the number of returned vector rather than that of the elements in all returned vectors, and is now able to jump out when the result set reaches given capacity. (#47, #84)
list.anybehave the same as
anyrespectively when the input is empty. (#87)
list.flattennow preserves names as specified. (#90)
list.groupworking with multi-key. (#93)
list.sortencounter such situation, they now report error rather than silently produced unreliable results. (#94)
equal()is removed and related packages are now suggested rather than imported. (#70)
summary.list()is deprecated. (#70)
x -> f(x)as a form a lambda expression. Use
x ~ f(x)instead. (#54)
desc(x)is no longer supported in
list.search now evaluates expression recursively in a list and supports lambda expression.
equal() function for logical and fuzzy filtering and searching which supports exact equality, atomic equality, inclusion, pattern matching, string-distance tolerance.
List() to provide an environment in which most list functions are defined for light-weight chaining that does not rely on external operators.
list.apply which is a wrapper function of lapply.
list.search that searches a list recursively.
Add exact search functions:
Add fuzzy search functions:
unlike based on stringdist package.
list.clean which now supports recursive cleaning.
list.common that returns the common cases of all list member by expression.
Improve performance (#26, #27)
list.flatten that flattens a nested list to one-level.
list.stack that binds list members to a data.frame.
list.zip that combines multiple lists element-wisely.
list.maps that performs mapping over multiple lists.
Minor maintainence updates.
list.cases supports list-like cases
list.select no longer accepts explicit lambda expressions.