Nested Scales and Multi-Dimensional Indexes for Modeling

Provides multi-level discrete representations of data dimensions for modeling, data processing and visualization. A nested scale records its finest units (atoms) and their membership in coarser levels (frames), including nested, cross-cutting and partially covered relationships. Recasts tabular data between levels using explicit, direction-aware rules for sums, means, standard deviations, copying and within-parent shares. Also supports labeling, subsetting, combining scales into multi-dimensional indexes, clustering units and storing data with its scale metadata. Members may carry attributes (a calorific value, an emission factor) and aliases, resolved along the hierarchy and applied row by row, and a scale is exchanged as a language-neutral definition file: a 'JSON' header with inline, 'Arrow' or 'CSV' tables ('YAML' accepted), with a catalogue of the definitions installed packages ship. Forms the dimension-agnostic core of 'timescales' < https://github.com/optimal2050/timescales> (calendars) and 'geoscales' < https://github.com/optimal2050/geoscales> (regions), designed for optimization and simulation models.

nestedscales

nestedscales represents a discrete data dimension at multiple resolutions and recasts tabular data between them. A nested scale stores one row per finest-grained unit, or atom, and one column for each coarser grouping, or frame. Each frame assigns atoms to units; frames may form a hierarchy or cross-cut one another, including product-like overlays. All mappings are derived from membership at the atom level.

The package defines:

• Structure: the levels of a scale, the units at each level, and the relations between levels (nested, crossed, uniform).
• Transition rules between levels, chosen per value column and aware of direction: sum, weighted_mean, mean, and copy; sd for aggregation; and share or logshare for within-parent shares.
• Operations: converting data between levels, attaching level labels to a dataset, subsetting, combining several scales into one multi-dimensional index, clustering units into new levels, and storing data together with its scales.
• Attributes and definitions: values that belong to a member but are not weights (a calorific value, an emission factor) and alternative labels, resolved along the hierarchy and applied row by row; and language-neutral definition files, with a catalogue of the ones installed packages ship.

The package is dimension-agnostic. Time and space are provided by two packages built on it, designed for optimization and simulation models, where labeled time slices and regions serve as the index sets of variables and constraints:

• timescales: Calendar, labeled discrete time slices (hours, days, seasons, representative days).
• geoscales: GeoScale, labeled geographical regions with geometry.

Both classes are subclasses of NestedScale. Any other dimension (industries, income brackets, technology vintages) is declared directly from a table. vignette("nestedscales") introduces the functions; the package website includes a roadmap of what is available and what is planned.

Installation

Until the first CRAN release, install the development version from GitHub:

pak::pkg_install("optimal2050/nestedscales")

The package is pure R; its non-base runtime dependencies are dplyr, rlang, S7 and jsonlite.

A scale in one call

Declare the levels, coarsest first:

library(nestedscales)

industries <- data.frame(
  section  = c("C", "C", "C", "D", "D"),
  division = c("C10", "C10", "C11", "D35", "D35"),
  class    = c("C101", "C102", "C110", "D351", "D352"),
  gva      = c(120, 80, 45, 300, 150)
)

ind <- scale_from_leaftable(
  industries,
  frames = c("section", "division", "class"),
  weights = "gva",
  name   = "nace"
)

ind
#> NestedScale: nace
#> Frames (3, coarsest first):
#>   - section (2)
#>     - division (3)
#>       - class (5)
#> Atoms: 5
#> Weights: gva (default: gva)

Every structural question is answered from the atoms, never from an assumed tree:

scale_frames(ind)
#> [1] "section"  "division" "class"
scale_units(ind, "division")
#> [1] "C10" "C11" "D35"
scale_nests(ind, "section", "division")
#> [1] TRUE

Hierarchies that cross-cut are normal and supported – one detailed code may belong to several aggregates. scale_nests(x, parent, child) tests whether every child code belongs to a single parent code; conversion never depends on that relationship.

summary(ind)
#> <summary of NestedScale 'nace'>
#>   frames:         section (2) / division (3) / class (5)
#>   atoms:          5
#>   weight totals:  gva = 695  (default: gva)
#>   nesting:        section > division: nested
#>   nesting:        division > class: nested

Moving data between levels

Data keyed at one level is recast to another with a rule per value column. With complete coverage, "sum" conserves totals; intensive quantities use "weighted_mean" with a declared weight:

x <- data.frame(
  class = c("C101", "C102", "C110", "D351", "D352"),
  jobs  = c(12, 8, 5, 30, 20),
  wage  = c(31, 29, 27, 45, 40)
)

recast_scale(x, ind, source = "class", target = "section",
             rule = c(jobs = "sum", wage = "weighted_mean"),
             weight = "gva")
#>   section jobs     wage
#> 1       C   25 29.61224
#> 2       D   50 43.33333

A shipped definition

Scales travel as definition files (a JSON header with the tables inline, or a bundle with Arrow or CSV tables) that the Python binding reads as well. The package ships one: the energypal fuel taxonomy with the IPCC 2006 default calorific values and CO2 factors attached as attributes.

carriers <- read_scale("carriers")
carriers
#> NestedScale: carriers
#> Description: Energy carriers: fuels and energy vectors with subtypes, from energypal's carriers palette (IPCC 2006 / IEA classifications) 
#> Frames (3, coarsest first):
#>   - group (8)
#>     - carrier (17)
#>       - fuel (71)
#> Atoms: 71
#> Attributes: lhv, ef_co2 (sources: IPCC2006) -- 26 at fuel
#> Aliases: 136 (energypal, commodity)
#> Source: energypal carriers.yml

fuel_use <- data.frame(fuel = c("Anthracite", "NaturalGas"), t = c(100, 100))
convert_scale(fuel_use, carriers, "lhv", from_unit = "t", to_unit = "GJ")
#>         fuel    t  lhv lhv.origin_frame
#> 1 Anthracite 2670 26.7             fuel
#> 2 NaturalGas 4800 48.0             fuel

scale_attributes() resolves a value for every member of a frame, walking to the nearest coarser level that holds one; scale_match() resolves labels and model codes through the alias table, strictly.

Design

• Atoms are the ground truth. Frames are partitions of the atoms, not necessarily a strict tree; conversion always routes through the atom layer.
• Weights are named and plural. A scale can carry gva, employment and count side by side, with one declared default.
• Attributes are not weights. A weight says how big an atom is and drives aggregation; an attribute is a per-member factor, consumed row by row and never summed.
• Partial coverage is first class. An atom with no code at a frame is NA, not an error, and a subset records what fraction of its parent it keeps.
• Backend-agnostic. The verbs use the dplyr interface, so the same pipeline runs on a data.frame, a data.table, or an arrow dataset; lazy inputs stay lazy.

Related packages

• scales maps data to graphical aesthetics; its discrete scales are palettes and label formatters, not resolutions of a model dimension.
• data.tree represents strict trees as node objects. nestedscales keeps the hierarchy as a flat atom table, allows frames that cross-cut, and converts data between any two frames with weights.
• hts reconciles forecasts across a fixed aggregation tree; reconcile_scale() here compares data aggregated with sum or weighted_mean against independent totals and can balance discrepancies.

License

Apache License 2.0.