PESTO

Parameter ESTimation Optimised – the PEST approach (Doherty 2015), brought to R and optimised, with APSIM as its flagship simulator partner.

PESTO is a high-performance R package for model-independent parameter estimation and uncertainty quantification. It brings the algorithms of PEST (Parameter ESTimation; Doherty 2015) and its C++ successor PEST++ (White et al. 2020) – notably the iterative ensemble smoother (IES) – natively into R, and is the first R-native implementation of that algorithm family. It adds a typed forward-model contract, an in-process simulator callback, multi-fidelity acceleration, covariance inflation / localisation, and surrogate methods.

PESTO is model-independent: any R function mapping parameters to outputs can be the forward model. Its flagship partner is APSIM, the agricultural-systems simulator, coupled in-process through the apsimx package – the use case PESTO’s callback was built for. The same callback couples any other process-based simulator just as directly: the groundwater / hydrology models of PEST’s own heritage, other crop models, and compartmental ODE systems.

Key Features

Feature	Description
Fast ensemble solvers	C++ (RcppEigen) implementation of IES and GLM update equations
In-process callback	Couple any R forward model directly – no file-exchange overhead
APSIM coupling	First-class apsim_callback() adapter drives apsimx ensembles in-process – PESTO’s flagship simulator partner
Adaptive SVD	Automatic backend selection: randomised SVD, LAPACK, or Eigen BDCSVD
Surrogate-accelerated IES	Gaussian-process surrogates classify which realisations could skip the model – projected savings are regime-dependent (large on smooth responses; near-zero with graceful fallback otherwise); the projection is not yet a realised compute reduction
Inflation & localisation	Mitigate ensemble under-dispersion and spurious correlations (partial remedy; see the Inflation and localisation vignette for the current calibration limits)
Adaptive ensemble sizing	ESS-based diagnostics prevent over/under-sampling
PEST++ integration	Read/write .pst control files, run PEST++ executables from R
Publication-ready plots	Convergence, ensemble distributions, identifiability, surrogate diagnostics

Performance

The optimised in the name is concrete – three measured properties, with the full definitions and maths in the Benchmarking PESTO against PEST and PEST++ vignette:

• Faster, at matched accuracy, not matched calibration. Against PEST 18.25 and pestpp-ies 5.2.16 on identical problems, PESTO is roughly 40–860x faster in wall-clock (median per inversion) while matching their posterior accuracy (RMSE to the true parameters) – the cause is in-process evaluation and a C++ update kernel avoiding per-solve file exchange, not fewer model solves. Raw-interval calibration is not matched: see the caveat below.
• Recovers the right answer. It recovers known-true parameters in a twin experiment and brackets the independent apsimx optimiser’s optimum on real observed data (the APSIM case study vignette).
• Fewer expensive solves, projected. The GP surrogate classifies which realisations are confident enough to skip the model; the classification is regime-dependent with a graceful fall-back to full evaluation, but the current release still evaluates every realisation regardless of the classification – so the saving is a projection, not yet a measured reduction in forward-model calls (the Surrogate-accelerated IES vignette).

Where PESTO does not win is stated just as plainly in the vignettes: total solve count is unchanged – the speed-up is wall-clock, not fewer evaluations (the Benchmarking PESTO against PEST and PEST++ vignette). That benchmark’s frozen table also carries a CI90 of about 0.13 against a nominal 0.90 for PESTO’s raw ensemble intervals. That figure predates the current development version: the smoother assimilated the same unperturbed observation vector into every realisation, which is now fixed by per-realisation observation perturbation under an ES-MDA schedule, graded against the closed-form conjugate posterior in both moments. Re-running the benchmark needs the external PEST and PEST++ binaries, so its calibration row still shows the old behaviour.

Related packages

• FME fits models to data and runs Monte Carlo, sensitivity and identifiability analysis, mainly for differential-equation models solved with deSolve or rootSolve.
• BayesianTools provides general-purpose MCMC and SMC samplers, with diagnostics, for calibrating complex system models.
• calibrar estimates the parameters of complex and stochastic models by optimisation, with the same syntax as optim().

PESTO uses the iterative ensemble smoother from PEST++ instead: the whole prior ensemble is updated against the observations at each iteration, and the result is an approximate posterior ensemble rather than a single best fit or a long chain. It also reads and writes PEST control files, so a problem set up for PEST or PEST++ can be run from R.

Installation

The canonical home for PESTO is max578/PESTO, published through the author’s personal CRAN-track channel and the max578.r-universe.dev registry. The AAGI-AUS/PESTO remote is retained as a read-only mirror.

# Development version from GitHub
# install.packages("pak")
pak::pak("max578/PESTO")

# Pre-built binaries from r-universe
install.packages("PESTO", repos = c(
  "https://max578.r-universe.dev",
  "https://cloud.r-project.org"
))

CRAN submission is in preparation.

Vignettes. GitHub source installs (pak, install_github) do not build vignettes by default, so browseVignettes("PESTO") will be empty. Install from r-universe (above) for pre-built vignettes, or add build_vignettes = TRUE to an install_github() call. All articles are also on the package website.

Usage

The examples below calibrate an R forward model, couple a simulator, and call the update kernels directly.

Bayesian inference with the Iterative Ensemble Smoother (IES)

PESTO performs approximate Bayesian inference for parameter estimation. You specify a prior as a parameter ensemble – a matrix of draws encoding your prior belief about each parameter (rows are realisations, columns are parameters) – together with the observations and their error standard deviation. PESTO conditions the ensemble on the observations through the IES update and returns an approximate posterior ensemble with uncertainty diagnostics. Any R function mapping a parameter matrix to an observation matrix can be the forward model.

library(PESTO)
set.seed(1)

# Forward model: y = G %*% theta (any R function theta-matrix -> obs-matrix works)
G <- matrix(rnorm(6 * 3), 6, 3)
forward <- function(theta) theta %*% t(G)
truth <- c(a = 1, b = -0.5, c = 2)
obs <- stats::setNames(as.numeric(G %*% truth) + rnorm(6, sd = 0.05),
                       paste0("o", 1:6))

# Prior: an ensemble of parameter draws (here a broad Gaussian over 3 parameters)
prior <- matrix(rnorm(80 * 3), 80, 3, dimnames = list(NULL, names(truth)))

fit <- pesto_ies_callback(forward, prior, obs, obs_sd = 0.05, noptmax = 6,
                          verbose = FALSE)
colMeans(as.matrix(fit$par_ensemble[, -1]))   # posterior mean -> (1, -0.5, 2)
#>          a          b          c 
#>  1.0155313 -0.4949939  1.9845372

Optional convergence-based early stopping (phi_tol), covariance inflation (pesto_inflation()), and localisation (pesto_localisation()) are documented in ?pesto_ies_callback and the Getting started vignette.

Coupling to APSIM and other simulators

The forward model above is a plain R function. To calibrate a process-based simulator, supply an adapter that maps a parameter vector to that simulator’s outputs. PESTO’s flagship partner is APSIM: apsim_callback() turns an .apsimx model file plus a parameter map into exactly the forward-model callable the IES driver expects, running the ensemble in-process via apsimx.

# Requires the apsimx package and a working APSIM installation.
fm <- apsim_callback(
  template         = "wheat.apsimx",
  param_map        = list(rue = "[Wheat].Leaf.Photosynthesis.RUE.FixedValue"),
  output_extractor = function(sim) sim$Wheat.AboveGround.Wt
)
fit <- pesto_ies_callback(fm, prior, obs, obs_sd = 0.05, noptmax = 6)

The full worked example is in the Calibrating APSIM with PESTO vignette (it needs APSIM installed, so its run blocks are disabled on CRAN). The same pesto_ies_callback() pattern couples any other simulator: a CRAN-safe, fully-runnable agricultural demonstration using the built-in crop_growth_forward_model() is in that vignette, and the apsim-callback article shows Python-bridge and multi-fidelity variants.

Low-level kernels

The C++ update kernels are exported for advanced use (no PEST++ binary needed):

set.seed(42)
npar <- 100; nobs <- 200; nreal <- 50
upgrade <- ensemble_solution(
  par_diff   = matrix(rnorm(npar * nreal), npar, nreal),
  obs_diff   = matrix(rnorm(nobs * nreal), nobs, nreal),
  obs_resid  = matrix(rnorm(nobs * nreal), nobs, nreal),
  par_resid  = matrix(rnorm(npar * nreal), npar, nreal),
  weights    = abs(rnorm(nobs)) + 0.1,
  parcov_inv = abs(rnorm(npar)) + 0.1,
  Am         = matrix(rnorm(npar * (nreal - 1)), npar, nreal - 1),
  cur_lam    = 1.0
)
dim(upgrade)
#> [1]  50 100

The classical .pst-file path is also supported:

pars <- data.table::data.table(
  parnme = paste0("k", 1:10), partrans = "log", parchglim = "factor",
  parval1 = runif(10, 0.1, 10), parlbnd = 0.001, parubnd = 1000,
  pargp = "hydraulic"
)
obs <- data.table::data.table(
  obsnme = paste0("h", 1:20), obsval = rnorm(20, 5, 1),
  weight = 1.0, obgnme = "heads"
)
pst <- create_pest_scenario(pars, obs, "python model.py")
write_pst(pst, file.path(tempdir(), "model.pst"))

Dependencies

• R >= 4.1.0
• C++17 compiler (clang on macOS, g++ on Linux, Rtools on Windows)
• LAPACK/BLAS (bundled with R)

R-package dependencies are declared in DESCRIPTION (Imports: Rcpp, data.table, ggplot2, S7, yaml, digest).

Documentation

• vignette("getting-started", package = "PESTO") – Introduction and basic usage
• vignette("apsim-callback", package = "PESTO") – Coupling APSIM (and other simulators) via the in-process callback
• vignette("apsim-case-study", package = "PESTO") – Publication-grade APSIM wheat calibration: synthetic-truth recovery + real-data calibration with uncertainty
• vignette("inflation-localisation", package = "PESTO") – Finite-ensemble countermeasures
• vignette("surrogate-ies", package = "PESTO") – Surrogate-accelerated IES
• vignette("pestpp-comparison-and-simulation", package = "PESTO") – Benchmark vs PEST/PEST++
• vignette("ensemble-manifest", package = "PESTO") – The reproducible run manifest

Contributing

Contributions are welcome. See CONTRIBUTING.md for the development workflow and pull-request convention; bug reports and feature requests go through GitHub Issues. All participants abide by the Code of Conduct.

Citation

citation("PESTO")

Moldovan, M. (2026). PESTO: Parameter ESTimation Optimised. R package version 0.11.1.9000. https://github.com/max578/PESTO

Acknowledgements

PESTO builds on the algorithmic legacy of the PEST++ project (US Geological Survey) and the underlying PEST framework by John Doherty. Developed at Adelaide University; the surrogate-acceleration, adaptive-ensemble-sizing, multi-fidelity, and convergence-aware components are original contributions of the author.

License

MIT. See LICENSE.md for the full text.