Software tools and code
This web page forms Deliverable 6.1 from the QUICS project: "Database of openly accessible software tools".
During their research, most fellows wrote code in languages such as Python, Matlab and R. Where the codes were likely to be useful to others, and in a format capable of being used, they have been shared through various different sites.
The table below provides a list of these tools, the fellows involved and a link to the tool. Click on the tool name for further details, including links to associated reports and papers.
EmiStatR: This R-package provides a fast and parallelised calculator to estimate wastewater emissions from combined sewers. It also supports the planning and design of urban drainage systems, without the requirement of extensive simulation tools.
stUPscales (Spatio-temporal Uncertainty Propagation across scales) - formerly known as AggProp: Provides several R methods and functions for temporal aggregation of environmental variables as precipitation and pollutants of Urban Drainage Models (UDMs). Also, it provides methods and functions for uncertainty propagation for lumped UDMs via Monte Carlo simulation.
These tools are described in deliverable report 2.3, although have continued to be developed so also so the journal papers mentioned for each tool.
The R package ‘spup’ implements uncertainty propagation analysis in spatial environmental modelling following the methodology described in Heuvelink et al. (2007) and Brown and Heuvelink (2007). The package provides functions for examining uncertainty propagation starting from input data and model parameters, via the environmental model onto model outputs. The functions include uncertainty model specification, stochastic simulation and propagation of uncertainty using Monte Carlo (MC) techniques. Uncertain variables are described by probability distributions. Both numerical and categorical data types are handled. Spatial auto-correlation within an attribute and cross-correlation between attributes is accommodated for. The MC realizations may be used as input to the environmental models called from R, or externally.
The package is available on CRAN.
The main author of this tool is Tobias Houska. Within the QUICS project, Carla has used and added to this tool.
A Statistical Parameter Optimization Tool, SPOTPY is a Python tool that enables the use of Computational optimization techniques for calibration, uncertainty and sensitivity analysis techniques of almost every (environmental) model. The package is puplished in the open source journal PLoS One: Houska, T, Kraft, P, Chamorro-Chavez, A and Breuer, L; SPOTting Model Parameters Using a Ready-Made Python Package; PLoS ONE; 2015 The simplicity and flexibility enables the use and test of different algorithms without the need of complex codes, doi: 10.1371/journal.pone.0145180.
Ensemble Generator.R: Geostatistical Radar Ensemble Generator (GREG)
KEDUD.R: Kriging with External Drift for Uncertain Data (KEDUD)
KED_with_optimal_BoxCox.R: KED radar-gauge merging with optimised Box-Cox lambda
Negentropy.R: KED radar-gauge merging with optimised Box-Cox lambda
OKUD.R: Ordinary Kriging for Uncertain Data (OKUD)
QQplots_function.R: KED radar-gauge merging with optimised Box-Cox lambda
RG_validation_function.R: KED radar-gauge merging with optimised Box-Cox lambda
Residual_gaussianity_function.R: KED radar-gauge merging with optimised Box-Cox lambda
Python scripts for water quality simulators WEST (MIKE DHI). Sensitivity analysis, inference and automated runs.
The scripts are available on Github.
The novelty of this work is the consideration of (short) time series for the simulation inputs and outputs. Such simulation setup is interesting in applications such as Model Predictive Control (MPC) in which numerous, fast and frequent simulation results are required. Here, an emulator is developed to predict a storage tank’s volume in a small case study in Luxembourg. Three main inputs are considered as the GPE’s parameters: initial volume in the tank, the level in which the outlet pump of the tank must start to work, and the time series of expected rainfall in the upcoming 2 hours. The output of interest is the total volume of the storage tank for the next 24 hours.
The code (in R) and the simulation and validation data-sets will be available after publication of this research.
Temporal Uncertainty Propogation Asross Scales, stUPscales, provides several R functions for temporal aggregation of enviornmental variables used in Urban Drainage Models, such as precipitation and pollutants. It also provides methods and functions for ucertainty propgation via Monte Carlo simulation; and provides specific analysis functions for urban drainage system simulation to evaluate water quantity and quality in combined sewer overflows (CSOs).
The package is available on Github.
One challenge in uncertainty estimation of environmental models is the high number of parameters, as well as model structure errors. The first make traditional optimisation routines difficult due to co-linearities of model parameters. The second often require additional error terms which capture these structural deficits with stochastic processes.
Inference with the latter requires Bayesian inference methods for model calibration, which typically rely on sampling from the posterior. Obtaining a reliable estimate of the posterior usually requires many thousand iterative model runs and there is the danger that sampling a very complex posterior distribution is inadequate due to too few iterations.
Efficient samplers have been proposed to relax this problem by increasing the acceptance rate due to sophisticated proposal algorithms, such as adaptive sampling or parallelizing the inference with ensembles.
Three open source computational libraries of efficient algorithms for adaptive (1) and ensemble-based MCMC (Monte Carlo Markov Chain) sampling (2). In addition, we explore and compare the effectiveness of each sampler in standardized inference problems are presented. For full details, see Deliverable 1.2.
These codes are closely linked to Deliverable 3.1 and the journal papers "Sampling design optimisation for rainfall prediction using a non-stationary geostatistical model" which is Open Access published in Advances in Water Resources and "Accounting for non-stationary variance in geostatistical mapping of soil properties" which is under review.
The codes are available on Github.
This software tool has been developed to enable evaluation of structural and parameter uncertainty in models to describe pollutants in rivers. This tool can therefore assist in selecting appropriate river pollutant transport models.
Research Excellence Framework 2021 results
The results demonstrate our research and impact excellence across a broad range of disciplines and confirm that our research is having a significant positive impact on lives across the globe.