Bias Adjustment and Downscaling Algorithms
The xclim.sdba submodule provides a collection of bias-adjustment methods meant to correct for systematic biases found in climate model simulations relative to observations. Almost all adjustment algorithms conform to the train - adjust scheme, meaning that adjustment factors are first estimated on training data sets, then applied in a distinct step to the data to be adjusted. Given a reference time series (ref), historical simulations (hist) and simulations to be adjusted (sim), any bias-adjustment method would be applied by first estimating the adjustment factors between the historical simulation and the observation series, and then applying these factors to sim, which could be a future simulation:
# Create the adjustment object by training it with reference and model data, plus certain arguments
Adj = Adjustment.train(ref, hist, group="time.month")
# Get a scenario by applying the adjustment to a simulated timeseries.
scen = Adj.adjust(sim, interp="linear")
Adj.ds.af # adjustment factors.
Most method support both additive and multiplicative correction factors. Also, the group argument allows adjustment factors to be estimated independently for different periods: the full time series, months, seasons or day of the year. For monthly groupings, the interp argument then allows for interpolation between adjustment factors to avoid discontinuities in the bias-adjusted series. See Grouping below.
The same interpolation principle is also used for quantiles. Indeed, for methods extracting adjustment factors by quantile, interpolation is also done between quantiles. This can help reduce discontinuities in the adjusted time series, and possibly reduce the number of quantile bins that needs to be used.
Modular Approach
The module attempts to adopt a modular approach instead of implementing published and named methods directly. A generic bias adjustment process is laid out as follows:
preprocessing on
ref,histandsim(using methods inxclim.sdba.processingorxclim.sdba.detrending)creating and training the adjustment object
Adj = Adjustment.train(obs, sim, **kwargs)(fromxclim.sdba.adjustment)adjustment
scen = Adj.adjust(sim, **kwargs)post-processing on
scen(for example: re-trending)
The train-adjust approach allows to inspect the trained adjustment object. The training information is stored in the underlying Adj.ds dataset and usually has a af variable with the adjustment factors. Its layout and the other available variables vary between the different algorithm, refer to Adjustment methods.
Parameters needed by the training and the adjustment are saved to the Adj.ds dataset as a adj_params attribute.
Parameters passed to the adjust call are written to the history attribute in the output scenario DataArray.
Grouping
For basic time period grouping (months, day of year, season), passing a string to the methods needing it is sufficient.
Most methods acting on grouped data also accept a window int argument to pad the groups with data from adjacent ones.
Units of window are the sampling frequency of the main grouping dimension (usually time). For more complex grouping,
one can pass an instance of xclim.sdba.base.Grouper directly. For example, if one wants to compute the factors
for each day of the year but across all realizations of an ensemble : group = Grouper("time.dayofyear", add_dims=['realization']).
In a conventional empirical quantile mapping (EQM), this will compute the quantiles for each day of year and all realizations together, yielding a single set of adjustment factors for all realizations.
Warning
If grouping according to the day of the year is needed, the xclim.core.calendar submodule contains useful
tools to manage the different calendars that the input data can have. By default, if 2 different calendars are
passed, the adjustment factors will always be interpolated to the largest range of day of the years but this can
lead to strange values, so we recommend converting the data beforehand to a common calendar.
Application in multivariate settings
When applying univariate adjustment methods to multiple variables, some strategies are recommended to avoid introducing unrealistic artifacts in adjusted outputs.
Minimum and maximum temperature
When adjusting both minimum and maximum temperature, adjustment factors sometimes yield minimum temperatures larger than the maximum temperature on the same day, which of course, is nonsensical. One way to avoid this is to first adjust maximum temperature using an additive adjustment, then adjust the diurnal temperature range (DTR) using a multiplicative adjustment, and then determine minimum temperature by subtracting DTR from the maximum temperature [Agbazo and Grenier, 2020, Thrasher et al., 2012].
Relative and specific humidity
When adjusting both relative and specific humidity, we want to preserve the relationship between both. To do this, Grenier [2018] suggests to first adjust the relative humidity using a multiplicative factor, ensure values are within 0-100%, then apply an additive adjustment factor to the surface pressure before estimating the specific humidity from thermodynamic relationships.
Radiation and precipitation
In theory, short wave radiation should be capped when precipitation is not zero, but there is as of yet no mechanism proposed to do that, see Hoffmann and Rath [2012].
Usage examples
The usage of this module is documented in two example notebooks: SDBA and SDBA advanced.
Discussion topics
Some issues were also discussed on the Github repository. Most of these are still open questions, feel free to participate to the discussion!
Experimental wrap of SBCK
The SBCK python package implements various bias-adjustment methods, with an emphasis on multivariate methods and with
a care for performance. If the package is correctly installed alongside xclim, the methods will be wrapped into
xclim.sdba.adjustment.Adjust classes (names beginning with SBCK_) with a minimal overhead so that they can
be parallelized with dask and accept xarray objects. For now, these experimental classes can’t use the train-adjust
approach, instead they only provide one method, adjust(ref, hist, sim, multi_dim=None, **kwargs) which performs all
steps : initialization of the SBCK object, training (fit) and adjusting (predict). All SBCK wrappers accept a
multi_dim argument for specifying the name of the “multivariate” dimension. This wrapping is still experimental and
some bugs or inconsistencies might exist. To see how one can install that package, see Extra Dependencies.
Notes for Developers
To be scalable and performant, the sdba module makes use of the special decorators :py:func`xclim.sdba.base.map_blocks`
and xclim.sdba.base.map_groups(). However, they have the inconvenient that functions wrapped by them are unable
to manage xarray attributes (including units) correctly and their signatures are sometime wrong and often unclear. For
this reason, the module is often divided in two parts : the (decorated) compute functions in a “private” file
(ex: _adjustment.py) and the user-facing functions or objects in corresponding public file (ex: adjustment.py).
See the sdba-advanced notebook for more info on the reasons for this move.
Other restrictions : map_blocks will remove any “auxiliary” coordinates before calling the wrapped function and will
add them back on exit.
User API
See: SDBA Module
Developer API
See: SDBA Utilities
SDBA Footnotes
Mégane Alavoine and Patrick Grenier. The distinct problems of physical inconsistency and of multivariate bias involved in the statistical adjustment of climate simulations. International Journal of Climatology, pages 1–23, September 2022. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/joc.7878. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.7878 (visited on 2022-11-16), doi:10.1002/joc.7878.
L. Baringhaus and C. Franz. On a new multivariate two-sample test. Journal of Multivariate Analysis, 88(1):190–206, January 2004. URL: https://www.sciencedirect.com/science/article/pii/S0047259X03000794 (visited on 2022-07-29), doi:10.1016/S0047-259X(03)00079-4.
Alex J. Cannon. Multivariate quantile mapping bias correction: an N-dimensional probability density function transform for climate model simulations of multiple variables. Climate Dynamics, 50(1):31–49, January 2018. URL: https://doi.org/10.1007/s00382-017-3580-6 (visited on 2022-07-29), doi:10.1007/s00382-017-3580-6.
Alex J. Cannon. MBC: Multivariate Bias Correction of Climate Model Outputs. October 2020. URL: https://CRAN.R-project.org/package=MBC (visited on 2022-07-29).
Alex J. Cannon, Stephen R. Sobie, and Trevor Q. Murdock. Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? Journal of Climate, 28(17):6938–6959, September 2015. Publisher: American Meteorological Society Section: Journal of Climate. URL: https://journals.ametsoc.org/view/journals/clim/28/17/jcli-d-14-00754.1.xml (visited on 2022-07-29), doi:10.1175/JCLI-D-14-00754.1.
William S. Cleveland. Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association, 74(368):829–836, December 1979. Publisher: Taylor & Francis _eprint: https://www.tandfonline.com/doi/pdf/10.1080/01621459.1979.10481038. URL: https://www.tandfonline.com/doi/abs/10.1080/01621459.1979.10481038 (visited on 2022-07-29), doi:10.1080/01621459.1979.10481038.
Michel Déqué. Frequency of precipitation and temperature extremes over France in an anthropogenic scenario: Model results and statistical correction according to observed values. Global and Planetary Change, 57(1):16–26, May 2007. URL: https://www.sciencedirect.com/science/article/pii/S0921818106002748 (visited on 2022-07-29), doi:10.1016/j.gloplacha.2006.11.030.
Alexandre Gramfort. LOWESS : Locally weighted regression. October 2015. URL: https://gist.github.com/agramfort/850437 (visited on 2022-08-05).
Jan Hnilica, Martin Hanel, and Vladimír Puš. Multisite bias correction of precipitation data from regional climate models. International Journal of Climatology, 37(6):2934–2946, 2017. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/joc.4890. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.4890 (visited on 2022-07-29), doi:10.1002/joc.4890.
Jonathan Jalbert. Extreme value analysis package for Julia. June 2022. original-date: 2019-01-20T03:26:32Z. URL: https://github.com/jojal5/Extremes.jl (visited on 2022-07-29).
Francesco Mezzadri. How to generate random matrices from the classical compact groups. February 2007. arXiv:math-ph/0609050 version: 2. URL: https://arxiv.org/abs/math-ph/0609050 (visited on 2022-11-16), doi:10.48550/arXiv.math-ph/0609050.
F. Pitie, A.C. Kokaram, and R. Dahyot. N-dimensional probability density function transfer and its application to color transfer. In Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, volume 2, 1434–1439 Vol. 2. October 2005. ISSN: 2380-7504. doi:10.1109/ICCV.2005.166.
Philippe Roy, Gabriel Rondeau-Genesse, Jonathan Jalbert, and Élyse Fournier. Climate scenarios of extreme precipitation using a combination of parametric and non-parametric bias correction methods. Submitted to Climate Services, april 2021.
Philippe Roy, Trevor James Smith, Tony Kelman, Éloïse Nolet-Gravel, Elliot Saba, Fidel Thomet, Julia TagBot, and Gael Forget. JuliaClimate/ClimateTools.jl: v0.23.1. September 2021. URL: https://zenodo.org/record/5399172 (visited on 2022-07-29), doi:10.5281/zenodo.5399172.
Jürg Schmidli, Christoph Frei, and Pier Luigi Vidale. Downscaling from GCM precipitation: a benchmark for dynamical and statistical downscaling methods. International Journal of Climatology, 26(5):679–689, 2006. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/joc.1287. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.1287 (visited on 2022-07-29), doi:10.1002/joc.1287.
Gabor Szekely and Maria Rizzo. Testing for equal distributions in high dimension. InterStat, November 2004.
Matthias Jakob Themeßl, Andreas Gobiet, and Georg Heinrich. Empirical-statistical downscaling and error correction of regional climate models and its impact on the climate change signal. Climatic Change, 112(2):449–468, May 2012. URL: https://doi.org/10.1007/s10584-011-0224-4 (visited on 2022-07-29), doi:10.1007/s10584-011-0224-4.