Skip to content

Commit

Permalink
Update the documentation + project maintenance (#70)
Browse files Browse the repository at this point in the history
  • Loading branch information
btschwertfeger authored Mar 14, 2024
1 parent 00f4152 commit 84fb40d
Show file tree
Hide file tree
Showing 22 changed files with 188 additions and 92 deletions.
10 changes: 10 additions & 0 deletions .github/dependabot.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
7 changes: 5 additions & 2 deletions .github/workflows/_build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@

name: Build

permissions:
contents: read

on:
workflow_call:
inputs:
Expand All @@ -23,10 +26,10 @@ jobs:
runs-on: ${{ inputs.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0

- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: ${{ inputs.python-version }}

Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/_build_doc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

name: Build Doc

permissions:
contents: read

on:
workflow_call:
inputs:
Expand All @@ -22,10 +25,10 @@ jobs:
runs-on: ${{ inputs.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0

- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: ${{ inputs.python-version }}

Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/_codecov.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@

name: CodeCov

permissions:
contents: read

on:
workflow_call:
inputs:
Expand All @@ -32,10 +35,10 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: ${{ inputs.python-version }}

Expand All @@ -49,7 +52,7 @@ jobs:
run: pytest --cov --cov-report=xml

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@54bcd8715eee62d40e33596ef5e8f0f48dbbccab #v4.1.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.xml
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/_codeql.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/_pre_commit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

name: Pre-Commit

permissions:
contents: read

on:
workflow_call:

Expand All @@ -15,5 +18,5 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: pre-commit/[email protected].0
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
7 changes: 5 additions & 2 deletions .github/workflows/_pypi_publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@

name: PyPI Publish

permissions:
contents: read

on:
workflow_call:
inputs:
Expand All @@ -28,12 +31,12 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0
with:
fetch-depth: 0 # IMPORTANT: otherwise the current tag does not get fetched and the build version gets worse

- name: Set up Python 3.11
uses: actions/setup-python@v5
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: 3.11

Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

name: Test Spot

permissions:
contents: read

on:
workflow_call:
inputs:
Expand All @@ -23,10 +26,10 @@ jobs:
runs-on: ${{ inputs.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0

- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: ${{ inputs.python-version }}

Expand Down
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,6 @@ clean:
find tests -name "__pycache__" | xargs rm -rf
find cmethods -name "__pycache__" | xargs rm -rf
find examples -name "__pycache__" | xargs rm -rf

# for file in `ls .github/workflows`; do sed -i '' 's/actions\/checkout@v4/actions\/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 # v4.0.0/g' .github/workflows/$file; done
# for file in `ls .github/workflows`; do sed -i '' 's/actions\/setup-python@v5/actions\/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0/g' .github/workflows/$file; done
94 changes: 52 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,27 +10,25 @@
[![CI/CD](https://github.com/btschwertfeger/python-cmethods/actions/workflows/cicd.yaml/badge.svg?branch=master)](https://github.com/btschwertfeger/python-cmethods/actions/workflows/cicd.yaml)
[![codecov](https://codecov.io/github/btschwertfeger/python-cmethods/branch/master/graph/badge.svg?token=OSO4PAABPD)](https://codecov.io/github/btschwertfeger/python-cmethods)

[![OpenSSF ScoreCard](https://img.shields.io/ossf-scorecard/github.com/btschwertfeger/python-cmethods?label=openssf%20scorecard&style=flat)](https://securityscorecards.dev/viewer/?uri=github.com/btschwertfeger/python-cmethods)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8666/badge)](https://www.bestpractices.dev/projects/8666)

![release](https://shields.io/github/release-date/btschwertfeger/python-cmethods)
![release](https://shields.io/github/v/release/btschwertfeger/python-cmethods?display_name=tag)
[![DOI](https://zenodo.org/badge/496160109.svg)](https://zenodo.org/badge/latestdoi/496160109)
[![Documentation Status](https://readthedocs.org/projects/python-cmethods/badge/?version=stable)](https://python-cmethods.readthedocs.io/en/latest/?badge=stable)

</div>

This Python module serves as a collection of different scale- and
distribution-based bias correction techniques for climate sciences.

The documentation is available at: [https://python-cmethods.readthedocs.io/en/stable/](https://python-cmethods.readthedocs.io/en/stable/)
Welcome to python-cmethods, a powerful Python package designed for bias
correction and adjustment of climate data. Built with a focus on ease of use and
efficiency, python-cmethods offers a comprehensive suite of functions tailored
for applying bias correction methods to climate model simulations and
observational datasets.

Please cite this project as described in
https://zenodo.org/doi/10.5281/zenodo.7652755.

> ⚠️ For the application of bias corrections on _large data sets_ it is
> recommended to also try the command-line tool
> [BiasAdjustCXX](https://github.com/btschwertfeger/BiasAdjustCXX).
---

## Table of Contents

1. [ About ](#about)
Expand All @@ -41,16 +39,16 @@ https://zenodo.org/doi/10.5281/zenodo.7652755.
6. [ Contribution ](#contribution)
7. [ References ](#references)

---

<a name="about"></a>

## 1. About

These programs and data structures are developed with the aim of reducing
discrepancies between modeled and observed climate data. Historical data is
utilized to calibrate variables from current and future time series to achieve
distributional properties that closely resemble the possible actual values.
Bias correction in climate research involves the adjustment of systematic errors
or biases present in climate model simulations or observational datasets to
improve their accuracy and reliability, ensuring that the data better represents
actual climate conditions. This process typically involves statistical methods
or empirical relationships to correct for biases caused by factors such as
instrument calibration, spatial resolution, or model deficiencies.

<figure>
<img
Expand All @@ -60,12 +58,24 @@ distributional properties that closely resemble the possible actual values.
<figcaption>Figure 1: Schematic representation of a bias adjustment procedure</figcaption>
</figure>

For instance, modeled data typically indicate values that are colder than the
actual values. To address this issue, an adjustment procedure is employed. The
figure below illustrates the observed, modeled, and adjusted values, revealing
that the delta adjusted time series ($T^{*DM}_{sim,p}$) are significantly more
similar to the observed data ($T{obs,p}$) than the raw modeled data
($T_{sim,p}$).
python-cmethods empowers scientists to effectively address those biases in
climate data, ensuring greater accuracy and reliability in research and
decision-making processes. By leveraging cutting-edge techniques and seamless
integration with popular libraries like [xarray](https://xarray.dev/) and
[Dask](https://docs.dask.org/en/stable/), this package simplifies the process
of bias adjustment, even when dealing with large-scale climate simulations and
extensive spatial domains.

In this way, for example, modeled data, which on average represent values that
are too cold, can be easily bias-corrected by applying any adjustment procedure
included in this package.

For instance, modeled data can report values that are way colder than the those
data reported by reanalysis time-series. To address this issue, an adjustment
procedure can be employed. The figure below illustrates the observed, modeled,
and adjusted values, revealing that the delta-adjusted time series
($T^{*DM}_{sim,p}$) is significantly more similar to the observational data
($T{obs,p}$) than the raw model output ($T_{sim,p}$).

<figure>
<img
Expand All @@ -75,6 +85,12 @@ similar to the observed data ($T{obs,p}$) than the raw modeled data
<figcaption>Figure 2: Temperature per day of year in observed, modeled, and bias-adjusted climate data</figcaption>
</figure>

The mathematical foundations supporting each bias correction technique
implemented in python-cmethods are integral to the package, ensuring
transparency and reproducibility in the correction process. Each method is
accompanied by references to trusted publications, reinforcing the reliability
and rigor of the corrections applied.

<a name="methods"></a>

## 2. Available Methods
Expand All @@ -96,16 +112,16 @@ https://python-cmethods.readthedocs.io/en/stable/
non-stochastic climate variables. Variance scaling can only be applied on
non-stochastic climate variables.

- Non-stochastic climate variables are those that can be predicted with relative
certainty based on factors such as location, elevation, and season. Examples
of non-stochastic climate variables include air temperature, air pressure, and
solar radiation.
- Non-stochastic climate variables are those that can be predicted with
relative certainty based on factors such as location, elevation, and season.
Examples of non-stochastic climate variables include air temperature, air
pressure, and solar radiation.

- Stochastic climate variables, on the other hand, are those that exhibit a high
degree of variability and unpredictability, making them difficult to forecast
accurately. Precipitation is an example of a stochastic climate variable
because it can vary greatly in timing, intensity, and location due to complex
atmospheric and meteorological processes.
- Stochastic climate variables, on the other hand, are those that exhibit a
high degree of variability and unpredictability, making them difficult to
forecast accurately. Precipitation is an example of a stochastic climate
variable because it can vary greatly in timing, intensity, and location due
to complex atmospheric and meteorological processes.

- Except for the detrended quantile mapping (DQM) technique, all methods can be
applied to 1- and 3-dimensional data sets. The implementation of DQM to
Expand All @@ -114,7 +130,8 @@ https://python-cmethods.readthedocs.io/en/stable/
- Except for DQM, all methods can be applied using `cmethods.adjust`. Chunked
data for computing e.g. in a dask cluster is possible as well.

- For any questions -- please open an issue at https://github.com/btschwertfeger/python-cmethods/issues
- For any questions -- please open an issue at
https://github.com/btschwertfeger/python-cmethods/issues

<a name="installation"></a>

Expand All @@ -124,8 +141,6 @@ https://python-cmethods.readthedocs.io/en/stable/
python3 -m pip install python-cmethods
```

---

<a name="examples"></a>

## 4. Usage and Examples
Expand Down Expand Up @@ -178,8 +193,9 @@ Notes:

Notebook with different methods and plots: `/examples/examples.ipynb`

There is also an example script (`/examples/biasadjust.py`) that can be used to apply the available bias correction methods
on 1- and 3-dimensional data sets (see `/examples/input_data/*.nc`).
There is also an example script (`/examples/biasadjust.py`) that can be used to
apply the available bias correction methods on 1- and 3-dimensional data sets
(see `/examples/input_data/*.nc`).

Help:

Expand Down Expand Up @@ -219,8 +235,6 @@ Notes:
- This script is far away from perfect - so please see it, as a starting point.
(:

---

<a name="notes"></a>

## 5. Notes
Expand All @@ -246,8 +260,6 @@ Notes:
is available in
[BiasAdjustCXX](https://github.com/btschwertfeger/BiasAdjustCXX).

---

<a name="contribution"></a>

## 6. 🆕 Contributions
Expand Down Expand Up @@ -277,5 +289,3 @@ Notes:
https://github.com/btschwertfeger/python-cmethods/issues/47. I appreciate all
the valuable suggestions provided throughout the implementation of the
subsequent changes.

---
21 changes: 21 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Security Policy

## Reporting Security Vulnerabilities

At python-cmethods, we take security seriously. We welcome and encourage
responsible disclosure of security vulnerabilities. If you believe you've found
a security vulnerability within our project, please report it immediately.

## How to Report a Security Vulnerability

To report a security vulnerability, please send an email to
[[email protected]](mailto:[email protected]) with a detailed
description of the vulnerability. We kindly request that you refrain from
disclosing the vulnerability publicly until we have had an opportunity to
address it.

## Our Commitment

We are committed to promptly addressing and resolving any security
vulnerabilities reported to us. We will investigate all reports and take
appropriate action to protect the security of our users and their data.
4 changes: 2 additions & 2 deletions cmethods/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,14 +107,14 @@ def adjust(
data sets. For more information please refer to the method specific
requirements and execution examples.
See https://python-cmethods.readthedocs.io/en/latest/src/methods.html
See https://python-cmethods.readthedocs.io/en/latest/methods.html
The time dimension of ``obs``, ``simh`` and ``simp`` must be named ``time``.
If the sizes of time dimensions of the input data sets differ, you have to
pass the hidden ``input_core_dims`` parameter, see
https://python-cmethods.readthedocs.io/en/latest/src/getting_started.html#advanced-usage
https://python-cmethods.readthedocs.io/en/latest/getting_started.html#advanced-usage
for more information.
:param method: Technique to apply
Expand Down
Loading

0 comments on commit 84fb40d

Please sign in to comment.