Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples page #7

Merged
merged 113 commits into from
Jul 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
b1d05ba
Add openmm cookbook
Yoshanuikabundi Apr 20, 2023
6ee79d7
Examples progress
Yoshanuikabundi Apr 27, 2023
413c1a7
Remove sphinx_gallery
Yoshanuikabundi May 31, 2023
75f45dc
Add gitpython to lint environment
Yoshanuikabundi May 31, 2023
45e6bd3
Remove nbsphinx_prolog
Yoshanuikabundi May 31, 2023
d8f197e
Download zip files with notebooks and files, not just notebooks
Yoshanuikabundi May 31, 2023
cd59e3e
Pre-processing notebooks basically works
Yoshanuikabundi Jun 6, 2023
7479768
Execute notebooks in parallel and only update changed notebooks
Yoshanuikabundi Jun 6, 2023
08dfb53
Set INTERCHANGE_EXPERIMENTAL and simplify delay code
Yoshanuikabundi Jun 6, 2023
2bacdde
Notes to self
Yoshanuikabundi Jun 6, 2023
d773f31
Interim fixes for extension
Yoshanuikabundi Jun 6, 2023
7926fb4
Fix module stuff
Yoshanuikabundi Jun 7, 2023
96dbb8f
Clean up removed notebooks and download latest release examples
Yoshanuikabundi Jun 7, 2023
a214387
Get the sphinx extension building the processed stuff
Yoshanuikabundi Jun 8, 2023
2c92744
Proof of concept gallery
Yoshanuikabundi Jun 8, 2023
975fa72
Simplify code and fix warnings
Yoshanuikabundi Jun 8, 2023
ac0a263
Fix css in cookbook
Yoshanuikabundi Jun 8, 2023
79f2e8a
Finish transition to MyST-NB
Yoshanuikabundi Jun 8, 2023
97a2705
Temporary fix for NGLView widgets.
Yoshanuikabundi Jun 20, 2023
7215221
Process all notebooks in single folder
Yoshanuikabundi Jun 20, 2023
b7a46e1
Refactor towards caching notebooks on Git branch
Yoshanuikabundi Jun 20, 2023
7fbaac3
Avoid unnecessary API requests AKA Generators are cool & I am a nerd
Yoshanuikabundi Jun 20, 2023
fa6b6b9
Order Sphinx hooks chronologically
Yoshanuikabundi Jun 20, 2023
0c956e7
Application -> Sphinx
Yoshanuikabundi Jun 20, 2023
23912d3
Typos
Yoshanuikabundi Jun 20, 2023
7071026
Merge branch 'cookbook_gha' into examples
Yoshanuikabundi Jun 21, 2023
3a644c8
Merge branch 'main' into examples
Yoshanuikabundi Jun 21, 2023
f8d3a90
Fix colab link generation
Yoshanuikabundi Jun 21, 2023
1b50d0f
Download cache if pre-processed notebooks are missing
Yoshanuikabundi Jun 21, 2023
69f9d21
Print out git version
Yoshanuikabundi Jun 21, 2023
a851793
Troubleshooting improvements for broken examples
Yoshanuikabundi Jun 21, 2023
eea7e04
Remove unnecessary + redundant remove_notebooks function
Yoshanuikabundi Jun 21, 2023
9d15655
Ping github in sphinx extension to test RTD internet access
Yoshanuikabundi Jun 21, 2023
512e8bd
Try multiping
Yoshanuikabundi Jun 21, 2023
4ed8dea
Remove ping test
Yoshanuikabundi Jun 21, 2023
3492931
Try cloning in RTD hook
Yoshanuikabundi Jun 21, 2023
8975f9c
Simplify git command
Yoshanuikabundi Jun 21, 2023
86a5b81
mv -> rsync
Yoshanuikabundi Jun 21, 2023
c09b28a
Install rsync + git
Yoshanuikabundi Jun 21, 2023
870fb42
Remove ls command
Yoshanuikabundi Jun 21, 2023
1746529
Don't print git version
Yoshanuikabundi Jun 21, 2023
5fde2ab
Fix include_css_files()
Yoshanuikabundi Jun 21, 2023
d3a993c
Remove unused imports
Yoshanuikabundi Jun 21, 2023
625736d
Move notebook grid CSS out of extension
Yoshanuikabundi Jun 21, 2023
5c8be5a
Update lint environment
Yoshanuikabundi Jun 21, 2023
0b8f567
Merge branch 'main' into examples
Yoshanuikabundi Jun 21, 2023
26f49bf
Exclude examples from link checker
Yoshanuikabundi Jun 21, 2023
fe95db1
Remove annoying print statement
Yoshanuikabundi Jun 21, 2023
7c3604b
Debug link check ignore conf
Yoshanuikabundi Jun 21, 2023
23cb681
Move linkcheck exclusions to sphinx extension
Yoshanuikabundi Jun 21, 2023
d3bdbce
Fix linkcheck exclusions
Yoshanuikabundi Jun 21, 2023
ff4b45b
Tell MyST-NB about codelinter
Yoshanuikabundi Jun 21, 2023
bf28b5c
Merge branch 'main' into examples
Yoshanuikabundi Jun 26, 2023
fd77c1e
Parse tag out of repo list
Yoshanuikabundi Jun 26, 2023
25c9e3e
Copy thumbnails correctly
Yoshanuikabundi Jun 27, 2023
d3e1308
Get thumbnails into gallery
Yoshanuikabundi Jun 27, 2023
6adbb09
Break up _cookbook.py
Yoshanuikabundi Jun 27, 2023
ba7ac0a
Formatting
Yoshanuikabundi Jun 27, 2023
a26c0a0
Pin nglview>=3.0.6 and remove JS included therein
Yoshanuikabundi Jun 27, 2023
42ba5b3
Initial category support
Yoshanuikabundi Jun 27, 2023
618337f
Rejib cache paths
Yoshanuikabundi Jun 27, 2023
ebdadbe
Add fragmenter and reminder for QCSubmit
Yoshanuikabundi Jun 27, 2023
5a9423d
Disable fragmenter for now
Yoshanuikabundi Jun 27, 2023
5c3bf09
Move zipped notebooks to source directory
Yoshanuikabundi Jun 28, 2023
1a822b1
Support categories in cookbook
Yoshanuikabundi Jun 28, 2023
9f17df0
Tweak categories and switch to new interchange branch
Yoshanuikabundi Jun 28, 2023
9576cd6
Print env in rtd and update TK/Interchange
Yoshanuikabundi Jul 3, 2023
4d30f76
Try and enable custom widgets in colab
Yoshanuikabundi Jul 3, 2023
189bbd9
Support branch/pr caches and update main every night
Yoshanuikabundi Jul 3, 2023
d76f3aa
Fix inputs
Yoshanuikabundi Jul 3, 2023
4087b98
Ergonomics
Yoshanuikabundi Jul 3, 2023
0b2b619
Ensure deploy subdirectory exists
Yoshanuikabundi Jul 3, 2023
532e455
Fix ergonomics
Yoshanuikabundi Jul 3, 2023
1b4b9bb
Add --ignore-unmatch
Yoshanuikabundi Jul 3, 2023
6969dff
Use multiline string in .readthedocs.yaml
Yoshanuikabundi Jul 3, 2023
1de772a
Use old-style test in readthedocs.yaml
Yoshanuikabundi Jul 3, 2023
d86b2d6
String equality in .readthedocs.yaml
Yoshanuikabundi Jul 3, 2023
6336d79
Debug rtd with ls
Yoshanuikabundi Jul 3, 2023
0823436
Try ls --tree
Yoshanuikabundi Jul 3, 2023
0c423ea
Work around absence of --tree
Yoshanuikabundi Jul 3, 2023
c1b4b81
Fix cache download path
Yoshanuikabundi Jul 3, 2023
222a143
Debug rtd
Yoshanuikabundi Jul 3, 2023
553b56b
Try apt-get tree
Yoshanuikabundi Jul 3, 2023
30ff339
Fix source path?
Yoshanuikabundi Jul 3, 2023
f610d71
Move glob outside of quotes
Yoshanuikabundi Jul 3, 2023
b1cdd08
Simplify .readthedocs.yaml
Yoshanuikabundi Jul 3, 2023
620964c
Include branch in colab link
Yoshanuikabundi Jul 3, 2023
03d1d40
Update colab links in Sphinx extension
Yoshanuikabundi Jul 3, 2023
d565d10
Consolidate code between .readthedocs.yaml and python
Yoshanuikabundi Jul 3, 2023
90231a6
Fix argument handling in proc_examples.py
Yoshanuikabundi Jul 3, 2023
5b13759
Fix import
Yoshanuikabundi Jul 3, 2023
c17a1c7
Swap quotes
Yoshanuikabundi Jul 3, 2023
b155ab9
Try backtick shell expansion
Yoshanuikabundi Jul 3, 2023
56fad31
Use shell logic in .readthedocs.yaml
Yoshanuikabundi Jul 3, 2023
139b9a4
Clean up PR cache when closed
Yoshanuikabundi Jul 4, 2023
7d43834
Trigger RTD build when cache is updated (except in PRs)
Yoshanuikabundi Jul 4, 2023
0c28089
Merge branch 'main' into examples
Yoshanuikabundi Jul 4, 2023
f60ed11
Clarify description
Yoshanuikabundi Jul 4, 2023
7143205
Single quotes
Yoshanuikabundi Jul 4, 2023
fa10eee
Show source repository in gallery
Yoshanuikabundi Jul 4, 2023
b45fdb2
Style top-of-notebook links
Yoshanuikabundi Jul 4, 2023
90effe8
Add experimental warnings and improve styling
Yoshanuikabundi Jul 4, 2023
3ed8f85
Newline in css
Yoshanuikabundi Jul 4, 2023
33b77bb
globals -> globals_
Yoshanuikabundi Jul 4, 2023
8ed4661
Switch run_notebook.sh to use jupyterlab
Yoshanuikabundi Jul 4, 2023
0689914
Add installation instructions
Yoshanuikabundi Jul 4, 2023
a2b33a9
zip -> tgz
Yoshanuikabundi Jul 4, 2023
3c53bfe
Add CLI example for run_notebook.sh
Yoshanuikabundi Jul 4, 2023
52d05dc
Simplify
Yoshanuikabundi Jul 4, 2023
457ae4b
Fix old imports
Yoshanuikabundi Jul 4, 2023
0350a44
More robust run_notebook.sh
Yoshanuikabundi Jul 4, 2023
dfcc8de
Migrate to setup-micromamba
Yoshanuikabundi Jul 4, 2023
708dc2e
ZIP -> .TGZ in install instructions
Yoshanuikabundi Jul 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/cookbook_pr_closed.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Clean up PR cache

on:
pull_request:
types:
- closed
branches:
- main

jobs:
preprocess:
runs-on: ubuntu-latest

steps:
- name: Checkout deployment branch
uses: actions/checkout@v3
with:
ref: _cookbook_data
clean: false

- name: Clean up PR cache
shell: bash -l {0}
run: |
rm -r PR${{ github.event.number }}
git config user.name github-actions
git config user.email [email protected]
git add .
git commit --amend -m "Deploy pre-processed notebook cache"
git push --force origin _cookbook_data
55 changes: 55 additions & 0 deletions .github/workflows/cookbook_pr_opened.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: Generate PR cache by copying main

on:
pull_request:
types:
- opened
- reopened
branches:
- main

jobs:
preprocess:
runs-on: ubuntu-latest

steps:
- name: Checkout deployment branch
uses: actions/checkout@v3
with:
ref: _cookbook_data
clean: false

- name: Copy cache from main to PR folder and deploy
shell: bash -l {0}
run: |
cp -r main PR${{ github.event.number }}
git config user.name github-actions
git config user.email [email protected]
git add .
git commit --amend -m "Deploy pre-processed notebook cache"
git push --force origin _cookbook_data

- name: Add descriptive comment to PR
uses: thollander/actions-comment-pull-request@v2
with:
message: >
This repository caches data generated from notebooks stored in other
repositories in the `_cookbook_data` branch. Regenerating this cache
takes about half an hour, so it is not done automatically in PRs.
The current value of the cache from the `main` branch has been
copied over to this PR's cache.


To regenerate the cache, navigate to the [`cookbook_preproc`
action], click "Run workflow", select the current branch and enter
the PR number (`${{ github.event.number }}`), and then click the
green "Run workflow" button.


Note that the RTD build must be executed after the
`cookbook_preproc` action has finished for its results to be
visible, which can be done from the appropriate [build page].


[`cookbook_preproc` action]: https://github.com/openforcefield/openff-docs/actions/workflows/cookbook_preproc.yaml
[build page]: https://readthedocs.org/projects/openff-docs/builds/
41 changes: 33 additions & 8 deletions .github/workflows/cookbook_preproc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,14 @@ name: Pre-process example notebooks

on:
workflow_dispatch:
# TODO: Run when source repo releases are available on Conda Forge
# TODO: Run when the examples Conda environment is changed
inputs:
pr_number:
description: 'Update the cache for PR#: (leave blank if not a PR)'
required: false
default: ""
type: string
schedule:
- cron: 0 0 * * * # 1/day at midnight UTC

jobs:
preprocess:
Expand All @@ -24,23 +30,42 @@ jobs:
path: 'deploy'
clean: false

- name: Choose deployment subdirectory
shell: bash -l {0}
run: |
if [[ ${{ github.event_name }} != "workflow_dispatch" ]]; then
echo "DEPLOY_SUBDIR=$GITHUB_REF_NAME" >> "$GITHUB_ENV"
elif [[ "${{ inputs.pr_number }}" == "" ]]; then
echo "DEPLOY_SUBDIR=$GITHUB_REF_NAME" >> "$GITHUB_ENV"
else
echo "DEPLOY_SUBDIR=PR${{ inputs.pr_number }}" >> "$GITHUB_ENV"
fi

- name: Prepare cache deployment directory
shell: bash -l {0}
run: |
cd deploy
mkdir -p deploy/$DEPLOY_SUBDIR
cd deploy/$DEPLOY_SUBDIR
git config user.name github-actions
git config user.email [email protected]
git rm -r .
git rm -r --ignore-unmatch .

- name: Pre-process and execute notebooks
shell: bash -l {0}
run: |
python source/_ext/proc_examples.py --prefix=deploy
python source/_ext/proc_examples.py --prefix=deploy/$DEPLOY_SUBDIR --cache-prefix=$DEPLOY_SUBDIR

- name: Deploy cache
shell: bash -l {0}
run: |
cd deploy
cd deploy/$DEPLOY_SUBDIR
git add .
git commit -m "Deploy pre-processed notebook cache"
git push origin _cookbook_data
git commit --amend -m "Deploy pre-processed notebook cache"
git push --force origin _cookbook_data

- name: Trigger RTD build
# RTD doesn't offer a way to manually trigger builds in PRs, so we'll just do this for other builds
if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.pr_number == '')
shell: bash -l {0}
run: |
curl -X POST -d "branches=$GITHUB_REF_NAME" -d "token=${{ secrets.RTD_WEBHOOK_TOKEN }}" https://readthedocs.org/api/v2/webhook/openff-docs/243876/
2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
- uses: actions/checkout@v3

- name: Install Conda environment
uses: mamba-org/provision-with-micromamba@main
uses: mamba-org/setup-micromamba@v1
with:
environment-file: devtools/conda-envs/lint_env.yml

Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
build/

# Cookbook files
source/examples/
source/_cookbook/

# Python stuff
Expand Down
14 changes: 14 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,19 @@ build:
os: ubuntu-20.04
tools:
python: "mambaforge-4.10"
apt_packages:
- rsync
jobs:
post_install:
- git clone -v --depth=1 --single-branch --branch=_cookbook_data -- https://github.com/openforcefield/openff-docs.git build/_cookbook_data
- |
if [ "$READTHEDOCS_VERSION_TYPE" = "external" ]; then
# This is a PR, so use the PR number
SOURCE_PATH="build/_cookbook_data/PR${READTHEDOCS_VERSION_NAME}"
else
# This is not a PR, so use the Git reference
SOURCE_PATH="build/_cookbook_data/${READTHEDOCS_GIT_IDENTIFIER}"
fi
rsync -a "${SOURCE_PATH}"/* ./
conda:
environment: devtools/conda-envs/rtd_env.yml
9 changes: 7 additions & 2 deletions devtools/conda-envs/examples_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,24 @@ channels:
dependencies:
- pip
- python=3.10
- jupyterlab>=4
- ipywidgets>=8
# Cookbook
- gitpython
- nbconvert
- nbformat
- requests
- packaging
# Examples
- openff-toolkit-examples
- openff-interchange==0.3.5 # Pin Interchange 0.3.5 because 0.3.6 has a broken example
- openff-toolkit-examples>=0.14.0
- openff-interchange>=0.3.7
- openff-nagl==0.2.2
# - openff-fragmenter
# - openff-qcsubmit
- gromacs >=2021=nompi*
- lammps
- rich
- jax
- dglteam/label/cu118::dgl
- parmed<4
- nglview>=3.0.6
8 changes: 8 additions & 0 deletions devtools/conda-envs/lint_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,13 @@ dependencies:
# readthedocs dependencies
- sphinx>=5,<7
- myst-parser>=1,<2
# - myst-nb
- sphinx-notfound-page
- ipython >=8.8
- sphinx-design
- gitpython
- nbconvert
- nbformat
# Code example deps
- openff-toolkit-base
- openff-interchange-base
Expand All @@ -18,3 +24,5 @@ dependencies:
- git+https://github.com/openforcefield/openff-sphinx-theme.git@main
# Lints
- sphinxawesome-codelinter
# Sphinx
- git+https://github.com/Yoshanuikabundi/MyST-NB.git@upgrade-to-1
9 changes: 9 additions & 0 deletions devtools/conda-envs/rtd_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,19 @@ dependencies:
# readthedocs dependencies
- sphinx>=5,<7
- myst-parser>=1,<2
# - myst-nb
- sphinx-notfound-page
- ipython >=8.8
- sphinx-design
# Examples
- gitpython
- nbconvert
- nbformat
# Theme
- pip:
# Theme
- git+https://github.com/openforcefield/openff-sphinx-theme.git@main
# Lints
- sphinxawesome-codelinter
# Sphinx
- git+https://github.com/Yoshanuikabundi/MyST-NB.git@upgrade-to-1
Loading