Testing the demo notebooks using the docker image locally

[navidcy]

I don't know how can I test the demo notebooks using the docker image locally. That is, I'd like to be able to reproduce the tests that the GitHub Actions do locally before I push.

Ideally we should update the Contributors section in the documentation to include these instructions.

[ashjbarnes]

Yeah this would be good! I've learned recently how to set up local docker envs so can have a go at this.

[navidcy]

cc @angus-g

[ashjbarnes]

I had a go using act which is meant to emulate github actions but unfortunately it doesn't work that well out of the box. Formatting was failing because it couldn't install black. I think it's not worth spending the time to troubleshoot!

I can have a go directly with Docker, but in my experience this then requires some fiddling around to get it to work on different machines so could add a bit of a burden of documentation and maintenance

[navidcy]

I think I'd like to resolve this before we submit to JOSS (#100)

[angus-g]

Yep, I'll take a look at this. I have a few ideas, I'll figure out how feasible and easy to implement they are.

[angus-g]

Okay, I've managed to get act working, which I think is probably the most consistent way of doing things. Here's a summary of what I did, maybe you can see if you can replicate, and then we can write that up.

1. Install Docker, then install act. For my setup I did gh extension install https://github.com/nektos/gh-act
2. Create event.json containing the following:

{
  "act": true
}

3. Use the following modification to testing.yml (which we can add to the repository version if this seems like the way to go). These changes only alter the workflow when we're running within act. Essentially it skips the formatting step (just run black locally...), and because our testing image doesn't include Node, it needs to install that so that the step for installing Conda works. Again, if this is the way we end up going, we should just include Node in the image itself.

diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
index f406515..d1da3c1 100644
--- a/.github/workflows/testing.yml
+++ b/.github/workflows/testing.yml
@@ -11,6 +11,7 @@ jobs:
     steps:
       - uses: actions/checkout@v3
       - uses: psf/black@stable
+        if: ${{ !github.event.act }}
         with:
           src: "./regional_mom6 ./tests"
 
@@ -27,6 +28,13 @@ jobs:
 
     steps:
       - uses: actions/checkout@v3
+      - name: Install Node
+        if: ${{ github.event.act }}
+        run: |
+          apt update
+          apt install -y wget
+          wget https://nodejs.org/dist/v18.20.1/node-v18.20.1-linux-x64.tar.xz
+          tar -xvf node-v18.20.1-linux-x64.tar.xz --strip-components=1 --directory=/usr
       - name: Set up Python ${{ matrix.python-version }}
         uses: conda-incubator/setup-miniconda@v2
         with:

4. Run the actual tests, within your checkout of the repository: gh act -j testing -e event.json (or just act -j testing -e event.json if you didn't install it as a gh extension). Eventually you should get a little "🏁 Job succeeded"! I think that if the tests fail, the container won't be deleted, so you can re-attach it, modify the files and re-run the failing pytest command. This is probably the part that needs a little more investigation: I think that the container has a bind-mount to the repository on disk, so any changes you make there are reflected in the container.

---

As I said above, I think this is the most consistent way of running the tests in that it pretty closely mimics what the actual GitHub runner is doing, although it's not exact because of the way act works. The downside is that the setup of the environment for each run is pretty expensive: it has to download and install Conda and the prerequisites every time. Indeed, when the tests run successfully, the container is deleted.

I think an alternative that's less consistent with what we're doing in GitHub Actions, but with a tighter development loop is to perhaps provide a Dockerfile that essentially does the environment setup (install Conda and prerequisites on top of the development image). That way, you can start fresh from that image, bind in the local repository, and run the tests, without having to pay the cost of the full setup every time.

[navidcy]

Install Docker, then install act. For my setup I did gh extension install https://github.com/nektos/gh-act

Does "install Docker" mean this? --> https://docs.docker.com/desktop/install/mac-install/

[navidcy]

these

+          wget https://nodejs.org/dist/v18.20.1/node-v18.20.1-linux-x64.tar.xz
+          tar -xvf node-v18.20.1-linux-x64.tar.xz --strip-components=1 --directory=/usr

read like linux-specific?

[angus-g]

Does "install Docker" mean this? --> https://docs.docker.com/desktop/install/mac-install/

If you're on a Mac, I guess so.

read like linux-specific?

Yes, they're running in the testing environment which is Linux (and running inside Docker).

[navidcy]

OK, so will that work on a Mac as well?

[angus-g]

It should do.

[navidcy]

Hm....

$ regional-mom6/ (main✗) $ act -j testing -e event.json                                                                                                                  [11:29:58]
WARN[0000] Couldn't get a valid docker connection: no DOCKER_HOST and an invalid container socket ''
WARN  ⚠ You are using Apple M-series chip and you have not specified container architecture, you might encounter issues while running act. If so, try running it with '--container-architecture linux/amd64'. ⚠
Error: open /Users/navid/Research/regional-mom6/event.json: no such file or directory

when I try to follow the suggestion I get:

$ :regional-mom6/ (main✗) $ act --container-architecture linux/amd64 -j testing -e event.json                                                                             [11:30:29]
WARN[0000] Couldn't get a valid docker connection: no DOCKER_HOST and an invalid container socket ''
[Testing/formatting] 🚀  Start image=catthehacker/ubuntu:act-latest
[Testing/formatting]   🐳  docker pull image=catthehacker/ubuntu:act-latest platform=linux/amd64 username= forcePull=true
Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?