Jupyterbook, Thebe, and Jupyterlab setup

[jmunroe]

Ensure that this content can be

1. Rendered statically (as JupyterBook site)
2. As a thebe backed myst site (there may be challenges around earthdata login to overcome here)
3. As content within a JupyterLab session.

[jnywong]

@agoose77 just signing off for today, but here are some running thoughts that I might find the answer to after more poking around tomorrow.

Regarding the above,

1. Rendered statically (as JupyterBook site)

I would ideally like to save notebook cell outputs, such as figures, to be rendered on the website (rather than relying on myst build --execute). However, the ScienceCore content is written in MyST markdown and not as a Jupyter Notebook, since we send the markdown files off for translation. I seem to recall a conversation before that I would need to Jupytext link the .md file with a .ipynb file to store the cell outputs, is that right?

If I simultaneously want to achieve

2. As a thebe backed myst site (there may be challenges around earthdata login to overcome here)

then would I need to use the :placeholder: key to label the cell to hide the static cell output from scenario 1. somehow? Or is the bigger question here that you either have a static website with the images already available as assets OR you have a thebe-backed site with no image cell outputs immediately served but are dynamically built through a kernel from BinderHub.

3. As content within a JupyterLab session.

jupyter-book/myst-theme#503 is looking great! 🙌 Bigger question from above applies here too.

[agoose77]

rather than relying on myst build --execute

Is that because the computation is costly? Or is there another reason to avoid compute? We can cache the outputs so that most builds aren't expensive.

Otherwise, yes -- something like jupytext to link or move between formats is probably a good idea.

Regarding James' (2), I think that the issue he anticipates is the lack of auth support in Thebe. You can/should still be able to use Thebe alongside pre-computed outputs.

Relatedly, we will need to communicate with our audience that Thebe's lack of edit support for code-cells is something that is tracked as a feature somewhere (todo).

[jnywong]

Thanks @agoose77 ! I'll explore the use of cached outputs and failing that I will link outputs with Jupytext.

I wanted to avoid myst build --execute because some of the computations take a while (e.g. processing satellite data with xarray) and I am not super confident a GitHub runner can handle it reliably.

You can/should still be able to use Thebe alongside pre-computed outputs.

The mechanics of this part I am not super clear.

FYI: I think the auth support is fine actually! You can use the earthaccess package and invoke earthaccess.login() to prompt the user to enter their credentials through Thebe

[agoose77]

Long-running compute is one of several totally valid reasons not to run on GHA. In the long run, I am keen to build out solutions for this (and we can at any point pull-apart binderbot for a WIP stopgap) in MyST, but it's a longer-term strat.

RE auth support -- my concern is about JupyterHubs that require logins to spin up a server e.g. for ephemeral hubs!

[yuvipanda]

I am keen to build out solutions for this (and we can at any point pull-apart binderbot for a WIP stopgap) in MyST, but it's a longer-term strat.

NASA-IMPACT/veda-jupyterhub#46 was an attempt to do this as part of the NASA VEDA work, but I don't think our processes were mature enough to handle that properly then. NASA-IMPACT/veda-jupyterhub#46 (comment) has a branch to do this on a binder.

(Don't let this block your other work though, just wanted to leave this here as an FYI)