Allow authors to link customer JavaScript scripts to their pages by adding HTML manually or via configuration

[choldgraf]

User story

As an author, I want to be able to include custom JavaScript libraries and scripts on any page in my MyST site, so that I can enable extra functionality for readers that isn't packaged with mystmd.

Examples of where this is useful

• Loading custom JS scripts to modify the DOM for readers
• Loading external visualization libraries that don't bundle the necessary JS with Jupyter execution.
• Making tables interactive with ITables
• Loading a social media preview generator
  Some authors may wish to load custom JavaScript with their sites. For example if there is a non-standard visualization library that can only work if you load JS manually on the page.

References

• In Sphinx there are two ways you can do this:
  1. Attach static JS (or CSS) files vi config with the html_js_files configuration option in conf.py.
  2. Embed raw HTML in your content and link the JS there, via .. raw:: html blocks (or if via MyST markdown, by just putting the HTML directly in the content

[nthiery]

And in JupyterBook: https://jupyterbook.org/en/stable/advanced/html.html#custom-css-or-javascript

[rowanc1]

This can likely be done with a raw:html feature in the very near term (which does not yet have a renderer).

[agoose77]

@rowanc1 my gut feeling is that we want to discourage people from embedding web markup / scripting inside singular pages. I wonder whether our UX can be built around global files / styles that are gated with the proper tags. We can discuss when we refine this work for beta.

[cboettig]

not sure if it counts as a special case of this, but we are doing a lot with python libraries that display nicely in the interactive notebooks via ipywidgets (leafmap, altair plots, etc), but then don't render at all in myst. It would be great to have a way to support these. Students always find this situation particularly confusing / frustrating because the are used to seeing on the web side the same thing they see in their jupyterlab.

[rowanc1]

Thanks @cboettig and @nthiery for your help here. Do you have examples to point to online/repo which require these scripts to be loaded? And/or use cases (beyond the leafmap/altair etc. that you mentioned) that can help inform our thinking around this?

[cboettig]

@rowanc1 e.g. https://espm-157.carlboettiger.info/spatial-2, rendered from https://github.com/espm-157/website/blob/main/reading/spatial-2.ipynb. note the desperate use of iframe :-)

[nthiery]

Here is my most immediate use case: for my online course notes, I wrote a tiny
JS+CSS library that lets the read show/hide solutions and notes for the instructors.
Here is an example with jupyter-book (sorry, in French); you can click on the solutions
to unfold them, or user the two sliders at the top of the page:

https://nicolas.thiery.name/Enseignement/Info111/Semaine2/TD.html

I am in the process of migrating to mystmd and would like to keep the same feature.

(for the curious: the solutions are marked in the notebook/md sources using nbgrader's
syntax; notes for instructors are written as admonitions with a special title).

That use case illustrates well a nice feature of jupyter-book / sphinx: that you can easily
customize the output to tackle special needs.

[choldgraf]

Two more use cases: both using an external library to enrich page content

A directive for social media previews

In my personal blog, I have a Sphinx extension called socialpost that lets you link to social media posts like this:

```{socialpost} https://x.com/ProjectJupyter/status/1850964152082448459
```

It generates a block of HTML that Twitter specifies for this kind of thing:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">
  <a href="https://twitter.com/choldgraf/status/1564614538309390345">Tweet from @choldgraf</a>
</blockquote>

And on page load, a JS library renders it into an interactive and nicely-formatted social preview

A library to enrich table output

2i2c maintains a list of running hubs for all the communities we're currently working with. This is stored as a "regular table" and I believe it is loaded with the CSV-table list.

On that page, we load the datatables library and then use the configuration below to load it and point it to our "normal" hubs table to turn it into a nice interactive one:

https://github.com/2i2c-org/infrastructure/blob/6c86b11ede01c25b33923a31bff3376ebdaaa0df/docs/reference/hubs.md?plain=1#L39-L63

[stevejpurves]

@cboettig I take it you mean that ipyleaflet, ipywidgets do not appear without a kernel connection from the myst site yes?
If so for those cases, getting #429 over the line should handle those.

For altair, we expect those plots to appear already without further work.

In other cases, where jupyter outputs like bokeh, etc... that require a lab extension, that would be better addressed by mystmd shipping the extension code and loading it (as opposed to end user intervention to get the scripts), we can look towards a jupyterlab-esque loading mechanism that can automatically get the correct version of extension JS code at site build time.

For some of the other cases mentioned, perhaps new directives should be considered?

[agoose77]

As @stevejpurves points out, there are some features for which we should really be incorporating features into our themes, e.g. widget/renderer support. For one-off widgets, one feature that we discussed at SciPy 2024 was the inclusion of anywidget as a mechanism for front-end interactivity. This would allow users to author trivial "widgets" that render DOM nodes, if they want custom extensions.

Like Jupyter Notebok 6 -> Jupyter Notebook 7 / Lab, a general theme for MyST vs JB (in my view) is to treat web exports more formally (by building a rich document model) so that all export types can leverage new features, and so that our documents aren't as fragile as e.g. Jupyter Book. For example, a core benefit of MyST document ASTs is that you can render them with new frontends after build, meaning that you can e.g. distribute content building but centralise the view layer. As such, the document-view separation is something we want to do as much work to preserve as possible.

For use-cases like @nthiery's solution show/hide, that should be ideally something that the theme takes charge of. Perhaps we can start to collect those kinds of usages together, and identify some useful theme components.

[cboettig]

Thanks @stevejpurves, sorry I was unclear. Yes, those maps are pure javascript (maplibre-js, not leaflet-js in this case); but if you were to run the same python code in a Jupyter notebook, I believe leafmap renders that map using maplibre's ipywidget? Apologies I'm not really up to speed on the internals, but I'll keep an eye on #429, that sounds fantastic, thanks!

[nthiery]

For use-cases like @nthiery's solution show/hide, that should be ideally something that the theme takes charge of.

Yes, it would be great indeed to have a theme for teaching purposes! Nonetheless, I believe it's nice to provide users with a quick js/css extension point to play around with ideas until they mature, like what JB provides.