Export as Static HTML

[rowanc1]

Which area is this feature request for?

Website User Interface, Command Line Tools

Describe the feature you'd like to request

Currently the website requires an application server (the theme) to render html, which is done on the fly from a json file + static resources.

The initial pages are server-side-rendered, and can be exported as a static website (e.g. hosted on github pages or sent in a zip file). Twitter thread of the general idea with pointers:
https://twitter.com/mjackson/status/1585795441907494912?t=7VDGKQXUn6nSIqWevT2ZpA&s=19

We may in the future want to look into other html templating solutions to create html assets from the rendered mdast.

Describe the solution you'd like

A static HTML build that I can click on a .html file and it works! :)

Places to get back to:

• Build static site executablebooks/meta#947
• https://twitter.com/neuralreckoning/status/1623067490463936530

[rowanc1]

I have made some progress on this, but it needs more work to get it over the line. I am going to use this issue to write down my findings so far.

The steps that I have taken are:

1. create a file that has a list of URLs to turn into static pages
2. run wget -r -nH -i static.txt -P static - creates a static folder.
3. start a web server in that folder (e.g. npx serve static works, but so would a python web server)

This does a pretty good job of pulling all of the assets, and I can confirm that even Thebe works on the site, so we shouldn't have problems there.

There are a few other problems though:

1. Links - we use fancy things for both navigation and internal links so that you don't have to reload the whole page. These need to be replaced with normal links.
2. All of the content needs to served from a single domain, currently there are two domains at play (app & content).
3. The index page is a redirect and that isn't possible to do client side
4. The theme is stored in a cookie so that there isn't any flicker on first refresh

I think that wget is actually overkill, and we should just use fetch in node script, and that script can live in the myst-cli.

I think that to get this done we:

• have a signal that we are creating a static build. This needs to be picked up in two places (1) in the document, and (2) in the loader (to switch things over to a single server)
• remove the _static nesting in the public folder on the site/public/_static.
  • ⚡️ Remove _static folder; keep site assets flat in public folder #371
• if there is a single project, we should be serving that from the index page, without a slug (this is probably the hardest thing)
  • 📄 Update article theme for single project, consume dataUrl myst-theme#68
  • 📄 Enable sites with single projects and no project slug #374
• have the themes alternatively persist the light/dark theme in local storage

After those things are done, we can myst build --html and it should spin up the server with a flag (process.env.TARGET = 'static' or something). This turns off fancy links and changes the loading pattern of the server to look to the local host. Then once that is loaded, we can fetch and write the HTML for the pages in the project list. We don't need to get all the assets, as those are already in the _builds/site/public folder, we also don't need the assets as those are in the site template build folder!

After we download and write the html, the cli copies over all of the files from both the site/public and the template/public folders (those are in static and build in the html site).

[rowanc1]

Ok, latest release has myst build --html which creates a static export of the site! You can also set the BASE_URL environment variable before that command to nest it at a different path (e.g. at the repo level on a github pages domain). Some of the nice pre-fetch actions don't work in static mode, but the API is still there (e.g. going to .json on a page), which means that cross-page imports for figures also work. I have tested it with @stevejpurves's thebe, and that works (at least the local and binder, jupyter-lite is still in the works).

This ticks off a major feature, and should allow myst sites to be used a bit more widely than just with a curvenote deploy.

I will leave this issue open to document the feature as well as think a bit more about an easy github action (I think we can also do PR preview deployments easily with this...!).

• documentation on deployment to github
• deploying to curvenote
• github action
• sitemap URLs need the correct hostname in static build

[rowanc1]

I have put the documentation (#383) so far here:
https://myst-tools.org/docs/mystjs/deployment

Feedback welcome - and other deployment targets! :)

[brian-rose]

I'm currently struggling with this. Followed the instructions at https://mystmd.org/guide/deployment and ran

myst build --html

but the resulting index.html renders in my browser as plain HTML with all the style apparently stripped away.

Here's a simple page using article-theme rendered live by myst:

And the same page rendered as static HTML:

I haven't found any documentation about this. Am I missing something obvious?

And thanks for this amazing writing tool, I am very excited about the possibilities.

[riziles]

@brian-rose , how are you opening the files?

[stevejpurves]

Hi @brian-rose -- as I think @riziles is pointing at, if you double click on the index.html you don't get the styles or js loaded correctly (if you look in the javascript console / dev tools you'll likely see messages on failed loading).

Instead, if you run a local web server in and access the static content that way it should work. e.g.

cd _build/html
npx http-server .

then open the url is gives you / is serving on.

[brian-rose]

I see, thank you! I guess I didn't pay enough attention to the docs. I was just opening index.html in a browser and assuming the styles would load.

Working now. Many thanks for the quick response.

[riziles]

@brian-rose , for what it's worth, there are tools out there that will do this for you, for example: https://github.com/richardtallent/vite-plugin-singlefile , but serving the files is definitely the better way to go. I just usually use VS Code's "Live Server" plugin. It's got 43 million downloads, so apparently I'm not alone.

[pac84]

Hi, I have the same problem as @brian-rose. But I want to upload the build html to my own web server. But when I upload the html folder and open the file on the server on my browser, I habe the same output like @brian-rose. What should I upload to get the same output like running it on a npx-server?

Kind regards
pac84

[riziles]

@pac84 , could be a path issue. Is index.html in your root folder or a sub-path?

[rowanc1]

Hi @pac84 - the BASE_URL environment must match the path that you are exporting and serving the HTML from. There is a bit of documentation here in some of the admonitions:

https://mystmd.org/guide/deployment

If you set that environment variable and then re-run myst build --html that should give you a folder that you can host at that specific path. The baseurl is needed to know where to lookup the static assets from, if that isn't set properly, then the css/js won't be found!

[pac84]

@pac84 , could be a path issue. Is index.html in your root folder or a sub-path?

Hi, thanks for your reply. It's in a subfolder.

[pac84]

Hi @pac84 - the BASE_URL environment must match the path that you are exporting and serving the HTML from. There is a bit of documentation here in some of the admonitions:

https://mystmd.org/guide/deployment

If you set that environment variable and then re-run myst build --html that should give you a folder that you can host at that specific path. The baseurl is needed to know where to lookup the static assets from, if that isn't set properly, then the css/js won't be found!

Hi, thanks for your help! It will be on a "normal" server in a subfolder, like

www.test.com/here/it/will/be

Does the Base URL have to be defined in the "myst.yaml"?

Kind regards
pac84

[LecrisUT]

I think this issue should be revisited. Can't we use relative paths (relative to index.md) to build the url links instead of defining BASE_URL?

[rowanc1]

This limitation is currently in Remix (we are building a web application, and then making a static html export of it). There is probably something in the configuration that we could do, but in my experimenting I have not found it yet.

Does the Base URL have to be defined in the "myst.yaml"?

@pac84 this is an environment variable in the build, you can run export BASE_URL="/here/it/will/be"; myst build --html

[LecrisUT]

This limitation is currently in Remix (we are building a web application, and then making a static html export of it).

Hmm, that happens after baseurl is parsed and used right? That's a though cookie, found that it is quite a limitation of the framework itself. That explains why the urls are not .html terminated as well.

The only thing I've found was remix-ssg which is abandoned and quite complex to replicate/update. Any post replacement would also be quite risky because it might alter the content itself. Silliest approach I could think of is to build twice with different baseurl and hopefully the diff have only the path variables.

Migration to next.js or something that has ssg would be difficult I suppose?

[pac84]

Hi, thanks for the two answers. My actual solution is to use Jupyter Book and export html. This is static code and could directly copied to a server.

Kind regards
pac84