docs: Document runtimes more comprehensively and centrally

Trying to provide a more useful resource to point people to than a
smattering of scattered doc snippets.

There's still some places this could be better cross-referenced (e.g. in
--help output) and more that could be said about runtimes in general and
each runtime specifically… but that can all come in due time.  I needed
to find a stopping point for this work _somewhere_.   A big gain here is
that now there's clear places to attach additional documentation in the
future.

Like the recent switch from dynamically- to statically-generated rST for
command documentation, it may also sooner-than-later make sense to ditch
the "automodule" directive and do something static instead.

Resolves <#288>.

README.md

@@ -18,10 +18,10 @@ email to <[email protected]>.
 
 [Augur]: https://docs.nextstrain.org/projects/augur/
 [Auspice]: https://docs.nextstrain.org/projects/auspice/
-[Docker]: https://docs.nextstrain.org/projects/cli/en/stable/installation/#docker
-[Conda]: https://docs.nextstrain.org/projects/cli/en/stable/installation/#conda
-[Singularity]: https://docs.nextstrain.org/projects/cli/en/stable/installation/#singularity
-[AWS Batch]: https://docs.nextstrain.org/projects/cli/en/stable/installation/#aws-batch
+[Docker]: https://docs.nextstrain.org/projects/cli/page/runtimes/docker/
+[Conda]: https://docs.nextstrain.org/projects/cli/page/runtimes/conda/
+[Singularity]: https://docs.nextstrain.org/projects/cli/page/runtimes/singularity/
+[AWS Batch]: https://docs.nextstrain.org/projects/cli/page/runtimes/aws-batch/
 [documentation]: https://docs.nextstrain.org/projects/cli/
 [development docs]: https://docs.nextstrain.org/projects/cli/page/development/
 [open an issue]: https://github.com/nextstrain/cli/issues/new

doc/commands/build.rst

@@ -120,7 +120,7 @@ default is not suitable.
 
 .. option:: --ambient
 
-    Run commands in the ambient environment, outside of any container image.
+    Run commands in the ambient environment, outside of any container image or managed environment.
 
 .. option:: --aws-batch
 

doc/commands/view.rst

@@ -115,7 +115,7 @@ default is not suitable.
 
 .. option:: --ambient
 
-    Run commands in the ambient environment, outside of any container image.
+    Run commands in the ambient environment, outside of any container image or managed environment.
 
 .. option:: --conda
 

doc/glossary.rst

@@ -36,17 +36,4 @@ documentation.
         environment) in which Nextstrain CLI expects to find and execute other
         Nextstrain programs.
 
-        Current runtimes:
-
-          - Docker with the `nextstrain/base image <https://hub.docker.com/r/nextstrain/base>`_
-          - Conda with the `nextstrain-base meta-package <https://anaconda.org/nextstrain/nextstrain-base>`__
-          - Ambient
-          - AWS Batch with the `nextstrain/base image`_
-
-        Each runtime provides specific versions of Nextstrain's software
-        components, like Augur and Auspice.
-
-        Runtimes are managed (maintained, tested, versioned, released) by the
-        Nextstrain team, except for the ambient runtime.  The ambient runtime
-        is special in that it's whatever computing environment in which
-        Nextstrain CLI itself is running (i.e. managed by the user).
+        See the :doc:`runtimes overview </runtimes/index>`.

doc/index.rst

@@ -11,10 +11,10 @@ The Nextstrain CLI is a program called ``nextstrain``.  It aims to provide a
 consistent way to run and visualize pathogen builds and access Nextstrain
 components like :doc:`Augur <augur:index>` and :doc:`Auspice <auspice:index>`
 computing platforms, such as
-:ref:`installation/docker`,
-:ref:`installation/conda`,
-:ref:`installation/singularity`, and
-:ref:`installation/aws-batch`.
+:doc:`/runtimes/docker`,
+:doc:`/runtimes/conda`,
+:doc:`/runtimes/singularity`, and
+:doc:`/runtimes/aws-batch`.
 
 .. note::
     If you're unfamiliar with Nextstrain builds, you may want to follow our
@@ -64,6 +64,17 @@ Table of Contents
     nextstrain.org <remotes/nextstrain.org>
     Amazon S3 <remotes/s3>
 
+.. toctree::
+    :caption: Runtimes
+    :titlesonly:
+
+    Overview <runtimes/index>
+    Docker <runtimes/docker>
+    Conda <runtimes/conda>
+    Singularity <runtimes/singularity>
+    AWS Batch <runtimes/aws-batch>
+    Ambient <runtimes/ambient>
+
 .. toctree::
     :caption: Development
     :titlesonly:

doc/installation.rst

@@ -106,156 +106,29 @@ The version you get will probably be different than the one shown in the
 example above.
 
 
-Runtimes
-========
+A Nextstrain runtime
+====================
 
-.. XXX TODO: Move this heading and subheadings (with modification) to their own
-   top-level doc section (e.g. like Remotes).
-     -trs, 12 Jan 2023
+If you intend to run commands like :doc:`/commands/build` and
+:doc:`/commands/view`, then you'll need to set up at least one :term:`runtime`.
+See the :doc:`runtimes overview </runtimes/index>` for a comparison of the
+options and brief set up instructions for each.  Runtime set up typically
+concludes by running:
 
-The Nextstrain CLI provides a consistent interface and computing environment
-for running and visualizing Nextstrain pathogen builds across several different
-computing platforms, such as
-:ref:`installation/docker`,
-:ref:`installation/conda`,
-:ref:`installation/singularity`, and
-:ref:`installation/aws-batch`.
-
-We call the provided computing environments the :term:`Nextstrain runtimes
-<docs:runtime>`.  Each runtime provides specific versions of Nextstrain's
-software components, like `Augur <https://github.com/nextstrain/augur>`__ and
-`Auspice <https://github.com/nextstrain/auspice>`__.
-
-At least one of these runtimes must be setup in order for many of
-``nextstrain``'s subcommands to work, such as ``nextstrain build`` and
-``nextstrain view``.
-
-The default runtime is Docker, using the `nextstrain/base`_ container image.
-Containers provide a tremendous amount of benefit for scientific workflows by
-isolating dependencies and increasing reproducibility. However, they're not
-always appropriate, so a Conda runtime, Singularity runtime, and "ambient"
-runtime are also supported.  The installation and setup of supported runtimes
-is described below.
-
-.. _nextstrain/base: https://github.com/nextstrain/docker-base
-
-.. _installation/docker:
-
-Docker
-------
-
-`Docker <https://docker.com>`__ is a very popular container system
-freely-available for all platforms. When you use Docker with the Nextstrain
-CLI, you don't need to install any other Nextstrain software dependencies as
-validated versions are already bundled into a container image by the Nextstrain
-team.
-
-On macOS, download and install `Docker Desktop`_, also known previously as
-"Docker for Mac".
-
-On Linux, install Docker with the standard package manager. For example, on
-Ubuntu, you can install Docker with ``sudo apt install docker.io``.
-
-On Windows, install `Docker Desktop`_ with its support for a WSL2 backend.
-
-Once you've installed Docker, proceed with :ref:`checking your setup
-<installation/check-setup>`.
-
-.. _Docker Desktop: https://www.docker.com/products/docker-desktop
-
-.. _installation/conda:
-
-Conda
------
-
-`Conda <https://docs.conda.io/en/latest/miniconda.html>`__ is a very popular
-packaging system freely-available for all platforms. When you use Nextstrain
-CLI's built-in Conda support, you don't need to install any other Nextstrain
-software dependencies yourself as they're automatically managed in an isolated
-location (isolated even from other Conda environments you may manage yourself).
-
-On macOS and Linux, run ``nextstrain setup conda`` to get started.
-
-This runtime is not directly supported on Windows, but you can use `WSL2
-<https://docs.microsoft.com/en-us/windows/wsl/wsl2-index>`__ to "switch" to
-Linux and run the above setup command.
-
-.. _installation/singularity:
-
-Singularity
------------
-
-Singularity is a container system freely-available for Linux platforms.  It is
-commonly available on institutional HPC systems as an alternative to Docker,
-which is often not supported on such systems.  When you use Singularity with
-the Nextstrain CLI, you don't need to install any other Nextstrain software
-dependencies as validated versions are already bundled into a container image
-by the Nextstrain team.
-
-Run ``nextstrain setup singularity`` to get started.
-Singularity version 3.0.0 or newer is required, but we recommend at least
-version 3.10.0 or newer when possible.
-
-Note that the Singularity project forked into two separate projects in late
-2021: `SingularityCE`_ under `Sylabs`_ and `Apptainer`_ under the `Linux
-Foundation`_.  Either fork should work with Nextstrain CLI, as both projects
-still provide very similar interfaces and functionality via the ``singularity``
-command.  You can read `Sylab's announcement`_ and `Apptainer's announcement`_
-for more information on the fork.
-
-.. _SingularityCE: https://sylabs.io/singularity/
-.. _Sylabs: https://sylabs.io/
-.. _Apptainer: https://apptainer.org
-.. _Linux Foundation: https://www.linuxfoundation.org/
-.. _Sylab's announcement: https://sylabs.io/2022/06/singularityce-is-singularity/
-.. _Apptainer's announcement: https://apptainer.org/news/community-announcement-20211130
-
-Ambient
--------
-
-The "ambient" runtime allows you to use the Nextstrain CLI with your own ambient
-setup, for when you cannot or do not want to have Nextstrain CLI manage its own
-runtime.
-
-However, you will need to make sure all of the Nextstrain software dependencies
-are available locally or "ambiently" on your computer. A common way to do this
-is by manually using `Conda <https://docs.conda.io/en/latest/miniconda.html>`__
-to manage your own environment that includes the required software, however
-you're responsible for making sure the correct software is installed and kept
-up-to-date. It is also possible to install the required Nextstrain software
-`Augur <https://github.com/nextstrain/augur>`__ and `Auspice
-<https://github.com/nextstrain/auspice>`__ and their dependencies manually,
-although this is not recommended.
-
-Once you've installed dependencies, proceed with :ref:`checking your setup
-<installation/check-setup>`.
-
-.. _installation/aws-batch:
-
-AWS Batch
----------
-
-`AWS Batch <https://aws.amazon.com/batch/>`__ is an advanced computing
-platform which allows you to launch and monitor Nextstrain builds in the
-cloud from the comfort of your own computer. The same image used by the local
-Docker runtime is used by AWS Batch, making your builds more reproducible, and
-builds have access to computers with very large CPU and memory allocations if
-necessary.
+.. code-block:: bash
 
-The initial setup is quite a bit more involved, but :doc:`detailed instructions
-<aws-batch>` are available.
+    nextstrain setup <runtime>
 
-Once you've setup AWS, proceed with :ref:`checking your setup
-<installation/check-setup>`.
 
 .. _installation/check-setup:
 
 Checking your setup
 ===================
 
-After installation and setup, run ``nextstrain check-setup --set-default`` to
-ensure everything works and automatically pick an appropriate default runtime
-based on what's available. You should see output similar to the following:
+After installation and runtime set up, run ``nextstrain check-setup
+--set-default`` to ensure everything works and automatically pick an
+appropriate default runtime based on what's available. You should see output
+similar to the following:
 
 .. code-block:: console
 

doc/runtimes/ambient.rst

@@ -0,0 +1,6 @@
+===============
+Ambient runtime
+===============
+
+.. automodule:: nextstrain.cli.runner.ambient
+    :noindex:

doc/runtimes/aws-batch.rst

@@ -0,0 +1,6 @@
+=================
+AWS Batch runtime
+=================
+
+.. automodule:: nextstrain.cli.runner.aws_batch
+    :noindex:

doc/runtimes/comparison.csv

@@ -0,0 +1,6 @@
+,Isolation level,Containerized?,Locality,External dependencies
+Docker,great (3),yes,local or remote,``docker`` command
+Conda,some (1),no,local,none
+Singularity,good (2),yes,local,``singularity`` command
+Ambient,none (0),no,local,many
+AWS Batch,great (3),yes,remote,AWS account with Batch set up

doc/runtimes/conda.rst

@@ -0,0 +1,6 @@
+=============
+Conda runtime
+=============
+
+.. automodule:: nextstrain.cli.runner.conda
+    :noindex:

doc/runtimes/docker.rst

@@ -0,0 +1,6 @@
+==============
+Docker runtime
+==============
+
+.. automodule:: nextstrain.cli.runner.docker
+    :noindex:

doc/runtimes/index.rst

@@ -0,0 +1,97 @@
+========
+Runtimes
+========
+
+Nextstrain's runtimes are specific :term:`computing environments <computing
+environment>` in which Nextstrain CLI expects to find and run other Nextstrain
+programs, like :doc:`Augur <augur:index>` and :doc:`Auspice <auspice:index>`.
+In turn, Nextstrain CLI provides a consistent set of commands to run and
+visualize Nextstrain pathogen builds regardless of the underlying runtime in
+use.  Together, this allows Nextstrain to be used across many different
+computing platforms and operating systems.
+
+The :doc:`/commands/build`, :doc:`/commands/view`, and :doc:`/commands/shell`
+commands all require a runtime, as they require access to other Nextstrain
+software.
+
+The :doc:`/commands/setup`, :doc:`/commands/check-setup`, and
+:doc:`/commands/update` commands manage the runtimes available for use by the
+commands above.  The :doc:`/commands/version` command's :option:`--verbose
+<nextstrain version --verbose>` option reports detailed version information
+about all available runtimes.
+
+Other Nextstrain CLI commands, such as the :doc:`/commands/remote/index` family
+of commands and the related :doc:`/commands/login` and :doc:`/commands/logout`
+commands, do not require a runtime.  They may be used without any prior set up
+of a runtime.
+
+The runtimes currently available are the:
+
+  - :doc:`/runtimes/docker`
+  - :doc:`/runtimes/singularity`
+  - :doc:`/runtimes/conda`
+  - :doc:`/runtimes/ambient`
+  - :doc:`/runtimes/aws-batch`
+
+Runtimes are managed (maintained, tested, versioned, released) by the
+Nextstrain team, except for the ambient runtime.  The ambient runtime is
+special in that it's whatever computing environment in which Nextstrain CLI
+itself is running (that is, it's managed by the user).
+
+You can set up and use multiple runtimes on the same computer, for example if
+you want to use them in different contexts.  Runtime-using commands let you
+select a different runtime than your default with command-line options (e.g.
+``--docker``, ``--conda``, and so on).  For example, you might use the Docker
+runtime to run small builds locally and then use the AWS Batch runtime to run
+large scale builds with more computing power.
+
+If you pick one runtime and later realize you want to switch, you can go back
+and set up the other.
+
+
+Comparison
+==========
+
+.. csv-table::
+    :file: comparison.csv
+    :header-rows: 1
+    :stub-columns: 1
+
+Isolation level
+    A relative ranking from least isolated (*none*, 0) to most isolated
+    (*great*, 3) from the underlying computer system.
+
+Containerized?
+    A containerized :term:`computing platform` provides a higher degree of
+    isolation, which in turn usually means a higher degree of portabililty and
+    reproducibility across different computers and users.
+
+Locality
+    *Local* means "on the same computer where ``nextstrain`` is being run".
+    *Remote* means "on a different computer".
+
+    Docker is most often used to run containers locally, but can also be used
+    to run them remotely.
+
+External dependencies
+    Third-party programs or configuration which are required to use a runtime
+    but not managed by :doc:`/commands/setup` and :doc:`/commands/update`.
+
+
+Compatibility
+=============
+
+Switching runtimes or updating the version of a runtime may result in different
+versions of Nextstrain components like Augur and Auspice as well as other
+programs, and thus different behaviour.
+
+Exact behavioural compatibility is not guaranteed between different runtimes
+(e.g. between the Docker vs. Conda runtimes) or between versions of the same
+runtime (e.g. between Docker runtime images
+``nextstrain/base:build-20230714T205715Z`` and
+``nextstrain/base:build-20230720T001758Z``).  However, the containerized
+runtimes (Docker, Singularity, AWS Batch; see comparison_ above) will usually
+have identical behaviour given the same runtime version (e.g. ``build-…``) as
+they are all based the same runtime image (``nextstrain/base``).  Any variance
+is typically due to use of external resources (intentional or otherwise) from
+outside the container.

doc/runtimes/singularity.rst

@@ -0,0 +1,6 @@
+===================
+Singularity runtime
+===================
+
+.. automodule:: nextstrain.cli.runner.singularity
+    :noindex: