Skip to content

Commit

Permalink
add docs for Snakemake container support
Browse files Browse the repository at this point in the history
  • Loading branch information
rahuldesai1 committed Jan 19, 2024
1 parent 27c7f16 commit f9343c1
Show file tree
Hide file tree
Showing 2 changed files with 83 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ cli/mv
snakemake/tutorial.md
snakemake/lifecycle.md
snakemake/metadata.md
snakemake/environments.md
snakemake/cloud.md
snakemake/debugging.md
snakemake/troubleshooting.md
Expand Down
82 changes: 82 additions & 0 deletions docs/source/snakemake/environments.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Environments

## Configuring Conda and Container Environments

Latch's Snakemake integration supports the use of both the `conda` and `container` directives in your Snakefile. To configure which environment to run tasks in (which is typically done through the use of `--use-conda` and `--use-singularity`), add the `env_config` field to your workflow's `SnakemakeMetadata` object. For example,

```
# latch_metadata.py
from latch.types.metadata import SnakemakeMetadata, SnakemakeFileParameter, EnvironmentConfig
from latch.types.directory import LatchDir
from latch.types.metadata import LatchAuthor, LatchMetadata, LatchParameter
from pathlib import Path
SnakemakeMetadata(
display_name="snakemake_tutorial_workflow",
author=LatchAuthor(
name="latchbio",
),
env_config=EnvironmentConfig(
use_conda=True,
use_container=True,
),
parameters={
"samples" : SnakemakeFileParameter(
display_name="Sample Input Directory",
description="A directory full of FastQ files",
type=LatchDir,
path=Path("data/samples"),
),
"ref_genome" : SnakemakeFileParameter(
display_name="Indexed Reference Genome",
description="A directory with a reference Fasta file and the 6 index files produced from `bwa index`",
type=LatchDir,
path=Path("genome"),
),
},
)
```

If there is no `env_config` defined, Snakemake tasks on Latch will use both containers and conda environments by default.

## Using Private Container Registries

When executing Snakemake workflows in containers, it is possible that the container images will exist in a private registry that the Latch cloud does not have access to. Downloading images from private registries at runtime requires two steps:

1. Upload the password / access token of your private container registry to the Latch platform. See [Storing and using Secrets](../basics/adding_secrets.md).
2. Add the `docker_metadata` field to your workflow's `SnakemakeMetadata` object so that the workflow engine knows where to pull your credentials from. For example:

```
# latch_metadata.py
from latch.types.metadata import SnakemakeMetadata, SnakemakeFileParameter, DockerMetadata
from latch.types.directory import LatchDir
from latch.types.metadata import LatchAuthor, LatchMetadata, LatchParameter
from pathlib import Path
SnakemakeMetadata(
display_name="snakemake_tutorial_workflow",
author=LatchAuthor(
name="latchbio",
),
docker_metadata=DockerMetadata(
username="user0",
secret_name="LATCH_SECRET_NAME",
),
parameters={
"samples" : SnakemakeFileParameter(
display_name="Sample Input Directory",
description="A directory full of FastQ files",
type=LatchDir,
path=Path("data/samples"),
),
"ref_genome" : SnakemakeFileParameter(
display_name="Indexed Reference Genome",
description="A directory with a reference Fasta file and the 6 index files produced from `bwa index`",
type=LatchDir,
path=Path("genome"),
),
},
)
```

**Note**: the `secret_name` field specifies the name of the Latch Secret uploaded in step #1, NOT the actual registry password.

0 comments on commit f9343c1

Please sign in to comment.