Describe persistent storage of notebook files and credential store

[ckunki]

Ticket #51 investigated how to store the Notebooks persistent, in case of Docker Container removal, resulting in the proposal to use Docker VOLUMES for notebook files and secret store entries.

The current idea is to support 2 scenarios

• S1) AI-Lab manages the data anonymously and encapsulated
• S2) user (partially) modifies the data, keeps a copy of the data and provides the data when using the AI-Lab

Following the results of ticket #51:

• By default, Ai-Lab will create an anonymous Docker Volume and mount it to $MOUNT inside the Docker Container, which implements scenario S1.
• For scenario S2
  • The user
    • Creates an empty Docker Volume (either by docker volume create or simply by using in docker -v)
    • mounts the empty volume to $MOUNT when starting the Docker Container
  • On Container start
    • Ai_lab will call entrypoint.py (as always)
    • entrypoint.py copies the data from $BACKUP to $MOUNT
    • hence, into the user's Docker volume, which is empty at that point in time
  • If the user runs the Container again
    • User again mounts their volume to $MOUNT
    • The volume is no longer empty but contains the files copied from $BACKUP
    • Additionally, the user may have modified the files in the last session
    • Docker will not overwrite or add any files in $MOUNT as the experiments in ticket Investigate how to store the Notebooks persistent, in case of Docker Container removal #51 showed.

The current ticket therefore requests to

• add a VOLUME directive to Dockerfile
• Document the usage so that end users can choose between scenario S1 and S2
  • How to create a docker volume?
  • How to access files in it?
  • How to mount it to the DSS Docker container?
  • How to backup it?

For mounting a Docker volume please do not add a slash as prefix to the volume name.

• -v /host:/container mounts host directory /host
• while -v vol:/container mounts Docker volume vol

The mount point inside the Docker container in both cases is /container.