-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathrunning_poreduck.html
263 lines (213 loc) · 6.8 KB
/
running_poreduck.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
<!DOCTYPE html>
<html>
<head>
<title>tarMyFast5 with Docker</title>
<meta charset="utf-8">
<meta name="author" content="Alexis Lucattini" />
<link href="libs/remark-css-0.0.1/default.css" rel="stylesheet" />
<link href="libs/remark-css-0.0.1/default-fonts.css" rel="stylesheet" />
</head>
<body>
<textarea id="source">
class: center, middle, inverse, title-slide
# tarMyFast5 with Docker
## In memory of two poor ducks.
### Alexis Lucattini
### 2018/04/17
---
# What is poreduck
.pull-left[
Two simple programs designed to:
1. tarMyFast5:
+ Pacakge fast5 data on a laptop ready for transfer to a server.
+ Provide some theoretical run metrics
2. albacoreHPC
+ Run albacore on said server.
+ And generate some actual run metrics.
]
.pull-right[
<img src=images/poreduck_logo.png width="70%">
]
---
# Downloading the test data.
We have been fortunate to have a repository on AARNET's cloud stor.
Please download the following [compressed zip file](https://cloudstor.aarnet.edu.au/plus/s/YF69Ppmh5xTg3LQ) and extract the reads folder into the appropriate directory.
---
# Tar my fast5 steps
1. Install Docker
2. Download docker image
3. Create a docker container
4. Copy across the samplesheet into the container
5. Commit the container to a new image
6. Mount the reads folder and run the new image
7. Rsyncing to a server
---
# Installing Docker (5-10 mins)
**Windows and Mac users**
Docker enables us to run the same program on every platform.
We will use the [Docker toolbox](https://docs.docker.com/toolbox/overview/) for Windows/Mac systems.
Docker toolbox is the legacy system and is compatible on Windows 7 and above.
.small[You should also install Git Bash and Kitematic as recommended by the docker installer]
**Ubuntu users**
`sudo apt-get install docker.io`
`sudo groupadd docker`
`sudo gpasswd -a $USER docker`
Now log out and back in or run `sudo run newgrp docker`.
Then `sudo service docker restart`
You should not have to use sudo when running docker commands from now on.
---
# Downloading the docker image (10-mins)
.pull-left[
## Terminal by platfrom
**Windows users**
Open up Git Bash
**Mac Users**
Open up Docker Quick Start Terminal
**Ubuntu users**
Open up a new terminal
]
.pull-right[
## Commands to run
```bash
# Download the image
docker pull alexiswl/poreduck
# List the images
docker images
# Create a docker instance
docker container create alexiswl/poreduck
# See the name of the instance
docker ps -a
# The container name will be on the far right
# It will be something like elegant_swan
# For the rest of this tutorial,
# please substitute elegant_swan with your container.
```
]
---
# Copy across the samplesheet (3 mins)
.pull-left[
### About the samplesheet
The samplesheet is a tab delimited file.
The samplesheet has the following header columns.
1. SampleName .small[as named in MinKNOW]
+ *LAMBDA_RAPID_TEST*
2. UTCMuxStartDate:
+ *20170628*
3. UTCMuxStartTime:
+ *0521*
4. UTCSeqStartDate:
+ *20170628*
5. UTCSeqStartTime:
+ *0526*
.small[*If you forgot to name your sample in MinKNOW, use '%' followed by your sample name of choice.*]
]
.pull-right[
### Commands to run
```bash
# Download the test samplesheet
wget -O samplesheet.tsv https://raw.github.com/alexiswl/ASimpleNanoporeTutorial/master/samplesheet.tsv
# View the samplesheet
cat samplesheet.tsv
# Copy the samplesheet into the container
docker container cp samplesheet.tsv elegant_swan:/samplesheet.tsv
```
]
---
# Committing the container (3 mins)
We will now convert the modified container back into an image.
.pull-left[
## Containers vs images
The terms 'containers' and 'images' are sometimes interchanged.
They are different, for more information
this [stack-overflow](https://stackoverflow.com/questions/23735149/what-is-the-difference-between-a-docker-image-and-a-container) thread is worth reads.
]
.pull-right[
## Commands to run
```bash
# Commit the container
docker container commit elegant_swan poreduck_lambda
# View the list of images
docker images
```
]
---
## Allowing a mount to docker. (3 mins) .tiny["Windows/Mac"]
.pull-left[
.small[
1. Shut down KiteMatic and open up VirtualBox.
2. By default, docker only allows mounts the user directory.
3. Open up virtual box, select default.
4. Mount `C:/data/` or `/Library/MinKNOW/data/` to `/data`.
5. Press Ctrl+F to shut down the Virtual box.
6. Reopen KiteMatic.
]
]
.pull-right[
<img src="./images/docker_settings.png" width="110%">
]
---
## Mounting the reads path / running the container
Mounting uses the --volume parameter.
It allows the running container to read/write and execute files on the laptop.
By default, no files or folders are accessible to the container.
.pull-left[
### Default /path/to/reads by platform
**Windows users**
`/c/data`
**Mac Users**
`/Library/MinKNOW/data/reads`
**Ubuntu users**
`/var/lib/MinKNOW/data/reads`
.small[`/path/to/reads` should be `/data/reads` if you completed the step on the previous page.]
]
.pull-right[
## Commands to run
```bash
# Make note of quotes
docker run --volume '/path/to/reads':/data/reads poreduck_lambda tarMyFast5 --reads_path /data/reads --samplesheet /samplesheet.tsv
```
]
---
You're done!
You will now see next to fast5 for each of your runs, there will be a 'metadata' folder and a 'plots' folder.
The metadata folder contains a tsv file for each folder in the fast5 folder.
# Transferring your data to a server with rsync
Running rsync to transfer data to your server.
Before running the rsync tutorial, please see the 'creating an ssh-key' tutorial.
This will mean you not have to enter a password, everytime the rsync runs.
</textarea>
<script src="https://remarkjs.com/downloads/remark-latest.min.js"></script>
<script>var slideshow = remark.create({
"highlight": "pygments",
"highlightLines": true,
"countIncrementalSlides": false
});
if (window.HTMLWidgets) slideshow.on('afterShowSlide', function (slide) {
window.dispatchEvent(new Event('resize'));
});
(function() {
var d = document, s = d.createElement("style"), r = d.querySelector(".remark-slide-scaler");
if (!r) return;
s.type = "text/css"; s.innerHTML = "@page {size: " + r.style.width + " " + r.style.height +"; }";
d.head.appendChild(s);
})();</script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
skipTags: ['script', 'noscript', 'style', 'textarea', 'pre']
}
});
</script>
<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
(function () {
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = 'https://cdn.bootcss.com/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML';
if (location.protocol !== 'file:' && /^https?:/.test(script.src))
script.src = script.src.replace(/^https?:/, '');
document.getElementsByTagName('head')[0].appendChild(script);
})();
</script>
</body>
</html>