dfget download file success with incorrect md5

[zcc35357949]

Ⅰ. Issue Description

I use master code. Currently dfget may download file sucessfully, but the md5 value is incorrect.
retry the same file many times, md5 values are different!!

root@nvm-14:~ # /data/app/src/github.com/dragonflyoss/Dragonfly/cmd/dfdaemon/dfget -u http://172.16.17.24:8000/go1.12.9.linux-amd64.tar.gz -o /tmp/go1.12.9.linux-amd64.tar.gz --pattern p2p -s 800MB --totallimit 800MB --node 172.16.17.24:8002,172.16.17.24:8002 --verbose
--2020-07-06 21:50:33--  http://172.16.17.24:8000/go1.12.9.linux-amd64.tar.gz
dfget version:
workspace:/root/.small-dragonfly
sign:21233-1594043433.459
client:172.16.17.24 connected to node:172.16.17.24:8002
start download by dragonfly...
download SUCCESS cost:1.240s length:127961523 reason:0
root@nvm-14:~ # md5sum /tmp/go1.12.9.linux-amd64.tar.gz
6a5e658f7c0eeac60f14dc425abc007e  /tmp/go1.12.9.linux-amd64.tar.gz
root@nvm-14:~ # /data/app/src/github.com/dragonflyoss/Dragonfly/cmd/dfdaemon/dfget -u http://172.16.17.24:8000/go1.12.9.linux-amd64.tar.gz -o /tmp/go1.12.9.linux-amd64.tar.gz --pattern p2p -s 800MB --totallimit 800MB --node 172.16.17.24:8002,172.16.17.24:8002 --verbose
--2020-07-06 21:50:45--  http://172.16.17.24:8000/go1.12.9.linux-amd64.tar.gz
dfget version:
workspace:/root/.small-dragonfly
sign:22691-1594043445.594
client:172.16.17.24 connected to node:172.16.17.24:8002
start download by dragonfly...
download SUCCESS cost:0.260s length:127961523 reason:0
root@nvm-14:~ # md5sum /tmp/go1.12.9.linux-amd64.tar.gz
24da2f521dbf610f01de747fe3e04dab  /tmp/go1.12.9.linux-amd64.tar.gz

The file size is 123MB.

I change the default writer buffer size to 4MB，the problem goes away!!
https://github.com/dragonflyoss/Dragonfly/blob/master/pkg/pool/writer_pool.go#L28
4M is the default piece size for under 200M file. I just don't know the relation between them.

Ⅱ. Describe what happened

Ⅲ. Describe what you expected to happen

Ⅳ. How to reproduce it (as minimally and precisely as possible)

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

• dragonfly version:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[zcc35357949]

@lowzj

[jim3ma]

which version or commit in you machine?

[zcc35357949]

which version or commit in you machine?

I use the latest master code to compile and run.

[jim3ma]

This function return a buffer, so content should not be reset.

[zcc35357949]

This function return a buffer, so content should not be reset.

maybe we should use an independent go routine to reset the piece buffer in order to make sure that the piece is really useless.
This bug also leads to issue #1410 .

[jim3ma]

We can just reset the piece buffer after io.Copy returns. See #1418

[lowzj]

fixed by #1411 , closed