Skip to content
This repository has been archived by the owner on Dec 20, 2024. It is now read-only.

add seed downloader #1297

Merged

Conversation

wangforthinker
Copy link
Contributor

Ⅰ. Describe what this PR did

Add downloader for seed file. Please see refer dragonflyoss/dragonfly#1284.

Ⅱ. Does this pull request fix one issue?

NONE.

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗?)

Add ut.

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@pouchrobot
Copy link
Collaborator

Thanks for your contribution. 🍻 @wangforthinker
While we thought PR TITLE could be more specific, longer than 20 chars.
Please edit this PR title instead of opening a new one.
More details, please refer to https://github.com/dragonflyoss/Dragonfly/blob/master/CONTRIBUTING.md

@wangforthinker wangforthinker force-pushed the feat/add-seed-downloader-prepare-pr branch 2 times, most recently from fdaf6e7 to 387d807 Compare April 23, 2020 02:54
@codecov-io
Copy link

Codecov Report

Merging #1297 into master will increase coverage by 0.01%.
The diff coverage is 54.28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1297      +/-   ##
==========================================
+ Coverage   51.11%   51.12%   +0.01%     
==========================================
  Files         122      124       +2     
  Lines        8082     8152      +70     
==========================================
+ Hits         4131     4168      +37     
- Misses       3622     3646      +24     
- Partials      329      338       +9     
Impacted Files Coverage Δ
pkg/errortypes/errortypes.go 71.42% <0.00%> (-21.17%) ⬇️
dfdaemon/seed/downloader.go 57.44% <57.44%> (ø)
dfdaemon/seed/utils.go 73.33% <73.33%> (ø)
supernode/daemon/mgr/scheduler/manager.go 21.91% <0.00%> (-0.69%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 06d2368...387d807. Read the comment docs.

Signed-off-by: allen.wq <[email protected]>
@wangforthinker wangforthinker force-pushed the feat/add-seed-downloader-prepare-pr branch from 387d807 to 1da7fba Compare April 23, 2020 03:01
@xujihui1985
Copy link

LGTM


// downloader manage the downloading of seed file.
type downloader interface {
DownloadToWriterAt(ctx context.Context, rangeStruct httputils.RangeStruct, timeout time.Duration, writeOff int64, writerAt io.WriterAt, rateLimit bool) (length int64, err error)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter list doesn't contain url and it's initialized in creating instance of a concrete implementation. Why is it designed this way?

And this method has 2 things to do:

  1. downloads data from remote server.
  2. writes data into target writer.

I think that the step 2 may be not the responsibility of downloader interface. Will it be better to remove it from the interface?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. a downloader binds to a seed, so the information of seed will be transferred by creating instance of a concrete implementation. In the other word, we may support other protocol like http/https/oss/hdfs ,so the parameters may not be easy to define.
  2. in our case, we want to cover the error handler of download. And the data finally should be write ,so we do write in the interface. And some optimization could be make in the interface, such as buffer Write-through which reduces the costs of copying buffer.

Copy link
Member

@lowzj lowzj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants