how do I find out what file didn't download?

[clarkevans]

Hello. So, I tried to use DataDeps, and it I get a stacktrace. But there's no information here about what file actually failed to download. I've checked them all individually. But I'm at a loss from the stacktrace what else to do.

ERROR: HTTP.ExceptionRequest.StatusError(404, HTTP.Messages.Response: """
HTTP/1.1 404 Not Found
ETag: "29908c07f9f25a28cdb6aa1ac8f09674:1461343688"
Last-Modified: Fri, 22 Apr 2016 16:48:08 GMT
Accept-Ranges: bytes
Content-Length: 25962
Content-Type: text/html
Cache-Control: max-age=0, no-cache, no-store, max-age=0, no-cache, no-store, max-age=0, no-cache, no-store                                                                                                                
Date: Sat, 25 May 2019 16:09:25 GMT
Connection: keep-alive
Cache-Control: max-age=0, no-cache, no-store
Access-Control-Allow-Origin: *
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Permitted-Cross-Domain-Policies: master-only
Strict-Transport-Security: max-age=31536000

""") Stacktrace: [1] (::getfield(Base, Symbol("##696#698")))(::Task) at ./asyncmap.jl:178 [2] foreach(::getfield(Base, Symbol("##696#698")), ::Array{Any,1}) at ./abstractarray.jl:1866 [3] maptwice(::Function, ::Channel{Any}, ::Array{Any,1}, ::Array{Array{String,1},1}) at ./asyncmap.jl:178 [4] wrap_n_exec_twice at ./asyncmap.jl:154 [inlined] [5] #async_usemap#681(::Int64, ::Nothing, ::Function, ::getfield(DataDeps, Symbol("##14#15")){typeof(DataDeps.fetch_http),String}, ::Array{Array{String,1},1}) at ./asyncmap.jl:103 [6] #async_usemap at ./none:0 [inlined] [7] #asyncmap#680 at ./asyncmap.jl:81 [inlined] [8] asyncmap at ./asyncmap.jl:81 [inlined] [9] run_fetch at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution_automatic.jl:104 [inlined] [10] #download#13(::Array{Array{String,1},1}, ::Nothing, ::Bool, ::Function, ::DataDep{Nothing,Array{Array{String,1},1},typeof(DataDeps.fetch_http),Array{getfield(SynPUF, Symbol("##3#4")),1}}, ::String) at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution_automatic.jl:78 [11] download at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution_automatic.jl:70 [inlined] [12] handle_missing at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution_automatic.jl:10 [inlined] [13] _resolve(::DataDep{Nothing,Array{Array{String,1},1},typeof(DataDeps.fetch_http),Array{getfield(SynPUF, Symbol("##3#4")),1}}, ::String) at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution.jl:83 [14] resolve(::DataDep{Nothing,Array{Array{String,1},1},typeof(DataDeps.fetch_http),Array{getfield(SynPUF, Symbol("##3#4")),1}}, ::String, ::String) at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution.jl:29 [15] resolve(::String, ::String, ::String) at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution.jl:54 [16] resolve(::String, ::String) at /home/cce/.julia/packages/DataDeps/LiEdA/src/resolution.jl:73 [17] top-level scope at none:0

[oxinabox]

Yes, I often wish HTTP.jl's error messages included the URL it was trying to fetch at the time.

We could add that in DataDeps,
by doing a try-catch in the default fetch_method, which throws a new exception that includes the request url
(And as a short term workaround you could add it to a custom fetch method)

But I think it should be addressed upstream in HTTP.jl

Though possible it should be done regardless of the fetch method so should be in the perform_fetch code in DataDeps