Frequent "HTTP session did not complete" warnings with wpull 2.0.x

[JustAnotherArchivist]

On wpull 2.0.3 in ArchiveBot, the "HTTP session did not complete" occurs fairly frequently. I don't think I've ever seen it with wpull 1.2.3 although the warning exists in the codebase since commit 20e051a, first released as v0.1007 as far as I can see.

We need to investigate why this warning occurs so much more frequently and whether it has any implications for the completeness of the payload. I suspect that it has something to do with 2.0's network stack, which also has various other problems that did not exist in 1.2.3.