Dulwich does not report progress during actual download

[danchr]

When cloning a large repository, Git will show you progress lines like this:

Receiving objects:  49% (196978/395839), 148.99 MiB | 21.35 MiB/s

By comparison, Dulwich is silent and does not output anything after the initial:

Compressing objects: 100% (36222/36222), done.

[jelmer]

0.21 should be better in terms of reporting progress, though it doesn't report during the receiving phase either.

Dulwich currently writes into a SpooledTemporaryFile first and then parses the file it's written. Ideally we'd be parsing while receiving (and writing to disk as bytes come in), and then we could provide a progress indicator. I did do some experimentation and that suggested there isn't a massive performance improvement to be gained from inlining this, though I'm sure there are some benefits.

The problem here is that the interface for verifying and copying a pack file expects to read from a file object, and the interface that receives a pack from a remote server calls a callback. We should do the refactoring to have those fit better onto each other; patches welcome :)