rpcenc: Support reader redirect

[magik6k]

Includes docs and tests!

This should allow market processes to send piece bytes directly to workers

---

Closes #7107

[magik6k]

(note that this should not require api version bump, as older clients should still "just work", they just won't support redirects)

[magik6k]

Here's a wireshark capture of a TestReaderRedirect run just to confirm that it does exactly what's expeced:

[aarshkshah1992]

cc @jacobheun for tracking.

[jennijuju]

This won't be in v1.11.1.
@jacobheun @raulk however, I think this would be really nice to have as most of the miners uses sealing workers and this would keep even less work on the main miner process IIUC. having it in one of the M1 tags just to test it out will also be nice.
Im adding the M1 label for now just for tracking purpose, feel free to remove it if you this is out of scope.

[raulk]

This would be great to have, but it's an optimisation and we should consider for 1.11.3.

[jennijuju]

@raulk is this still need if storage.Json is copied overs/shared to the market nodes too?

[magik6k]

Yes, when adding pieces without this data would still flow through the main miner process

[benjaminh83]

@jennijuju I would like to challenge the decision on postponing for future releases.
IRL, the main reason for splitting out the market-node, is to protect the miner-node from deal overload. If all deal data still have to passthrough the miner-node in the add piece process, then we are only half way.
We need this implemented into production along with the market-split upgrade! I and multiple miners will be happy to test it out in M1, just to make sure its working. We want to help, as it is very important for us that this lands in a production release asap!

[raulk]

When do each of these cases happen?

[raulk]

Won't this interrupt the redirect flow on the first occurrence? 🤔

[magik6k]

It's meant to, we want to handle the redirect ourselves in https://github.com/filecoin-project/lotus/pull/6952/files#diff-309dd201d9384284851cf021bce98a146c76f5db8a982949b87f1e4b3d63cb26R138 (we need to save the last URL form the HEAD requests for the final POST. Unfortunately we can't just do a POST and have Go handle the redirects, because Go's http client will send the request body with the first request, evan if it then gets the redirect response - which is why we do this whole HEAD request dance here)

[magik6k]

(and we have to use this weird CheckRedirect to be able to save the last redirect Location header, as otherwise if you just let the http client handle the redirect, there is no way to get that URL)

[magik6k]

Note that this error has a very special meaning, perhaps not obvious:

ErrUseLastResponse can be returned by Client.CheckRedirect hooks to control how redirects are processed. If returned, the next request is not sent and the most recent response is returned with its body unclosed

[raulk]

Hoist to godoc for this method.

[jennijuju]

Hey team - what’s the status on this PR? Is anyone actively working on it and address the comments?

[jacobheun]

We'll aim to land this in 1.11.3 as the 1.11.2-rc is going out today and more testing is needed here, so we don't need to worry about backporting this to the 1.11.2 release.

[nonsense]

I validated this on sofiaminer (with nethogs) with a lotus-worker process alongside a lotus-miner (which does no sealing operations), and confirmed that we are no longer seeing IP traffic between markets process and lotus-miner, when there is a worker involved on AddPiece.