batch publish deal messages

[dirkmc]

Fixes #4878

TODO:

• Fix GetCurrentDealInfo to work with publish messages that have multiple deals
• Integration tests
• Remove dealID from go-fil-markets calls to OnDealSectorPreCommitted & OnDealSectorCommitted Simplify on deal commit handlers #5413
• Fix storage sealing so that it can handle publish messages that have multiple deals Handle batch publish storage deals message in sealing recovery #5411
• Move handling of publish deals message return value from markets into lotus Move waiting for publish deals from markets into lotus #5428 Move publish deals msg response parsing from markets into lotus go-fil-markets#475
• Document config params - add config options for batching deal publish filecoin-docs#658

[dirkmc]

Most of the changes in this file are just refactoring some common code into functions

[dirkmc]

These are placeholder defaults for PublishPeriod and MaxDealsPerMsg, open to suggestions for these values

[s0nik42]

What is the MaxDealsPerMsg supported by the network ?

[f8-ptrk]

i have seen up to 30 on the chain already -but this should be proper documented with the max values

[s0nik42]

@dirkmc how deal batching will be exposed to miners via lotus ? specific command line or automatically ?

[dirkmc]

It will be automatic, with two config values:

• The maximum number of deals to include in a publish message
• The amount of time to wait for more deals after the first deal arrives, before publishing the deal

[magik6k]

Just a couple of nits, generally looks good.

Would also be great to have commands to see pending deals / force send a publish message

[dirkmc]

Would also be great to have commands to see pending deals / force send a publish message

Good idea, I created a ticket: #5491

[jennijuju]

Seems like if I’m sending a message for deals 1,2,3,4,5, with start epoch increasingly and when the time of sending the message has passed the 3’s start epoch, the message will fail for all deals.

will one be able to republish the valid deals, say for deals 4,5?

assuming we add some checks to run the message locally first before sending it out, what will happen if by the time of the message landed on chain, one of the deal expired? @dirkmc @magik6k

[dirkmc]

#5505 implements a fix for the case where the deal proposal start epoch has already elapsed by the time the publish message is about to be sent. The fix is that we will filter this deal out of the publish message.

I'm not sure if we can do much in lotus about the case where the publish message takes a few epochs to land on chain, and in the mean time the start epoch of one of the deals expires. I believe we would need a change in specs-actors to account for this case:

• Currently the market actor validates each deal
• If one of the deals fails validation, the whole message fails

[magik6k]

LGTM. Also did some manual testing on my mining setup, and it seems to work well

[f8-ptrk]

can you give a quick overview on the config params until they are properly documented?

[dirkmc]

@PatrickDeuse I just opened a PR against filecoin docs here: filecoin-project/filecoin-docs#658
Please let me know if that explanation is clear.