Support multiple chain nodes per miner node

[magik6k]

Currently each lotus-miner instance can only be backed by a single lotus chain node. This is obviously not ideal as there is no redundancy for chain operations - if the chain node dies, the miner node will become unusable until the chain node is fixed or switched.

This can be done in two ways:

• Make it possible to attach multiple chain nodes to a single miner, balance api calls between them
  • This needs Cross-node nonce coordination #9129 to work well
• Same as above, but use one node, and only switch when that node dies
  • Even in this case we most likely want Cross-node nonce coordination #9129 to avoid weird races on nonces

The need for #9129 could be mitigated by requiring use of separate control addresses on each node, but that makes setup harder, and requires a lot of on-chain accounts, which is not ideal.

[magik6k]

Whichever way we go, we likely want to start by writing a helper which takes multiple API connections - basically a method accepting multiple api interfaces and outputting one api proxy.

We already have some code doing similar transforms on the API interfaces:

• For auth - https://github.com/filecoin-project/go-jsonrpc/blob/master/auth/auth.go#L34; https://github.com/filecoin-project/lotus/blob/master/api/permissioned.go#L22
• For metrics -

  lotus/metrics/proxy/proxy.go

  Line 13 in edcb287

  func MetricedStorMinerAPI(a api.StorageMiner) api.StorageMiner {

Then we just need to make it possible to connect to multiple chain nodes in https://github.com/filecoin-project/lotus/blob/master/cmd/lotus-miner/run.go#L77 using that helper

[Fatman13]

Venus team have a proposal to approach this problem too. Comments and questions are welcomed!

Proposal:

• RFP Application - chain-co devgrants#770