Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network upgrade support #3781

Merged
merged 110 commits into from
Sep 23, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
d359483
[WIP] Network upgrade support
Stebalien Sep 12, 2020
38f8798
Fix some build errors
magik6k Sep 14, 2020
6809713
Fix vm build
magik6k Sep 14, 2020
683a581
More terraforming in chain/
magik6k Sep 14, 2020
06ec571
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
magik6k Sep 14, 2020
fb3f710
Some post-master merge fixes
magik6k Sep 14, 2020
2088335
rename to simplify merge
Stebalien Sep 14, 2020
02dcb5e
Merge branch 'master' into refactor/net-upgrade
Stebalien Sep 14, 2020
cc4d530
Progress
Stebalien Sep 14, 2020
7fa6c1e
add info method to miner state API
Stebalien Sep 14, 2020
36f920b
progress
Stebalien Sep 15, 2020
4dabab5
state manager progress
magik6k Sep 15, 2020
4419850
state api impl fixes
magik6k Sep 15, 2020
9dad38c
gofmt
magik6k Sep 15, 2020
00c6397
more progress
Stebalien Sep 15, 2020
6bf7976
fix decode
Stebalien Sep 15, 2020
4cd92d8
remove final WithStateTree
Stebalien Sep 15, 2020
53384e8
migrate wallet access
Stebalien Sep 15, 2020
c64f983
migrate StateMinserSectorCount
Stebalien Sep 15, 2020
91e9573
Compile fixes
Stebalien Sep 15, 2020
4e01fad
start multisig abstraction
Stebalien Sep 15, 2020
e1ba4eb
sub reward actor
Stebalien Sep 15, 2020
02bc5fa
add power actor address alias
Stebalien Sep 15, 2020
92471d4
migrate precommit deposit function
Stebalien Sep 15, 2020
454c382
migrate StateMinerInitialPledgeCollateral
Stebalien Sep 16, 2020
05c1153
feat(paych): convert paych actor
hannahhoward Sep 16, 2020
b4ee519
Partial progress towards switching to miner and power interfaces
arajasek Sep 16, 2020
90853e2
Add a boolean HasMinPower to return of GetPower
arajasek Sep 16, 2020
72d19f3
Incremental progress towards using new power state interface
arajasek Sep 16, 2020
ccce1a9
Merge pull request #3877 from filecoin-project/refactor/net-upgrade-p…
magik6k Sep 16, 2020
b530f25
Migrate miner actor
arajasek Sep 16, 2020
7115485
Add a getter for network version
arajasek Sep 17, 2020
80b6994
feat(market): update state diffing for market actor
hannahhoward Sep 17, 2020
81004ea
Merge pull request #3889 from filecoin-project/refactor/net-upgrade-m…
arajasek Sep 17, 2020
b5ba7a0
Miner migration
arajasek Sep 17, 2020
9e48dd2
Fixups
arajasek Sep 17, 2020
b606149
Migrate reward actor
arajasek Sep 17, 2020
e2295c3
Migrate multisig actor
arajasek Sep 17, 2020
691bd9f
feat(markets): complete markets conversion
hannahhoward Sep 17, 2020
7bf165c
Remove miner-related specs actors types from API
arajasek Sep 17, 2020
053cfc1
Migrate verifreg actor
arajasek Sep 17, 2020
31ff523
Get State API almost working
arajasek Sep 17, 2020
e632643
api: Test return types
magik6k Sep 17, 2020
6eda535
Most tests passing
magik6k Sep 17, 2020
68e884e
docsgen
magik6k Sep 17, 2020
82b95e3
cbor gen
magik6k Sep 17, 2020
c40c136
fix: paych To()
dirkmc Sep 17, 2020
18bc59b
uncruft
Stebalien Sep 17, 2020
708aa36
remove dep on v0 miner
Stebalien Sep 17, 2020
ae38970
remove WpostProvignPeriod function
Stebalien Sep 17, 2020
dc58f71
remove unnecessary code
Stebalien Sep 17, 2020
5bcfee0
make market diffs work across version upgrades
Stebalien Sep 17, 2020
b2ee590
improve diff logic
Stebalien Sep 18, 2020
f2a0779
remove specs-actors import
Stebalien Sep 18, 2020
8bd6791
more fixups
Stebalien Sep 18, 2020
daa441b
simplify market diff
Stebalien Sep 18, 2020
37de154
Fixup tests
arajasek Sep 18, 2020
fce423c
Appease the linter and get everything building
arajasek Sep 18, 2020
694463f
More lint fixes
arajasek Sep 18, 2020
70faa36
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
magik6k Sep 18, 2020
ccd0a67
wip fixing wdpost tests
magik6k Sep 18, 2020
c3046f4
Don't use -1 as default sector number
arajasek Sep 18, 2020
fb2b25c
finish upgrading chainwatch
Stebalien Sep 18, 2020
298efa2
fix lint errors
Stebalien Sep 18, 2020
8747c60
abstract over account actor
Stebalien Sep 18, 2020
24df873
rename imports
Stebalien Sep 18, 2020
e60027c
remove todo
Stebalien Sep 18, 2020
8285eda
migrate storage miner info
Stebalien Sep 18, 2020
4a7055c
fix state loading in vm
Stebalien Sep 18, 2020
f741ce6
fixup some more imports
Stebalien Sep 18, 2020
c130806
compile fix
Stebalien Sep 18, 2020
1bf3b49
rename imports to match actors code
Stebalien Sep 18, 2020
35bce5a
revert post changes
Stebalien Sep 18, 2020
7c3f638
Abstract FilterEstimate, PreCommitDepositForPower, and InitialPledgeF…
arajasek Sep 19, 2020
b25dd2a
More correct / fasterer GetSectorsForWinningPoSt
magik6k Sep 20, 2020
ed285f1
Abstract SectorOnChainInfo and SectorPreCommitOnChainInfo
arajasek Sep 20, 2020
a95e34f
Fix build
arajasek Sep 21, 2020
b355eb7
Add compile-time checks that actors interfaces are correctly implemented
arajasek Sep 21, 2020
286fe04
Add some TODOs
arajasek Sep 21, 2020
a307e45
wdpost: Fix TestWDPostDoPost
magik6k Sep 21, 2020
0b5e4a9
Make GetSectorsForWinningPoSt fast again
magik6k Sep 21, 2020
f7934b0
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
magik6k Sep 21, 2020
0256631
non-destructive diff
Stebalien Sep 21, 2020
4cf0c10
optimize sector loading
Stebalien Sep 21, 2020
24ae920
cross-version state tree diff
Stebalien Sep 21, 2020
f9a896f
fix compile
Stebalien Sep 21, 2020
0ab2459
convert multisig to new actor abstractions
Stebalien Sep 21, 2020
a41bf74
fix: remove incorrect variable
Stebalien Sep 21, 2020
916421b
convert lotus-shed balances
Stebalien Sep 21, 2020
4bab784
migrate lotus-shed verifreg to specs-actors abstractions
Stebalien Sep 21, 2020
3f0106c
migrate lotus-shed/genesis-verify to actor abstraction
Stebalien Sep 21, 2020
63f026f
migrate some more imports
Stebalien Sep 21, 2020
d33dd4f
more renames
Stebalien Sep 21, 2020
d56da1b
Refinements to stmgr and utils
arajasek Sep 22, 2020
1dc69e3
Resolve some unnecessary actor upgrade TODOs
arajasek Sep 22, 2020
fc15888
fixup some imports
Stebalien Sep 22, 2020
5314ba8
remove ptr indirection
Stebalien Sep 22, 2020
ab070f2
copy actor object when iterating over actors
Stebalien Sep 22, 2020
bc24fdb
finish migrating statemanager
Stebalien Sep 22, 2020
441d7ff
cleanup imports some more
Stebalien Sep 22, 2020
c91774b
remove old comment
Stebalien Sep 22, 2020
2967c4e
use abstract actor type methods
Stebalien Sep 22, 2020
46fb0e7
add deal state iterator
Stebalien Sep 22, 2020
77f81fc
Test fix
arajasek Sep 22, 2020
7737147
update oni
Stebalien Sep 23, 2020
ed4bf9b
API shouldn't depend on actors directly
arajasek Sep 23, 2020
e27fc03
Reward state interface only needs cbor.Marshaler
arajasek Sep 23, 2020
476e799
Add an error return to all actor state interface methods
arajasek Sep 23, 2020
819180f
Implement inefficient OnAddressMapChange predicate
arajasek Sep 23, 2020
a876a0b
Use actor state addresses
arajasek Sep 23, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 42 additions & 44 deletions api/api_full.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@ import (
"fmt"
"time"

"github.com/filecoin-project/go-state-types/dline"

"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/filecoin-project/go-state-types/network"

"github.com/ipfs/go-cid"
"github.com/libp2p/go-libp2p-core/peer"
Expand All @@ -20,12 +18,13 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/go-state-types/dline"
"github.com/filecoin-project/lotus/chain/actors/builtin/market"
"github.com/filecoin-project/lotus/chain/actors/builtin/miner"
"github.com/filecoin-project/lotus/chain/actors/builtin/paych"

"github.com/filecoin-project/lotus/chain/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors/builtin/power"
"github.com/filecoin-project/lotus/chain/types"
marketevents "github.com/filecoin-project/lotus/markets/loggers"
"github.com/filecoin-project/lotus/node/modules/dtypes"
Expand Down Expand Up @@ -231,7 +230,7 @@ type FullNode interface {
WalletSignMessage(context.Context, address.Address, *types.Message) (*types.SignedMessage, error)
// WalletVerify takes an address, a signature, and some bytes, and indicates whether the signature is valid.
// The address does not have to be in the wallet.
WalletVerify(context.Context, address.Address, []byte, *crypto.Signature) bool
WalletVerify(context.Context, address.Address, []byte, *crypto.Signature) (bool, error)
// WalletDefaultAddress returns the address marked as default in the wallet.
WalletDefaultAddress(context.Context) (address.Address, error)
// WalletSetDefault marks the given address as as the default one.
Expand Down Expand Up @@ -314,22 +313,20 @@ type FullNode interface {
// StateNetworkName returns the name of the network the node is synced to
StateNetworkName(context.Context) (dtypes.NetworkName, error)
// StateMinerSectors returns info about the given miner's sectors. If the filter bitfield is nil, all sectors are included.
// If the filterOut boolean is set to true, any sectors in the filter are excluded.
// If false, only those sectors in the filter are included.
StateMinerSectors(context.Context, address.Address, *bitfield.BitField, bool, types.TipSetKey) ([]*ChainSectorInfo, error)
StateMinerSectors(context.Context, address.Address, *bitfield.BitField, types.TipSetKey) ([]*miner.SectorOnChainInfo, error)
// StateMinerActiveSectors returns info about sectors that a given miner is actively proving.
StateMinerActiveSectors(context.Context, address.Address, types.TipSetKey) ([]*ChainSectorInfo, error)
StateMinerActiveSectors(context.Context, address.Address, types.TipSetKey) ([]*miner.SectorOnChainInfo, error)
// StateMinerProvingDeadline calculates the deadline at some epoch for a proving period
// and returns the deadline-related calculations.
StateMinerProvingDeadline(context.Context, address.Address, types.TipSetKey) (*dline.Info, error)
Stebalien marked this conversation as resolved.
Show resolved Hide resolved
// StateMinerPower returns the power of the indicated miner
StateMinerPower(context.Context, address.Address, types.TipSetKey) (*MinerPower, error)
// StateMinerInfo returns info about the indicated miner
StateMinerInfo(context.Context, address.Address, types.TipSetKey) (MinerInfo, error)
StateMinerInfo(context.Context, address.Address, types.TipSetKey) (miner.MinerInfo, error)
// StateMinerDeadlines returns all the proving deadlines for the given miner
StateMinerDeadlines(context.Context, address.Address, types.TipSetKey) ([]*miner.Deadline, error)
// StateMinerPartitions loads miner partitions for the specified miner/deadline
StateMinerPartitions(context.Context, address.Address, uint64, types.TipSetKey) ([]*miner.Partition, error)
StateMinerDeadlines(context.Context, address.Address, types.TipSetKey) ([]Deadline, error)
// StateMinerPartitions returns all partitions in the specified deadline
StateMinerPartitions(ctx context.Context, m address.Address, dlIdx uint64, tsk types.TipSetKey) ([]Partition, error)
// StateMinerFaults returns a bitfield indicating the faulty sectors of the given miner
StateMinerFaults(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error)
// StateAllMinerFaults returns all non-expired Faults that occur within lookback epochs of the given tipset
Expand All @@ -349,9 +346,9 @@ type FullNode interface {
// expiration epoch
StateSectorGetInfo(context.Context, address.Address, abi.SectorNumber, types.TipSetKey) (*miner.SectorOnChainInfo, error)
// StateSectorExpiration returns epoch at which given sector will expire
StateSectorExpiration(context.Context, address.Address, abi.SectorNumber, types.TipSetKey) (*SectorExpiration, error)
StateSectorExpiration(context.Context, address.Address, abi.SectorNumber, types.TipSetKey) (*miner.SectorExpiration, error)
// StateSectorPartition finds deadline/partition with the specified sector
StateSectorPartition(ctx context.Context, maddr address.Address, sectorNumber abi.SectorNumber, tok types.TipSetKey) (*SectorLocation, error)
StateSectorPartition(ctx context.Context, maddr address.Address, sectorNumber abi.SectorNumber, tok types.TipSetKey) (*miner.SectorLocation, error)
// StateSearchMsg searches for a message in the chain, and returns its receipt and the tipset where it was executed
StateSearchMsg(context.Context, cid.Cid) (*MsgLookup, error)
// StateMsgGasCost searches for a message in the chain, and returns details of the messages gas costs, including the penalty and miner tip
Expand Down Expand Up @@ -388,13 +385,15 @@ type FullNode interface {
// StateVerifiedClientStatus returns the data cap for the given address.
// Returns nil if there is no entry in the data cap table for the
// address.
StateVerifiedClientStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*verifreg.DataCap, error)
StateVerifiedClientStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error)
// StateDealProviderCollateralBounds returns the min and max collateral a storage provider
// can issue. It takes the deal size and verified status as parameters.
StateDealProviderCollateralBounds(context.Context, abi.PaddedPieceSize, bool, types.TipSetKey) (DealCollateralBounds, error)

// StateCirculatingSupply returns the circulating supply of Filecoin at the given tipset
StateCirculatingSupply(context.Context, types.TipSetKey) (CirculatingSupply, error)
// StateNetworkVersion returns the network version at the given tipset
StateNetworkVersion(context.Context, types.TipSetKey) (network.Version, error)

// MethodGroup: Msig
// The Msig methods are used to interact with multisig wallets on the
Expand Down Expand Up @@ -476,21 +475,12 @@ type FileRef struct {
}

type MinerSectors struct {
Sectors uint64
Active uint64
}

type SectorExpiration struct {
OnTime abi.ChainEpoch

// non-zero if sector is faulty, epoch at which it will be permanently
// removed if it doesn't recover
Early abi.ChainEpoch
}

type SectorLocation struct {
Deadline uint64
Partition uint64
// Live sectors that should be proven.
Live uint64
// Sectors actively contributing to power.
Active uint64
// Sectors with failed proofs.
Faulty uint64
}

type ImportRes struct {
Expand Down Expand Up @@ -556,11 +546,6 @@ type Message struct {
Message *types.Message
}

type ChainSectorInfo struct {
Info miner.SectorOnChainInfo
ID abi.SectorNumber
}

type ActorState struct {
Balance types.BigInt
State interface{}
Expand Down Expand Up @@ -632,8 +617,9 @@ type VoucherCreateResult struct {
}

type MinerPower struct {
MinerPower power.Claim
TotalPower power.Claim
MinerPower power.Claim
TotalPower power.Claim
HasMinPower bool
}

type QueryOffer struct {
Expand Down Expand Up @@ -802,7 +788,7 @@ type CirculatingSupply struct {
type MiningBaseInfo struct {
MinerPower types.BigInt
NetworkPower types.BigInt
Sectors []proof.SectorInfo
Sectors []builtin.SectorInfo
WorkerKey address.Address
SectorSize abi.SectorSize
PrevBeaconEntry types.BeaconEntry
Expand All @@ -819,7 +805,7 @@ type BlockTemplate struct {
Messages []*types.SignedMessage
Epoch abi.ChainEpoch
Timestamp uint64
WinningPoStProof []proof.PoStProof
WinningPoStProof []builtin.PoStProof
}

type DataSize struct {
Expand All @@ -843,6 +829,18 @@ const (
MsigCancel
)

type Deadline struct {
PostSubmissions bitfield.BitField
}

type Partition struct {
AllSectors bitfield.BitField
FaultySectors bitfield.BitField
RecoveringSectors bitfield.BitField
LiveSectors bitfield.BitField
ActiveSectors bitfield.BitField
}

type Fault struct {
Miner address.Address
Epoch abi.ChainEpoch
Expand Down
4 changes: 2 additions & 2 deletions api/api_storage.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ type StorageMiner interface {
stores.SectorIndex

MarketImportDealData(ctx context.Context, propcid cid.Cid, path string) error
MarketListDeals(ctx context.Context) ([]storagemarket.StorageDeal, error)
MarketListDeals(ctx context.Context) ([]MarketDeal, error)
MarketListRetrievalDeals(ctx context.Context) ([]retrievalmarket.ProviderDealState, error)
MarketGetDealUpdates(ctx context.Context) (<-chan storagemarket.MinerDeal, error)
MarketListIncompleteDeals(ctx context.Context) ([]storagemarket.MinerDeal, error)
Expand All @@ -83,7 +83,7 @@ type StorageMiner interface {
MarketDataTransferUpdates(ctx context.Context) (<-chan DataTransferChannel, error)

DealsImportData(ctx context.Context, dealPropCid cid.Cid, file string) error
DealsList(ctx context.Context) ([]storagemarket.StorageDeal, error)
DealsList(ctx context.Context) ([]MarketDeal, error)
DealsConsiderOnlineStorageDeals(context.Context) (bool, error)
DealsSetConsiderOnlineStorageDeals(context.Context, bool) error
DealsConsiderOnlineRetrievalDeals(context.Context) (bool, error)
Expand Down
69 changes: 69 additions & 0 deletions api/api_test.go
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
package api

import (
"encoding/json"
"os"
"os/exec"
"path/filepath"
"reflect"
"runtime"
"strings"
"testing"

"github.com/stretchr/testify/require"
)

func goCmd() string {
Expand All @@ -32,3 +36,68 @@ func TestDoesntDependOnFFI(t *testing.T) {
}
}
}

func TestReturnTypes(t *testing.T) {
errType := reflect.TypeOf(new(error)).Elem()
bareIface := reflect.TypeOf(new(interface{})).Elem()
jmarsh := reflect.TypeOf(new(json.Marshaler)).Elem()

tst := func(api interface{}) func(t *testing.T) {
return func(t *testing.T) {
ra := reflect.TypeOf(api).Elem()
for i := 0; i < ra.NumMethod(); i++ {
m := ra.Method(i)
switch m.Type.NumOut() {
case 1: // if 1 return value, it must be an error
require.Equal(t, errType, m.Type.Out(0), m.Name)

case 2: // if 2 return values, first cant be an interface/function, second must be an error
seen := map[reflect.Type]struct{}{}
todo := []reflect.Type{m.Type.Out(0)}
for len(todo) > 0 {
typ := todo[len(todo)-1]
todo = todo[:len(todo)-1]

if _, ok := seen[typ]; ok {
continue
}
seen[typ] = struct{}{}

if typ.Kind() == reflect.Interface && typ != bareIface && !typ.Implements(jmarsh) {
t.Error("methods can't return interfaces", m.Name)
}

switch typ.Kind() {
case reflect.Ptr:
fallthrough
case reflect.Array:
fallthrough
case reflect.Slice:
fallthrough
case reflect.Chan:
todo = append(todo, typ.Elem())
case reflect.Map:
todo = append(todo, typ.Elem())
todo = append(todo, typ.Key())
case reflect.Struct:
for i := 0; i < typ.NumField(); i++ {
todo = append(todo, typ.Field(i).Type)
}
}
}

require.NotEqual(t, reflect.Func.String(), m.Type.Out(0).Kind().String(), m.Name)
require.Equal(t, errType, m.Type.Out(1), m.Name)

default:
t.Error("methods can only have 1 or 2 return values", m.Name)
}
}
}
}

t.Run("common", tst(new(Common)))
t.Run("full", tst(new(FullNode)))
t.Run("miner", tst(new(StorageMiner)))
t.Run("worker", tst(new(WorkerAPI)))
}
Loading