Missing data via SectorInfo, V7SectorInfo, SectorDeals, and MinerLockedFunds Extractors.

[frrist]

Describe the bug:

The following extractors contain a bug that causes them to miss data while extracting their respective models:

• V7SectorInfoExtractor: Extracts the latest state of miner sectors for all miners using actor version v7-vN forward (includes snapped sectors)
  • Code section with bug
  • Model effected: miner_sector_infos_v7
  • Fix: Unify dev and CI linting #990
• SectorInfoExtractor: Extracts the latest state of miner sectors for all miners using actor version v0-v6 (doesn’t include snapped sectors)
  • Code section with bug
  • Model effected: miner_sector_infos
  • Fix: Unify dev and CI linting #990
• SectorDealsExtractor: Extracts a mapping of Deal IDs to their respective Miner and Sector IDs.
  • Code section with bug
  • Model effected: miner_sector_deals
  • Fix: Unify dev and CI linting #990
• MinerLockedFundsExtractor: Extractors details of Miner funds locked and unavailable for use.
  • [Code Section with bug](https://github.com/filecoin-project/lily/blob/v0.10.0/chain/indexer/integrated/tipset.go#L113-
    L116)
  • Model effected: miner_locked_funds
  • Fix: fix: register MinerLockedFunds extractor #996

These bugs causes the models miner_sector_infos, miner_sector_infos_v7, and miner_sector_deals to undercount data and miner_locked_funds to miss all data, thus queries over these tables will not accurately represent the state of the chain. The models miner_sector_infos and miner_sector_infos_v7 record the state of newly added sectors:

• Height and StateRoot a sector was added
• Miner adding the sector and ID of the sector
• Sectors SealedCID, ActivationEpoch, ExpirationEpoch, DealWeight, InitialPledge, ExpectedDayReward, ExpectedStoragePledge, and if the sector contains verified data

The model miner_sector_deals records which deals belong to which sectors:

• Height deal was added to a sector
• Miner adding sector and its ID
• DealID’s contained in the sector.

The model miner_locked_funds records details of miner funds locked for adding sectors:

• Height funds were changed
• MinerID, initial pledge, pre-commit deposit, and locked funds.

It is worth pointing out that the model miner_sector_events is not affected by this and aggregations over its data will still return accurate counts of the different miner sector events (sectors added, snapped, faulted, recovered, etc.)

Lily Version: v0.9.0 - v0.10.0

This bug affects lily v0.9.0 and v0.10.0

TODO:

• create lily v0.10.1-rc1: https://github.com/filecoin-project/lily/tree/v0.10.1-rc1
• Deploy lily v0.10.1-rc1 to staging: https://github.com/filecoin-project/sentinel-infra/pull/282
• Verify staging deployment is producing data in miner_sector_infos_v7, miner_sector_deals and miner_locked_funds that is no contained in production
• Release lily v0.10.1 with fix.
• Backfill analysis database from April 29th forward. tracking: https://github.com/filecoin-project/sentinel-backfill/issues/42
• Produce lily data exports for affected tables from April 29th forward. tracking: https://github.com/filecoin-project/sentinel-backfill/issues/42

[frrist]

Release candidate created here: https://github.com/filecoin-project/lily/tree/v0.10.1-rc1