Add Verifreg APIs to support tooling migration

[jennijuju]

Many ecosystem tooling has being tracking datacap activities by monitoring f05 via StateMarketDeals, instead of use the verifreg state. It has been fine given all fil+ pieces can only be onboarded via f05, however DDO will change that in the upcoming nv22. We want to help toolings to migrate their system by exposing APIs that allow them to the datacap activities, and to be more precise, all allocation/claims.

There are two options we can consider:

• Option 1: adding an API that returns all verified clients, a.k.a all the ActorIds that holds datacap (can get from f07 state https://github.com/helix-onchain/filecoin/blob/666ac555876ec60370f507f8cded17d4af2ece85/frc46_token/src/token/state.rs#L115). The get all allocations & claims for each client via StateGetAllocations & StateGetClaims.
• Option 2: adding 2 new APIs that returns all allocations/claims: StateGetAllAllocations & StateGetAllClaims but reading through and returning allocations and claims from f06. (similar to StateGetAllocations/StateGetClaims just without a given client address)

The API should also be added to gateway so that its available from API service provider side.

[rvagg]

@jennijuju @rjan90 maybe this issue should close/replace these:

• Add an API that returns all FIL+ pieces  #11249
• Market Data API for direct data onboarding #11173 (some good background context in here tho)

[willscott]

Do we have a timeline for when we can an expected API for this functionality where be formalized and when it might become available?

tooling will really want access to this without digging into verifreg state manually before NV22 or we risk disruption of dashboards. We need some time to update things on top of it as well.

[rvagg]

I'll take a look into this next week as I continue work on related things.

tooling will really want access to this without digging into verifreg state manually

Can you give examples of the types of access that'll be needed? From what I understand, this kind of information would already be collected and stored in queryable form by Lily, and consumers of that information don't seem to want to be querying Lotus directly for additional information. I wouldn't mind finding direct consumers of Lotus APIs and understand their current workflows to figure out what appropriate augmentation for DDO looks like. This has been one of the challenges to date—finding users that want queryable APIs for this stuff and not just "store messages/receipts in my own format and query later" - or a variation "download relevant state snapshot as a json file from glif and query it in my own way".

[willscott]

the glif snapshots of marketdeals and consumers will need some way to get verifreg state. in e.g https://datacapstats.io/ there are lots of dashboards currently built on top of an API built currently on those market deals but which needs to change to use verifreg data. Like many such views, it wants to see new data and is calculating an integration over time of usage and behavior.

[jennijuju]

Chatted Julien from CID gravity they prefer option 2.

[rvagg]

@willscott @s0nik42 would you care to give some feedback for @LexLuthr in #11631 implementing option 2.