handling support and claim reorgs in leveldb

[kauffj]

leveldb claimtrie
does not touch claim_search, removes usage of claims.db

• fix blockchain.claimtrie.getclaimbyid, swap single claim searches (in tests) to use this.
• handle abandon
• store undo info
• what to do about invalid signatures? no result vs return result and say signature is invalid. resolution: clients should validate signatures
• handle same-block claims/supports
• sync chain
• PR the code
• finish rebase
• TAKEOVER/ACTIVATION HEIGHTS
• wait till victor is done with claim_search because jack's branch drops claim search completely
• confirm that app no longer uses trending fields from resolve and claim_search responses, since they have been dropped
• handle takeovers from spent supports
• feed ES claims from the block processor
• claim search tests pass
• test takeover due to a dropped support in a reorg
• test invalidated channel resolution for stream claims where the channel is dropped in a reorg
• track repost counts in ES(45min)
• make sure these make it back https://github.com/lbryio/lbry-sdk/blob/master/tests/unit/wallet/server/test_sqldb.py
• test channel state changes
  • invalidate prev claims for a channel when channel key changes
  • check existing invalid claims in chan to see if any have become valid
  • claim signature changes (adds sig, drops sig, signed by diff channel)
• blocking censored content in resolve
  • updates to plugins should let them know that update happened because censoring state changed
• write a summary of code changes in the PR to aid review
• run on jack's local server so tom can do performance and correctness testing
  • check that as reorgs happen in the wild, this server stays correct

after PR is merged:

• ship a new snapshot (no more trending data)
• update all wallet servers
• consider doing Add last uploaded date to channel metadata #3131
• have tests use claim_search and getclaimbyid, not just the latter

(later)

maybe redo this stuff inside hub instead of fetching from lbrycrd

• activation delay (calc in Add takeover_estimate to resolve response #1031) - DONT LET THIS BOG YOU DOWN IF ITS NOT SIMPLE
• add other height fields: active at, created
• compare takeover heights against lbrycrd
• compare effective amounts
• compare controlling status
• replace block prefetcher with zmq client
• replace mempool polling with zmq client
• make zmq subscriber in Go (search plugin API design #3222)
• make plugin zmq notification sender (search plugin API design #3222)

(more later)

• replace usage of claim_hash with truncated form
• proofs
  • re-enable resolve proofs
  • include proofs in resolve proto response
  • non-winning proofs (by claim_id)
• reduce boilerplate
• wrap plyvel database (e.g. electrumx storage classes)

[lyoshenka]

• ~2.5h to sync
• next: finish claim sync

[lyoshenka]

api change: drop trending fields from resolve response. app doesnt use them

[lyoshenka]

estimate: 3 days left

[lyoshenka]

• get supports done
• run reorg tests
• wind chain back and forward a bunch

est: 2 days left

[lyoshenka]

jack says it feels 80% done

[lyoshenka]

est: somewhere between "will finish today" and "unknown unknowns"

[lyoshenka]

found an unknown unknown

[lyoshenka]

est: tom can test against a build in the next 2 days

[lyoshenka]

• address tom's feedback
• ask devs in discord to try sdk.lbry.tech
• fix performance issues
• run in docker
• fix content blocking
• expose plyvel-lru cache size as env var
• convert claim_txo to be lru cache and expose size as setting
• make a snapshot
• deploy to spv16