Skip to content

v1.5.1
Compare
Choose a tag to compare
@arajasek arajasek released this 11 Mar 00:13
· 10912 commits to master since this release
v1.5.1
3fbfb7a
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

REGRESSION NOTE

This release has a regression which can affect miner and some client nodes, making it impossible to restart the node, producing an error message like this on startup.

ERROR: creating node: starting node: could not build arguments for function [....] opening backupds: opening log: reading backup part of the logfile: unmarshaling log entry: expected byte array

This is caused by the node creating a corrupted metadata write log. If you encounter this error:

  • Remove files in .lotus[miner]/kvlog/metadata/ directory
  • Optionally, to avoid this issue from reoccurring, update to v1.5.3-rc1
  • Restart the node

Release announcement

This is an optional release of Lotus that introduces an important fix to the WindowPoSt computation process. The change is to wait for some confidence before drawing beacon randomness for the proof. Without this, invalid proofs might be generated as the result of a null tipset.

Splitstore

This release also introduces the splitstore, a new optional blockstore that segregates the monolithic blockstore into cold and hot regions. The hot region contains objects from the last 4-5 finalities plus all reachable objects from two finalities away. All other objects are moved to the cold region using a compaction process that executes every finality, once 5 finalities have elapsed.

The splitstore allows us to separate the two regions quite effectively, using two separate badger blockstores. The separation
means that the live working set is much smaller, which results in potentially significant performance improvements. In addition, it means that the coldstore can be moved to a separate (bigger, slower, cheaper) disk without loss of performance.

The design also allows us to use different implementations for the two blockstores; for example, an append-only blockstore could be used for coldstore and a faster memory mapped blockstore could be used for the hotstore (eg LMDB). We plan to experiment with these options in the future.

Once the splitstore has been enabled, the existing monolithic blockstore becomes the coldstore. On the first head change notification, the splitstore will warm up the hotstore by copying all reachable objects from the current tipset into the hotstore. All new writes go into the hotstore, with the splitstore tracking the write epoch. Once 5 finalities have elapsed, and every finality thereafter, the splitstore compacts by moving cold objects into the coldstore. There is also experimental support for garbage collection, whereby unreachable objects are simply discarded.

To enable the splitstore, add the following to config.toml:

[Chainstore]
  EnableSplitstore = true

Highlights

Other highlights include:

  • Improved deal data handling - now multiple deals can be adding to sectors in parallel
  • Rewriten sector pledging - it now actually cares about max sealing sector limits
  • Better handling for sectors stuck in the RecoverDealIDs state
  • lotus-miner sectors extend command
  • Optional configurable storage path size limit
  • Config to disable owner/worker fallback from control addresses (useful when owner is a key on a hardware wallet)
  • A write log for node metadata, which can be restored as a backup when the metadata leveldb becomes corrupted (e.g. when you run out of disk space / system crashes in some bad way)

Changes

  • avoid use mp.cfg directly to avoid race (#5350)
  • Show replacing message CID is state search-msg cli (#5656)
  • Fix riceing by importing the main package (#5675)
  • Remove sectors with all deals expired in RecoverDealIDs (#5658)
  • storagefsm: Rewrite input handling (#5375)
  • reintroduce Refactor send command for better testability (#5668)
  • Improve error message with importing a chain (#5669)
  • storagefsm: Cleanup CC sector creation (#5612)
  • chain list --gas-stats display capacity (#5676)
  • Correct some logs (#5694)
  • refactor blockstores (#5484)
  • Add idle to sync stage's String() (#5702)
  • packer provisioner (#5604)
  • add DeleteMany to Blockstore interface (#5703)
  • segregate chain and state blockstores (#5695)
  • fix(multisig): The format of the amount is not correct in msigLockApp (#5718)
  • Update butterfly network (#5627)
  • Collect worker task metrics (#5648)
  • Correctly format disputer log (#5716)
  • Log block CID in the large delay warning (#5704)
  • Move api client builders to a cliutil package (#5728)
  • Implement net peers --extended (#5734)
  • Command to extend sector expiration (#5666)
  • garbage collect hotstore after compaction (#5744)
  • tune badger gc to repeatedly gc the value log until there is no rewrite (#5745)
  • Add configuration option for pubsub IPColocationWhitelist subnets (#5735)
  • hot/cold blockstore segregation (aka. splitstore) (#4992)
  • Customize verifreg root key and remainder account when making genesis (#5730)
  • chore: update go-graphsync to 0.6.0 (#5746)
  • Add connmgr metadata to NetPeerInfo (#5749)
  • test: attempt to make the splitstore test deterministic (#5750)
  • Feat/api no dep build (#5729)
  • Fix bootstrapper profile setting (#5756)
  • Check liveness of sectors when processing termination batches (#5759)
  • Configurable storage path storage limit (#5624)
  • miner: Config to disable owner/worker address fallback (#5620)
  • Fix TestUnpadReader on Go 1.16 (#5761)
  • Metadata datastore log (#5755)
  • Remove the SR2 stats, leave just the network totals (#5757)
  • fix: wait a bit before starting to compute window post proofs (#5764)
  • fix: retry proof when randomness changes (#5768)
Assets 2
Loading