Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ui): Add a terminal-based progress bar to Zebra #6235

Merged
merged 49 commits into from
Apr 13, 2023
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
098fd85
Implement Display and to_string() for NetworkUpgrade
teor2345 Feb 27, 2023
660916e
Add a progress-bar feature to zebrad
teor2345 Feb 27, 2023
3060100
Add the progress bar writer to the tracing component
teor2345 Feb 27, 2023
5894e20
Add a block progress bar transmitter
teor2345 Feb 27, 2023
c6faa66
Correctly shut down the progress bar, and shut it down on an interrupt
teor2345 Feb 27, 2023
0d5b999
Make it clearer that the progress task never exits
teor2345 Mar 2, 2023
12960b8
Add a config for writing logs to a file
teor2345 Feb 27, 2023
1d9e0fa
Add a progress-bar feature to zebra-network
teor2345 Feb 28, 2023
f363419
Add a progress bar for the address book size
teor2345 Feb 28, 2023
381cd22
Add progress bars for never attempted and failed peers
teor2345 Feb 28, 2023
c6aa3a9
Add an optional limit and label to connection counters
teor2345 Feb 28, 2023
3f9eabc
Add open connection progress bars
teor2345 Feb 28, 2023
ce05970
Improve CheckpointList API and CheckpointVerifier debugging
teor2345 Feb 28, 2023
68afb65
Add checkpoint index and checkpoint queue progress bars
teor2345 Feb 28, 2023
970a7ec
Security: Limit the number of non-finalized chains tracked by Zebra
teor2345 Feb 28, 2023
d96b107
Make some NonFinalizedState methods available with proptest-impl
teor2345 Feb 28, 2023
647f6a1
Add a non-finalized chain count progress bar
teor2345 Feb 28, 2023
ad9e739
Track the last fork height for newly forked chains
teor2345 Mar 2, 2023
b6f4c3e
Add a should_count_metrics to Chain
teor2345 Mar 2, 2023
2bb6ba5
Add a display method for PartialCumulativeWork
teor2345 Mar 2, 2023
cdbf801
Add a progress bar for each chain fork
teor2345 Mar 2, 2023
a60c936
Add a NonFinalizedState::disable_metrics() method and switch to using it
teor2345 Mar 2, 2023
4cdcc9e
Move metrics out of Chain because we can't update Arc<Chain>
teor2345 Mar 2, 2023
db8d52e
Fix: consistently use best chain order when searching chains
teor2345 Mar 2, 2023
4619a84
Track Chain progress bars in NonFinalizedState
teor2345 Mar 2, 2023
31de093
Display work as bits, not a multiple of the target difficulty
teor2345 Mar 2, 2023
fa32eaf
Handle negative fork lengths by reporting "No fork"
teor2345 Mar 30, 2023
47ec6c2
Correctly disable unused fork bars
teor2345 Apr 2, 2023
92944de
clippy: rewrite using `match _.cmp(_) { ... }`
teor2345 Apr 2, 2023
c9439a8
Initial mempool progress bar implementation
teor2345 Mar 2, 2023
42d6f30
Update Cargo.lock
teor2345 Mar 30, 2023
21cfa09
Add the actual transaction size as a description to the cost bar
teor2345 Mar 31, 2023
0b3e145
Only show mempool progress bars after first activation
teor2345 Mar 31, 2023
f78d37c
Add queued and rejected mempool progress bars
teor2345 Mar 31, 2023
1aa9883
Clarify cost note is actual size
teor2345 Mar 31, 2023
331b55d
Add tracing.log_file config and progress-bar feature to zebrad docs
teor2345 Apr 3, 2023
993d0a5
Derive Clone for Chain
teor2345 Apr 4, 2023
54cd891
Upgrade to howudoin 0.1.2 and remove some bug workarounds
teor2345 Apr 4, 2023
fba9d4a
Directly call the debug formatter to Display a Network
teor2345 Apr 13, 2023
4f815cd
Rename the address count metric to num_addresses
teor2345 Apr 13, 2023
5af9b3d
Simplify reverse checkpoint lookup
teor2345 Apr 13, 2023
d846062
Simplify progress bar shutdown code
teor2345 Apr 13, 2023
5557214
Remove unused MIN_TRANSPARENT_TX_MEMPOOL_SIZE
teor2345 Apr 13, 2023
edf4948
Document that the progress task runs forever
teor2345 Apr 13, 2023
04fe2e7
Fix progress log formatting
teor2345 Apr 13, 2023
cf6d407
If progress-bar is on, log to a file by default
teor2345 Apr 13, 2023
fbeb79a
Create missing directories for log files
teor2345 Apr 13, 2023
1eecebe
Add file security docs for running Zebra with elevated permissions
teor2345 Apr 13, 2023
08a8c64
Document automatic log file, spell progress-bar correctly
teor2345 Apr 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 69 additions & 1 deletion Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -806,6 +806,7 @@ dependencies = [
"encode_unicode",
"lazy_static",
"libc",
"unicode-width",
"windows-sys 0.42.0",
]

Expand Down Expand Up @@ -1456,6 +1457,19 @@ dependencies = [
"miniz_oxide",
]

[[package]]
name = "flume"
version = "0.10.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1657b4441c3403d9f7b3409e47575237dac27b1b5726df654a6ecbf92f0f7577"
dependencies = [
"futures-core",
"futures-sink",
"nanorand",
"pin-project",
"spin 0.9.7",
]

[[package]]
name = "fnv"
version = "1.0.7"
Expand Down Expand Up @@ -1632,8 +1646,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c05aeb6a22b8f62540c194aac980f2115af067bfe15a0734d7277a768d396b31"
dependencies = [
"cfg-if 1.0.0",
"js-sys",
"libc",
"wasi 0.11.0+wasi-snapshot-preview1",
"wasm-bindgen",
]

[[package]]
Expand Down Expand Up @@ -1890,6 +1906,17 @@ dependencies = [
"winapi",
]

[[package]]
name = "howudoin"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f34059280f617a59ee59a0455e93460d67e5c76dec42dd262d38f0f390f437b2"
dependencies = [
"flume",
"indicatif",
"parking_lot 0.12.1",
]

[[package]]
name = "http"
version = "0.2.9"
Expand Down Expand Up @@ -2088,6 +2115,18 @@ dependencies = [
"serde",
]

[[package]]
name = "indicatif"
version = "0.17.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cef509aa9bc73864d6756f0d34d35504af3cf0844373afe9b8669a5b8005a729"
dependencies = [
"console",
"number_prefix",
"portable-atomic",
"unicode-width",
]

[[package]]
name = "inferno"
version = "0.11.15"
Expand Down Expand Up @@ -2604,6 +2643,15 @@ version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5ce46fe64a9d73be07dcbe690a38ce1b293be448fd8ce1e6c1b8062c9f72c6a"

[[package]]
name = "nanorand"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a51313c5820b0b02bd422f4b44776fbf47961755c74ce64afc73bfad10226c3"
dependencies = [
"getrandom 0.2.8",
]

[[package]]
name = "native-tls"
version = "0.2.11"
Expand Down Expand Up @@ -2710,6 +2758,12 @@ dependencies = [
"libc",
]

[[package]]
name = "number_prefix"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "830b246a0e5f20af87141b25c173cd1b609bd7779a4617d6ec582abaf90870f3"

[[package]]
name = "object"
version = "0.30.3"
Expand Down Expand Up @@ -3676,7 +3730,7 @@ dependencies = [
"cc",
"libc",
"once_cell",
"spin",
"spin 0.5.2",
"untrusted",
"web-sys",
"winapi",
Expand Down Expand Up @@ -4273,6 +4327,15 @@ version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e63cff320ae2c57904679ba7cb63280a3dc4613885beafb148ee7bf9aa9042d"

[[package]]
name = "spin"
version = "0.9.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0959fd6f767df20b231736396e4f602171e00d95205676286e79d4a4eb67bef"
dependencies = [
"lock_api",
]

[[package]]
name = "stable_deref_trait"
version = "1.2.0"
Expand Down Expand Up @@ -5728,6 +5791,7 @@ dependencies = [
"futures-util",
"halo2_proofs",
"hex",
"howudoin",
"jubjub 0.9.0",
"lazy_static",
"metrics",
Expand Down Expand Up @@ -5767,6 +5831,7 @@ dependencies = [
"chrono",
"futures",
"hex",
"howudoin",
"humantime-serde",
"indexmap",
"lazy_static",
Expand Down Expand Up @@ -5857,6 +5922,7 @@ dependencies = [
"futures",
"halo2_proofs",
"hex",
"howudoin",
"indexmap",
"insta",
"itertools",
Expand Down Expand Up @@ -5944,9 +6010,11 @@ dependencies = [
"futures",
"gumdrop",
"hex",
"howudoin",
"humantime-serde",
"hyper",
"indexmap",
"indicatif",
"inferno",
"jsonrpc-core",
"lazy_static",
Expand Down
7 changes: 7 additions & 0 deletions zebra-chain/src/parameters/network_upgrade.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,13 @@ pub enum NetworkUpgrade {
Nu5,
}

impl fmt::Display for NetworkUpgrade {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
// Same as the debug representation for now
fmt::Debug::fmt(self, f)
}
}

/// Mainnet network upgrade activation heights.
///
/// This is actually a bijective map, but it is const, so we use a vector, and
Expand Down
7 changes: 5 additions & 2 deletions zebra-chain/src/transaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,12 @@ pub use joinsplit::JoinSplitData;
pub use lock_time::LockTime;
pub use memo::Memo;
pub use sapling::FieldNotPresent;
pub use serialize::SerializedTransaction;
pub use serialize::{
SerializedTransaction, MIN_TRANSPARENT_TX_SIZE, MIN_TRANSPARENT_TX_V4_SIZE,
MIN_TRANSPARENT_TX_V5_SIZE,
};
pub use sighash::{HashType, SigHash};
pub use unmined::{UnminedTx, UnminedTxId, VerifiedUnminedTx};
pub use unmined::{UnminedTx, UnminedTxId, VerifiedUnminedTx, MEMPOOL_TRANSACTION_COST_THRESHOLD};

use crate::{
amount::{Amount, Error as AmountError, NegativeAllowed, NonNegative},
Expand Down
12 changes: 11 additions & 1 deletion zebra-chain/src/transaction/serialize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -962,9 +962,19 @@ pub(crate) const MIN_TRANSPARENT_OUTPUT_SIZE: u64 = 8 + 1;
///
/// Shielded transfers are much larger than transparent transfers,
/// so this is the minimum transaction size.
pub(crate) const MIN_TRANSPARENT_TX_SIZE: u64 =
pub const MIN_TRANSPARENT_TX_SIZE: u64 =
MIN_TRANSPARENT_INPUT_SIZE + 4 + MIN_TRANSPARENT_OUTPUT_SIZE;

/// The minimum transaction size for v4 transactions.
///
/// v4 transactions also have an expiry height.
pub const MIN_TRANSPARENT_TX_V4_SIZE: u64 = MIN_TRANSPARENT_TX_SIZE + 4;

/// The minimum transaction size for v5 transactions.
///
/// v5 transactions also have an expiry height and a consensus branch ID.
pub const MIN_TRANSPARENT_TX_V5_SIZE: u64 = MIN_TRANSPARENT_TX_SIZE + 4 + 4;

/// No valid Zcash message contains more transactions than can fit in a single block
///
/// `tx` messages contain a single transaction, and `block` messages are limited to the maximum
Expand Down
2 changes: 1 addition & 1 deletion zebra-chain/src/transaction/unmined.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ mod zip317;
/// > transparent transactions because of their size.
///
/// [ZIP-401]: https://zips.z.cash/zip-0401
const MEMPOOL_TRANSACTION_COST_THRESHOLD: u64 = 4000;
pub const MEMPOOL_TRANSACTION_COST_THRESHOLD: u64 = 4000;

/// When a transaction pays a fee less than the conventional fee,
/// this low fee penalty is added to its cost for mempool eviction.
Expand Down
28 changes: 28 additions & 0 deletions zebra-chain/src/work/difficulty.rs
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,34 @@ impl PartialCumulativeWork {
pub fn as_u128(self) -> u128 {
self.0
}

/// Returns a floating-point work multiplier that can be used for display.
/// The returned value is the work as a multiple of the target difficulty limit for `network`.
pub fn difficulty_multiplier_for_display(&self, network: Network) -> f64 {
// This calculation is similar to the `getdifficulty` RPC, see that code for details.

let pow_limit = ExpandedDifficulty::target_difficulty_limit(network)
.to_compact()
.to_work()
.expect("target difficult limit is valid work");

// Convert to u128 then f64.
let pow_limit = pow_limit.as_u128() as f64;
let work = self.as_u128() as f64;

work / pow_limit
}

/// Returns floating-point work bits that can be used for display.
/// The returned value is the number of hash bits represented by the work.
pub fn difficulty_bits_for_display(&self) -> f64 {
// This calculation is similar to `zcashd`'s bits display in its logs.

// Convert to u128 then f64.
let work = self.as_u128() as f64;

work.log2()
}
}

impl From<Work> for PartialCumulativeWork {
Expand Down
8 changes: 8 additions & 0 deletions zebra-consensus/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ default = []

# Production features that activate extra dependencies, or extra features in dependencies

progress-bar = [
"howudoin",
"zebra-state/progress-bar",
]

# Experimental mining RPC support
getblocktemplate-rpcs = [
"zebra-state/getblocktemplate-rpcs",
Expand Down Expand Up @@ -56,6 +61,9 @@ zebra-state = { path = "../zebra-state" }
zebra-node-services = { path = "../zebra-node-services" }
zebra-chain = { path = "../zebra-chain" }

# prod feature progress-bar
howudoin = { version = "0.1.2", optional = true }

# Test-only dependencies
proptest = { version = "1.1.0", optional = true }
proptest-derive = { version = "0.3.0", optional = true }
Expand Down
Loading