diff --git a/README.md b/README.md index 19ba884e..26688a59 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,49 @@ # SimLN -SimLN is a simulation tool that can be used to generate realistic -payment activity on any lightning network topology. It is intentionally -environment-agnostic so that it can be used across many environments - -from integration tests to public signets. +SimLN is a simulation tool that can be used to generate realistic +payment activity on any lightning network topology. It is intentionally +environment-agnostic so that it can be used across many environments - +from integration tests to public signets. -This tool is intended to serve developers who are familiar with +This tool is intended to serve developers who are familiar with lightning network development. It may be useful to you if you are: -* A protocol developer looking to test proposals. -* An application developer load-testing your application. -* A signet operator interested in a hands-off way to run an active node. -* A researcher generating synthetic data for a target topology. + +- A protocol developer looking to test proposals. +- An application developer load-testing your application. +- A signet operator interested in a hands-off way to run an active node. +- A researcher generating synthetic data for a target topology. ## Pre-Requisites -SimLN requires you to "bring your own network" to generate activity + +SimLN requires you to "bring your own network" to generate activity on. You will need: -* A [lightning network](#lightning-environments) connected with any + +- A [lightning network](#lightning-environments) connected with any topology of channels. -* Access to execute commands on _at least_ one node in the network. -* Rust compiler [installed](https://www.rust-lang.org/tools/install). +- Access to execute commands on _at least_ one node in the network. +- Rust compiler [installed](https://www.rust-lang.org/tools/install). ## LN Implementation Support -* LND ✅ -* CLN ✅ -* Eclair 🏗️ -* LDK-node 🏗️ + +- LND ✅ +- CLN ✅ +- Eclair 🏗️ +- LDK-node 🏗️ See our [tracking issue](https://github.com/bitcoin-dev-project/sim-ln/issues/26) for updates on implementation support (contributions welcome!). ## Getting Started -Clone the repo: + +Clone the repo: + ``` git clone https://github.com/bitcoin-dev-project/sim-ln cd sim-ln ``` -Install the CLI: +Install the CLI: + ``` cargo install --locked --path sim-cli/ ``` @@ -52,9 +59,10 @@ Run `sim-cli -h` for details on `--data-dir` and `--sim-file` options that allow Interested in contributing to the project? See [CONTRIBUTING](CONTRIBUTING.md) for more details. ### Simulation File Setup -The simulator requires access details for a set of `nodes` that you -have permission to execute commands on. Note that the current version -of the simulator uses keysend to execute payments, which must be + +The simulator requires access details for a set of `nodes` that you +have permission to execute commands on. Note that the current version +of the simulator uses keysend to execute payments, which must be enabled in LND using `--accept-keysend` (for CLN node it is enabled by default). The required access details will depend on the node implementation. For LND, the following @@ -72,7 +80,7 @@ information is required: Whereas for CLN nodes, the following information is required: ``` -{ +{ "id": , "address": https://, "ca_cert": , @@ -84,17 +92,18 @@ Whereas for CLN nodes, the following information is required: **Note that node addresses must be declare with HTTPS transport, i.e. ** Payment activity can be simulated in two different ways: -* Random activity: generate random activity on the `nodes` provided, + +- Random activity: generate random activity on the `nodes` provided, using the graph topology to determine payment frequency and size. -* Defined activity: provide descriptions of specific payments that +- Defined activity: provide descriptions of specific payments that you would like the generator to execute. ### Setup - Random Activity -To run the simulator with random activity generation, you just need to -provide a set of nodes and the simulator will generate activity based -on the topology of the underlying graph. Note that payments will only -be sent between the `nodes` that are provided so that liquidity does +To run the simulator with random activity generation, you just need to +provide a set of nodes and the simulator will generate activity based +on the topology of the underlying graph. Note that payments will only +be sent between the `nodes` that are provided so that liquidity does not "drain" from the simulation. ``` @@ -106,7 +115,7 @@ not "drain" from the simulation. "macaroon": "/path/admin.macaroon", "cert": "/path/tls.cert" }, - { + { "id": "0230a16a05c5ca120136b3a770a2adfdad88a68d526e63448a9eef88bddd6a30d8", "address": "https://localhost:10013", "ca_cert": "/path/ca.pem", @@ -118,38 +127,41 @@ not "drain" from the simulation. ``` Nodes can be identified by an arbitrary string ("Alice", "CLN1", etc) or -by their node public key. If a valid public key is provided it *must* +by their node public key. If a valid public key is provided it _must_ match the public key reported by the node. -There are a few cli flags that can be used to toggle the characteristics -of the random activity that is generated: -* `--expected-payment-amount`: the approximate average amount that +There are a few cli flags that can be used to toggle the characteristics +of the random activity that is generated: + +- `--expected-payment-amount`: the approximate average amount that will be sent by nodes, randomness will be introduced such that larger nodes send a wider variety of payment sizes around this expectation. -* `--capacity-multiplier`: the number of times over that each node in +- `--capacity-multiplier`: the number of times over that each node in the network sends their capacity in a calendar month, for example: - * `capacity-multiplier=2` means that each node sends double their - capacity in a month. - * `capacity-multiplier=0.5` means that each node sends half their + - `capacity-multiplier=2` means that each node sends double their + capacity in a month. + - `capacity-multiplier=0.5` means that each node sends half their capacity in a month. ### Setup - Defined Activity -If you would like SimLN to generate a specific payments between source -and destination nodes, you can provide `activity` descriptions of the -source, destination, frequency, amount for payments that you'd like -to execute and an optional name for each `activity` description that is used -as the logging prefix to relate logs to their corresponding activity description -(The index of each `activity` description will be used if a name is not -provided for the `activity` description). Note that `source` nodes *must* be contained in `nodes`, -but destination nodes can be any public node in the network (though -this may result in liquidity draining over time). + +If you would like SimLN to generate a specific payments between source +and destination nodes, you can provide `activity` descriptions of the +source, destination, frequency and amount for payments that you'd like +to execute. Note that `source` nodes _must_ be contained in `nodes`, +but destination nodes can be any public node in the network (though +this may result in liquidity draining over time). You can also add an optional name for each `activity` description that is +used as the logging prefix to relate logs to their corresponding activity +description (The index of each `activity` description will be used if a name is not provided for the `activity` description). The example simulation file below sets up the following simulation: -* Connect to `Alice` running LND to generate activity. -* Connect to `Bob` running CLN to generate activity. -* Dispatch 2000 msat payments from `Alice` to `Carol` every 1 seconds. -* Dispatch 140000 msat payments from `Bob` to `Alice` every 50 seconds. -* Dispatch 1000 msat payments from `Bob` to `Dave` every 2 seconds. + +- Connect to `Alice` running LND to generate activity. +- Connect to `Bob` running CLN to generate activity. +- Dispatch 2000 msat payments from `Alice` to `Carol` every 1 seconds. +- Dispatch 140000 msat payments from `Bob` to `Alice` every 50 seconds. +- Dispatch 1000 msat payments from `Bob` to `Dave` every 2 seconds. + ``` { "nodes": [ @@ -194,10 +206,10 @@ The example simulation file below sets up the following simulation: **Note that node addresses must be declare with HTTPS transport, i.e ** -Nodes can be identified by their public key or an id string (as -described above). Activity sources and destinations may reference the -`id` defined in `nodes`, but destinations that are not listed in `nodes` -*must* provide a valid public key. +Nodes can be identified by their public key or an id string (as +described above). Activity sources and destinations may reference the +`id` defined in `nodes`, but destinations that are not listed in `nodes` +_must_ provide a valid public key. ### Simulation Output @@ -208,10 +220,12 @@ Run `sim-cli -h` for details on data directory (`--data-dir`) and other options which affect how simulation outputs are persisted ## Lightning Environments + If you're looking to get started with local lightning development, we -recommend [polar](https://lightningpolar.com/). For larger deployments, -see the [Scaling Lightning](https://github.com/scaling-lightning/scaling-lightning) +recommend [polar](https://lightningpolar.com/). For larger deployments, +see the [Scaling Lightning](https://github.com/scaling-lightning/scaling-lightning) project. ## Docker + If you want to run the cli in a containerized environment, see the docker set up docs [here](./docker/README.md) diff --git a/sim-cli/src/main.rs b/sim-cli/src/main.rs index 3013d207..50dd547f 100644 --- a/sim-cli/src/main.rs +++ b/sim-cli/src/main.rs @@ -132,7 +132,7 @@ async fn main() -> anyhow::Result<()> { alias_node_map.insert(node_info.alias.clone(), node_info); } - let mut validated_activities = vec![]; + let mut validated_activities: Vec = vec![]; // Make all the activities identifiable by PK internally for act in activity.into_iter() { // We can only map aliases to nodes we control, so if either the source or destination alias @@ -182,6 +182,14 @@ async fn main() -> anyhow::Result<()> { }, }; + for validated_activity in &validated_activities { + if validated_activity.activity_name == act.activity_name { + anyhow::bail!(LightningError::ValidationError( + "Duplicate activity name is not allowed".to_string(), + )); + } + } + validated_activities.push(ActivityDefinition { source, destination, diff --git a/sim-lib/src/defined_activity.rs b/sim-lib/src/defined_activity.rs index 98bbe07e..69d380a5 100644 --- a/sim-lib/src/defined_activity.rs +++ b/sim-lib/src/defined_activity.rs @@ -7,14 +7,16 @@ pub struct DefinedPaymentActivity { destination: NodeInfo, wait: Duration, amount: u64, + activity_name: String, } impl DefinedPaymentActivity { - pub fn new(destination: NodeInfo, wait: Duration, amount: u64) -> Self { + pub fn new(destination: NodeInfo, wait: Duration, amount: u64, activity_name: String) -> Self { DefinedPaymentActivity { destination, wait, amount, + activity_name, } } } @@ -23,8 +25,8 @@ impl fmt::Display for DefinedPaymentActivity { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { write!( f, - "static payment of {} to {} every {:?}", - self.amount, self.destination, self.wait + "{} activity: static payment of {} to {} every {:?}", + self.activity_name, self.amount, self.destination, self.wait ) } } @@ -68,9 +70,14 @@ mod tests { let source = get_random_keypair(); let payment_amt = 50; + let activity_name = String::from("Coffee Purchase"); - let generator = - DefinedPaymentActivity::new(node.clone(), Duration::from_secs(60), payment_amt); + let generator = DefinedPaymentActivity::new( + node.clone(), + Duration::from_secs(60), + payment_amt, + activity_name, + ); let (dest, dest_capacity) = generator.choose_destination(source.1); assert_eq!(node.pubkey, dest.pubkey); diff --git a/sim-lib/src/lib.rs b/sim-lib/src/lib.rs index 953311c6..34abe84a 100644 --- a/sim-lib/src/lib.rs +++ b/sim-lib/src/lib.rs @@ -276,7 +276,7 @@ pub enum PaymentOutcome { } /// Describes a payment from a source node to a destination node. -#[derive(Debug, Clone, Copy, Serialize)] +#[derive(Debug, Clone, Serialize)] struct Payment { /// Pubkey of the source node dispatching the payment. source: PublicKey, @@ -284,6 +284,8 @@ struct Payment { destination: PublicKey, /// Amount of the payment in msat. amount_msat: u64, + /// Name of the the activity. + activity_name: String, /// Hash of the payment if it has been successfully dispatched. #[serde(with = "serializers::serde_option_payment_hash")] hash: Option, @@ -296,7 +298,8 @@ impl Display for Payment { fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result { write!( f, - "Payment {} dispatched at {:?} sending {} msat from {} -> {}", + "{} activity: Payment {} dispatched at {:?} sending {} msat from {} -> {}", + self.activity_name, self.hash.map(|h| hex::encode(h.0)).unwrap_or_default(), self.dispatch_time.duration_since(UNIX_EPOCH).unwrap(), self.amount_msat, @@ -312,7 +315,7 @@ impl Display for Payment { enum SimulationEvent { /// Dispatch a payment of the specified amount to the public key provided. /// Results in `SimulationOutput::SendPaymentSuccess` or `SimulationOutput::SendPaymentFailure`. - SendPayment(NodeInfo, u64), + SendPayment(NodeInfo, u64, String), } /// SimulationOutput provides the output of a simulation event. @@ -579,17 +582,18 @@ impl Simulation { // just populate with each type as configured. if !self.activity.is_empty() { for (index, description) in self.activity.iter().enumerate() { + let activity_name = match &description.activity_name { + Some(name) => name.clone(), + None => format!("Index {}", index), + }; + let activity_generator = DefinedPaymentActivity::new( description.destination.clone(), Duration::from_secs(description.interval_secs.into()), description.amount_msat, + activity_name.clone(), ); - let activity_name = match &description.activity_name { - Some(name) => name.clone(), - None => format!("index {}", index), - }; - generators.push(ExecutorKit { source_info: description.source.clone(), // Defined activities have very simple generators, so the traits required are implemented on @@ -743,7 +747,7 @@ async fn consume_events( while let Some(event) = receiver.recv().await { match event { - SimulationEvent::SendPayment(dest, amt_msat) => { + SimulationEvent::SendPayment(dest, amt_msat, activity_name) => { let mut node = node.lock().await; let mut payment = Payment { @@ -752,6 +756,7 @@ async fn consume_events( amount_msat: amt_msat, destination: dest.pubkey, dispatch_time: SystemTime::now(), + activity_name, }; let outcome = match node.send_payment(dest.pubkey, amt_msat).await { @@ -838,7 +843,7 @@ async fn produce_events { if amt == 0 { - log::debug!("{activity_name} activity => Skipping zero amount payment for {source} -> {destination}."); + log::debug!("{activity_name} activity : Skipping zero amount payment for {source} -> {destination}."); continue; } amt @@ -852,7 +857,7 @@ async fn produce_events Generated random payment: {source} -> {}: {amount} msat.", destination); // Send the payment, exiting if we can no longer send to the consumer. - let event = SimulationEvent::SendPayment(destination.clone(), amount); + let event = SimulationEvent::SendPayment(destination.clone(), amount, activity_name.clone()); if let Err(e) = sender.send(event).await { log::debug!( "{activity_name} activity => Stopped random producer for {amount}: {source} -> {destination}. Consumer error: {e}.", @@ -933,11 +938,12 @@ async fn write_payment_results( } /// PaymentResultLogger is an aggregate logger that will report on a summary of the payments that have been reported. -#[derive(Default)] +#[derive(Default, Clone)] struct PaymentResultLogger { success_payment: u64, failed_payment: u64, total_sent: u64, + activity_name: String, } impl PaymentResultLogger { @@ -954,19 +960,32 @@ impl PaymentResultLogger { } self.total_sent += details.amount_msat; + self.activity_name = details.activity_name.clone(); } } impl Display for PaymentResultLogger { fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result { let total_payments = self.success_payment + self.failed_payment; - write!( - f, - "Processed {} payments sending {} msat total with {:.2}% success rate.", - total_payments, - self.total_sent, - (self.success_payment as f64 / total_payments as f64) * 100.0 - ) + + if self.activity_name.is_empty() { + write!( + f, + "Processed {} payments sending {} msat total with {:.2}% success rate.", + total_payments, + self.total_sent, + (self.success_payment as f64 / total_payments as f64) * 100.0 + ) + } else { + write!( + f, + "{} activity: Processed {} payments sending {} msat total with {:.2}% success rate.", + self.activity_name, + total_payments, + self.total_sent, + (self.success_payment as f64 / total_payments as f64) * 100.0 + ) + } } }