Updating progress report 3

progress_reports/progress_report_3.md

@@ -2,23 +2,21 @@
 July 15, 2024 - October 15, 2024
 
 ## BLAST TUI
-The development of a Terminal User Interface (TUI) is a key component of our larger effort to create a useful modeling and simulation tool for the Lightning Network. A TUI is a text-based interface that allows users to interact with software applications directly through a command-line interface (CLI). This interface utilizes keyboard inputs to execute commands, configure settings, and display outputs. Unlike graphical user interfaces (GUIs), which rely on visual elements like windows and icons, TUIs provide a more streamlined and efficient interaction method, especially beneficial for users who prefer keyboard navigation or operate in environments where graphical capabilities are limited.
+The development of a Terminal User Interface (TUI) is a key component of the larger effort to create a useful modeling and simulation tool for the Lightning Network. A TUI is a text-based interface that allows users to interact with software applications directly through a terminal window. This interface utilizes keyboard inputs to execute commands, configure settings, and display outputs. Unlike graphical user interfaces (GUIs), which rely on visual elements like windows and icons, TUIs provide a more streamlined and efficient interaction method, especially beneficial for users who prefer keyboard navigation or operate in environments where graphical capabilities are limited.
 
-TUIs can be developed more rapidly than full GUIs due to several factors. Firstly, the complexity of creating visual elements—such as buttons, sliders, and menus—can significantly extend development timelines. In contrast, TUIs primarily focus on text-based inputs and outputs, which simplifies the coding process. Additionally, TUIs require less design iteration since they lack the visual design considerations that GUIs demand, allowing developers to concentrate on functionality. This made the TUI an ideal first step for our project, as it provided a usable interface quickly and effectively. By implementing a TUI, we could introduce a user-friendly interaction model that enhances accessibility and prepares the groundwork for potential future GUI development.
+TUIs can be developed more rapidly than full GUIs due to several factors. Firstly, the complexity of creating visual elements—such as buttons, sliders, and menus—can significantly extend development timelines. In contrast, TUIs primarily focus on text-based inputs and outputs, which simplifies the coding process. Additionally, TUIs require less design iteration since they lack the visual design considerations that GUIs demand, allowing developers to concentrate on functionality. This made the TUI an ideal first step for BLAST, as it provided a usable interface quickly and effectively. By implementing a TUI, a user-friendly front-end was introduced to BLAST that enhances accessibility and prepares the groundwork for potential future GUI development.
 
-By integrating a TUI into our modeling and simulation application, we aim to enhance accessibility for users who work in terminal environments, cater to advanced users who prefer command-line operations, and ultimately provide a more versatile tool for simulation management.
+By integrating a TUI into BLAST, the goal is to enhance accessibility for users who work in terminal environments, cater to advanced users who prefer command-line operations, and ultimately provide a more versatile tool for simulation management. The TUI consists of four main pages, each designed to facilitate different aspects of the simulation process:
 
-The TUI consists of four main pages, each designed to facilitate different aspects of the simulation process:
+1. **New Page:** This page allows users to create new simulations by inputting the number of each node type they would like in the simulation. This will start up the nodes and then pass control to the `Configure` page, where the user can further set up the network and simulation.
 
-1. **New Page:** This page allows users to create new simulations by inputting the number of each node type they would like in the simulation. This will start up the nodes and then pass control to the `Configure` page where the user can further set up the network and simulation.
-
-2. **Load Page:** Users can load pre-existing simulations from this page, streamlining the process of accessing and reusing previous work. After the simulation is loaded the TUI will switch over the `Configure` page where the user can edit the loaded simulation if they would like.
+2. **Load Page:** Users can load pre-existing simulations from this page, streamlining the process of accessing and reusing previous work. After the simulation is loaded the TUI will switch over the `Configure` page, where the user can edit the loaded simulation if they would like.
 
 3. **Configure Page:** On this page, users can modify various simulation parameters before execution, ensuring that each simulation can be tailored to specific needs. Users can add events and payment activity, open channels, view information about the nodes, etc. When the user is content with the simulation/network they can start the simulation and this will switch the TUI to the `Run` page.
 
 4. **Run Page:** This final page enables users to execute their simulations, providing real-time feedback and output as the simulation runs. After the simulation ends the TUI returns to the `Configure` page where the user can inspect the network, re-run the simulation, or shutdown the network and start over.
 
-Running BLAST simulations involves two parts: the network and the simulation. The network refers to the nodes and channels that make up the regtest Lightning Network. The simulation refers to the payment activity and the events. These parts are started/stopped independently, meaning the user can start up the network and make changes to the network before starting the payments and events. After the payments and events are run the simulation will stop but the network stays up so that the user can inspect the nodes and see how the payments and events impacted the network. At this point the user can either stop the network and start over or re-run the simulation on the network. This workflow promotes quick reproducibility and flexibility for the user. The BLAST TUI helps facilitate this process.
+Running BLAST simulations involves two parts: the network and the simulation. The network refers to the nodes and channels that make up the regtest Lightning Network. The simulation refers to the payment activity and the events. These parts are started and stopped independently, meaning the user can start up the network and make changes to the network before starting the payments and events. After the payments and events are run the simulation will stop but the network stays up so that the user can inspect the nodes and see how the payments and events impacted the network. At this point the user can either stop the network and start over or re-run the simulation on the network. This workflow promotes quick reproducibility and flexibility for the user. The BLAST TUI helps facilitate this process.
 
 ![](../images/tui_flow.png)
 
@@ -46,13 +44,13 @@ On the `Configure` page there is a CLI with the following commands:
 - [BLAST PR #1](https://github.com/bjohnson5/blast/pull/1)
 
 ## Runtime Statistics
-Part of creating the TUI for BLAST was implementing a new feature that displays runtime statistics of simulations. On the `Run` page of the TUI BLAST will display the number of payment attempts and the success rate of those attempts. The payment statistics were already being tracked and recorded by the payment generation project SimLN, which is a dependency of BLAST. This feature required specific functions and data to be publicly exposed by the SimLN library. The following section outlines the detailed process I undertook to achieve this integration.
+Part of creating the TUI for BLAST was implementing a new feature that displays runtime statistics of simulations. On the Run page BLAST will display the number of payment attempts and the success rate of those attempts. The payment statistics were already being tracked and recorded by the payment generation project SimLN, which is a dependency of BLAST. This feature required specific functions and data to be publicly exposed by the SimLN library. Here is the process that was executed to achieve this:
 
 ### Step 1: Identifying Requirements
 To implement the runtime stats feature in BLAST, I conducted a thorough analysis of the necessary data points and functions required from SimLN. This included identifying specific simulation metrics, such as execution time, resource usage, and performance bottlenecks.
 
 ### Step 2: Collaboration with SimLn
-Upon identifying the dependencies, I created an issue in the SimLN repository outlining the need for certain functions and data to be exposed to facilitate the integration with BLAST. This issue included detailed descriptions of the required functionalities and how they would enhance both SimLN and BLAST.
+Upon identifying the dependencies, I created an issue (Issue #196) in the SimLN repository outlining the need for certain functions and data to be exposed to facilitate the integration with BLAST. This issue included detailed descriptions of the required functionalities and how they would enhance both SimLN and BLAST.
 
 ### Step 3: Implementing Changes in SimLN
 After receiving feedback from the SimLN maintainers, I proceeded to implement the necessary changes. This involved adding new public functions to calculate and retrieve the required simulation metrics. Once the modifications were completed, I submitted a pull request for review. The changes were discussed with the SimLN team, and after addressing their comments and suggestions, the pull request was successfully merged into the main branch of SimLN.
@@ -64,7 +62,7 @@ With the new functions and data exposed in SimLN, I updated BLAST to utilize the
 - Ensuring that the runtime stats were accurately calculated and presented in a user-friendly manner.
 - Conducting thorough testing to confirm that the integration worked seamlessly and did not introduce any bugs.
 
-The successful integration of the runtime stats feature in BLAST was made possible through effective collaboration with the SimLN project and timely implementation of necessary changes. This enhancement not only improves the user experience for BLAST users but also strengthens the relationship between the two projects.
+The successful integration of the runtime stats feature in BLAST was made possible through effective collaboration with the SimLN project and timely implementation of necessary changes. This enhancement not only improves the user experience for BLAST users but also strengthens SimLn by allowing others tools to be built on top of it more easily.
 
 ### Interesting Commits
 - [SimLN PR #197](https://github.com/bitcoin-dev-project/sim-ln/pull/197)
@@ -73,13 +71,13 @@ The successful integration of the runtime stats feature in BLAST was made possib
 - [b6f9229121a7a497bf635788713c4e3e07c0e298](https://github.com/bjohnson5/blast/pull/1/commits/b6f9229121a7a497bf635788713c4e3e07c0e298)
 
 ## LND Deadlock
-During development and testing of BLAST a potential deadlock was discovered in `LND`. By creating a simulation with several nodes and channels and using SimLN to generate network traffic a mutex deadlock was found that caused an LND node to freeze up and quit processing peer messages or RPC calls. A few important points to note:
+During development and testing of BLAST a potential deadlock was discovered in LND. By creating a simulation with several nodes and channels and using SimLN to generate network traffic, a mutex deadlock was found that caused an LND node to freeze up and quit processing peer messages or RPC calls. A few important points to note:
 
 - BLAST was using `LND` nodes with the `graphCache` turned off to save memory. This is not the default configuration so this bug went unnoticed. However, if other LND users are trying to save memory they may run without the `graphCache` and are then vulnerable to this issue.
-- SimLN was vital in finding this problem because it generated enough network traffic that this problem was occuring regularly. Manually sending a few transactions from one node to another would not have triggered the deadlock. It was found because the node was in the process of finding a route for its next payment and also managing incoming messages from its peers about previous transactions. This multi-tasking was caused by the automation of BLAST and SimLN and is ultimately responsible for uncovering the bug.
-- The repeadablity that BLAST allows for is what enabled the troubleshooting of this bug and understanding what the underlying problem was. We started noticing the nodes freezing up during a BLAST simulation and so we saved the simulation and began running it again. Because the saved simulation executes the same way each time, we repeatedly were seeing the freezing issue. Debugging was possible because we could start and stop the simulation over and over again and observe what was happening in the LND code.
+- SimLN was vital in finding this problem because it generated enough network traffic that this problem was occuring regularly. Manually sending a few transactions from one node to another would not have triggered the deadlock. It was found because the node was in the process of finding a route for its next payment and also managing incoming messages from its peers about previous transactions. This multitasking was caused by the automation of BLAST and SimLN.
+- The repeatablity that BLAST allows for is what enabled the troubleshooting of this bug and understanding what the underlying problem was. I started noticing the nodes freezing up during a BLAST simulation and so I saved the simulation and began running it again. Because the saved simulation executes the same way each time, I was repeatedly seeing the freezing issue. Debugging was possible because I could start and stop the simulation over and over again and observe what was happening in the LND code.
 
-This is an excellent example of the use case for testing tools like BLAST and SimLN. Research like this will uncover edge case conditions like this particular deadlock that can then be fixed and make lightning node software more reliable and secure.
+This is an excellent example of the use case for testing tools like BLAST and SimLN. Research like this will uncover edge case conditions, like this particular deadlock, that can then be fixed and make lightning node software more reliable and secure.
 
 ### Technical Analysis
 In `lnwallet/channel.go` the `LightningChannel` struct defines several methods that the comments explain as the "state machine which corresponds to the current commitment protocol wire spec". These methods are: `SignNextCommitment`, `ReceiveNewCommitment`, `RevokeCurrentCommitment`, and `ReceiveRevocation`. Each of these will first lock the LightningChannel: `lc.lock()` and then they will typically attempt to update the channel db.
@@ -155,7 +153,8 @@ Continuing Phase III: Integrate the results of Phase I and Phase II into the BLA
     - Create a web server that will allow users to use BLAST from a browser. This will help with deployment and let BLAST be used within a docker container on operating systems other than linux.
 2.  Use LDK to build a super lightweight node that only handles forwarding payments.
     - This node can then be added to BLAST as another model that is available to the users
-    - So far we have been exploring ways to create lightweight implementation-specific nodes that are still very accurate representations of the full implementation. We created an LND node that is just a little more efficient, but is still essentially a full LND node. A more effective strategy might be to create generic simulation nodes that simply help make up a test network and then use full implementations for the nodes of interest that the user is trying test. These "generic simulation nodes" would not be very realistic, but that does not really matter because you are not interested in how these nodes operate, they are just there to help facilitate the simulation. For example, if you are trying to test out some simulated attack on LND nodes, you could have a few full LND nodes running but you need several different routes to and from these nodes. You could start up these "generic simulation nodes" and open channels to them to create a network around the LND nodes. These nodes do not have to be explicitly defined in SimLN because they won't initiate any payments, they will just forward payments between the real LND nodes. This could increase the size of the test network that is possible.
+    - So far I have been exploring ways to create lightweight implementation-specific nodes that are still very accurate representations of the full implementation. I created an LND node that is just a little more efficient, but is still essentially a full LND node. A more effective strategy might be to create generic simulation nodes that simply help make up a test network and then use full implementations for the nodes of interest that the user is trying test. These "generic simulation nodes" would not be very realistic, but that does not really matter because you are not interested in how these nodes operate, they are just there to help facilitate the simulation. For example, if you are trying to test out some simulated attack on LND nodes, you could have a few full LND nodes running but you need several different routes to and from these nodes. You could start up these "generic simulation nodes" and open channels to them to create a network around the LND nodes. These nodes do not have to be explicitly defined in SimLN because they won't initiate any payments, they will just forward payments between the real LND nodes. This could increase the size of the test network that is possible.
+    - This type of node can be developed and then added to BLAST as another "model" through the model interface that is already built.
 
 ## What was the OpenSats funding used for?
 The grant money was used as part of my income in order to pay for living expenses.