Tracking: Add regtest network

[arya2]

Motivation

This has been requested, and could be useful for testing.

Depends-On: #7845

Possible Design

• Add a regtest() constructor method to Network #7839
• Add a name field to NetworkParameters to be used at the end of the cache dir path #8327
• Disable peer connections (or only connect to nodes with the same network magic)
• Skip loading checkpoints and update NetworkUpgrade::activation_height to use zcashd defaults or user-defined heights
  • Use network genesis_hash as first checkpoint when checkpoints are missing #7846
• Update target_difficulty_limit() and expected_difficulty returned by difficulty_time_and_history_tree() to match zcashd target difficulty for regtest #8479
  • remove the implementation constraint that PoWLimit must be less than or equal to (2^256 - 1) / PoWAveragingWindow (approximately 2^251) to avoid overflow
  • calculate difficulty adjustment for heights less than 17
• Skip SLOW_START_INTERVAL checks (nSubsidySlowStartInterval is 0 for regtest in zcashd)
• Allow for coinbases to be spent to transparent outputs on regtest #8478
• Add a disable_peers field to testnet::Parameters #8361
• Allow local peer connections on Regtest #9182
• Add protocol version information to fields on NetworkParameters #8367
• Update prefixes (optional)
• Match other differences between zcashd testnet vs regtest in chainparams

[teor2345]

These are just some notes for if we eventually do this ticket. No action is needed at this time.

---

We might also want to take a look at:

• Refactor Network and NetworkUpgrade enums into structs and a trait #1135

It seems like some of these changes would benefit from the Regtest variant containing a consensus rule configuration. (Like zcashd has for all its network variants.)

It would also be a lot easier to make this change if we refactored all our network conditions into methods on the Network enum. For example, a has_slow_start() method, rather than a match statement buried in the zebra-consensus code.

• update From<Network> for &'static str for db_path()

We either need to work out how to make sure the consensus rules match the cached state, or make regtest ephemeral by default.

This isn't an issue for mainnet or testnet because their rules are fixed and can't be configured.

• Set peerset_initial_target_size to 0 (Zebra should not make any peer connections)

This change isn't enough to stop Zebra making outbound connections, because Zebra will try to connect to new peers every minute or so. Instead, the initial seed peers list needs to be empty for regtest.

• Update prefixes

Which prefixes?

• Match other differences between zcashd testnet vs regtest in chainparams

If we're not going to connect to other nodes by default, then compatibility with zcashd seems like a lower priority. So we might want to categorise this ticket into:

• changes needed to have a regtest mode
• changes requested by the people who requested regtest mode (please link to the request in the ticket)
• other changes that are useful for full compatibility with zcashd

That way, when we schedule this work, we can split the ticket, give the categories different priorities, then do the estimates.

[teor2345]

Here's one request for regtest in Zebra:
https://twitter.com/thecodebuffet/status/1674606998908727297

Screenshot in case twitter goes down:

[teor2345]

This is currently blocked by some kind of network refactor. (It doesn't have to be #1135 specifically.)

[mpguerra]

Hey team! Please add your planning poker estimate with Zenhub @arya2 @oxarbitrage @teor2345 @upbqdn

[teor2345]

@mpguerra this ticket is outdated. @arya2 and I have had some private conversations about it, and I think we should update it before we estimate it. (Or replace it with another ticket.)

[teor2345]

For example:

I'd like to split the tasks in this ticket based on what the ZSA team needs for their testing, then do the rest of regtest later.

• Add a Regtest variant to Network, update From<Network> for &'static str for db_path(), default port, and genesis hash

It will be much easier to gradually add network settings inside the existing Testnet variant. Adding new variants requires changing a lot of code.

• Add config field or CLI option for accepting network upgrade heights

This should be split into a different ticket, it's not required to match the default zcashd regtest.

[teor2345]

I added these issues to the ticket description, we might need to fix them depending on the required scope:

• difficulty adjustment for heights less than 17
• contextual validation of the genesis block (it has to be a checkpoint)

#7770 (comment)

[hhanh00]

Personally, the lack of regtest support blocks me from moving to zebrad because it is too hard to write integration tests.

[arya2]

Personally, the lack of regtest support blocks me from moving to zebrad because it is too hard to write integration tests.

@hhanh00 Is it enough for Zebra to support NU5 on its Regtest, or is it important that Zebra support Canopy and prior network upgrades on Regtest as well?

[hhanh00]

Just nu5 is enough but I would also need the ability to generate, invalidate and reconsider blocks. This is how the project I work on tests forks and reorganizations.

[mpguerra]

@arya2 it seems like there are still some meaningful gaps between zebra regtest mode and zcashd regtest mode. See: https://discord.com/channels/809218587167293450/811004767043977288/1280987474918772858
Do we know what these are? If so we should update this issue with the remaining work to make them compatible

[arya2]

it seems like there are still some meaningful gaps between zebra regtest mode and zcashd regtest mode. See: https://discord.com/channels/809218587167293450/811004767043977288/1280987474918772858 Do we know what these are? If so we should update this issue with the remaining work to make them compatible

Most of the outstanding work is probably in adding the invalidate and reconsiderblock RPCs, but the other remaining differences that come to mind are:

• mining or generating blocks with valid Equihash solutions, and
• allowing local peer connections

There's also some cleanup to be done in #8367 that would simplify some code and allow for configuring network protocol versions on custom Testnets.