Epoch phase performance improvements, access node open slot limits, and EpochStart event

[joshuahannan]

Makes three changes to improve the performance of the epoch transition:

1. Moves FlowEpoch.calculateAndSetRewards to the end of the epoch setup phase/beginning of epoch commit, which currently does not have much complexity. This removes calculating rewards from the end of the epoch, which when combined with moveTokens, was causing the execution to take a long time.

2. Changes FlowIDTableStaking.setNewMovesPending() to be able to add a new node or delegator to the moves pending list without loading and storing to and from storage. If many nodes and delegators were unstaking tokens, this was very costly at the end of each epoch. The change uses a reference to allow the moves pending list to be modified in memory and not loaded or stored during node registration transactions.

3. Updates the methods that are run at the end of the service auction to construct the proposed Node ID list as they go instead of the epoch contract having to query it fully at the end, which requires iterating through all nodes in the identity table to construct it.

Adds some new features to the epoch and staking contracts

5. Adds a FlowEpoch.EpochStart event that is emitted at the start of every epoch and includes important information related to FlowToken and epoch information.

6. Adds a feature to open up new access node slots each epoch instead of the service account committee having to do it manually

Also updates some comments and error messages for some transactions

No tests are needed for the first two improvements because they don't affect the behavior of the contracts, just the performance

[janezpodhostnik]

Can you explain in more detail why we can do this? I'm not sure I understand.

If the rewards are calculated earlier is there not a chance they would change?

[joshuahannan]

I'm moving the rewards calculation to earlier so that it doesn't have to happen in the same transaction where the epoch transitions and tokens are moved. We currently don't have any logic to change rewards once they have been calculated anyway, so it doesn't really matter if they are calculated earlier.

[joshuahannan]

@jordanschalm No rush, but I could use another review on this when you get the chance. We could also schedule a call so I can walk you through the changes if you need help

[joshuahannan]

@jordanschalm Can you take a look at my last two commits? I've added in all the performance improvements we discussed this week

[bjartek]

I think it is very important that every set of service chunk events that transitions something contains the counter field. Atm NewEpoch and Payout event from FlowIDTableStaking does not contain this. This makes it very hard to identify what epoch this payout was for. In some cases these events happen before NewEpoch and in others it hapends after. If we have to "fix stuff"

[joshuahannan]

@bjartek That would be nice to have, but that might require the staking contract to import the FlowEpoch contract, which would be a circular dependency, and we want to avoid that. I guess we could pass the epoch counter as an argument to the payRewards() and moveTokens() functions so we can use it in the events, but that still feels a little weird. It isn't enough that you can infer the epoch from when the event was emitted? We could potentially include it if it is that important to you

[bjartek]

Is it possible to have another event emitted when rewards are paid from FlowEpoch that has the counter?

[jordanschalm]

I guess we could pass the epoch counter as an argument to the payRewards() and moveTokens() functions so we can use it in the events

Assuming it's relatively straightforward, I think we should just do this and include the epoch counter in all events where it could be useful. Inferring auxiliary data about events based on the current state when they are emitted, or based on other events, is error-prone and more complex.

[joshuahannan]

Okay. I'll add those to the events in the staking contract then

[joshuahannan]

I did the upgrade on canary and everything seems to be working properly. We'll upgrade testnet on monday the 27th and stick with the dec 6th mainnet upgrade. I'm going to go ahead and merge this though unless anyone has any objections