[EFM] Smart Contract represents DKG key vectors as mappings

[jordanschalm]

Problem Definition

General Context

Currently DKG committees are stored and represented in most places in our codebase as an ordered list of keys $K$ corresponding to a consensus committee $C$. There are equal numbers of keys and consensus committee members, and if $C$ is ordered according to canonical ordering, then $K[i]$ is the key for $C[i]$. The index of a node in the consensus committee is always equal to the index of that node's key in the DKG key list.

Let $D$ be the set of nodes that participated in the DKG yielding $K$. Our model works when $D=C$, but does not work in any other case.

• This representation does not map well to the underlying DKG index concept, where indices are meaningful and not necessarily consecutive.
• This representation cannot represent cases where:
  • $\exists n : n \in D, n \notin C$ - there is a node that participated in the DKG, but isn't in the committee
  • $\exists n : n \in C, n \notin D$ - there is a node that is in the committee, but didn't participate in the DKG

We would like to change the way DKG committees are represented to support the above cases.

New DKG Data Model

The updated data model should include the following fields:

• PubKeys - A list of all individual public keys, ordered by DKG index.
  • []crypto.PublicKey in Go
  • [String] in Cadence, each element being a hex-encoded string
• IndexMap - A mapping from node ID to DKG index (includes node IDs that do not exist in the consensus committee)
  • map[flow.Identifier]int in Go
  • {String:Int} in Cadence, keys being hex-encoded identifiers
• GroupKey - The single group public key. crypto.PublicKey in Go, String (hex-encoded) in Cadence.

It has the invariants:

• len(PubKeys) = len(IndexMap)
• IndexMap values form the set {0, 1, ..., n-1} where n=len(PubKeys)

Specifics for this issue

The smart contract stores DKG keys as a list of hex-encoded keys (implicit mapping from node to key).

Proposed Solution

• Modify FlowDKG and relevant transaction scripts to add a {String:Int} type to represent the DKG index mapping, and a separate argument to represent the group public key (which has no index)
  • The key vector should map node IDs (dict keys) to DKG indices (dict values)
• Modify FlowEpoch.EpochCommit event to include the new {String:Int} field for the DKG index mapping.
  • ⚠️ Caution: Moving the DKG group public key to a separate field is a non-backward compatible change in general. However, since the data originates from Consensus Nodes, we can coordinate the data model upgrade using a protocol HCU.
  • ⚠️ Caution: What happens if a node is replaying old blocks with a new software version, and must process first an old EpochCommit event then a new one?

Note: It is critical that instances of the {String:Int} key vector mapping have a single canonical representation, as this is how the smart contract determines a "winning" key vector. Verify this is the case for Dictionaries in Cadence. Alternatively we could use a list of tuples/structs in a pre-defined order.

Implementing this issue on its own will be a breaking change for the DKG, to be addressed in #6214.

[jordanschalm]

Closing as complete: onflow/flow-core-contracts#465