[WIP] Light client protocol + Req/Resp interaction

README.md

@@ -27,11 +27,14 @@ The current features are:
 
 ### Altair
 
-* [Beacon chain changes](specs/altair/beacon-chain.md)
+* [Beacon Chain changes](specs/altair/beacon-chain.md)
 * [Altair fork](specs/altair/fork.md)
 * [Light client sync protocol](specs/altair/sync-protocol.md)
 * [Honest Validator guide changes](specs/altair/validator.md)
 * [P2P Networking](specs/altair/p2p-interface.md)
+* [Light Client](specs/altair/light-client/README.md) - In active R&D
+    * [Sync Protocol](specs/altair/light-client/sync-protocol.md)
+    * [P2P Networking](specs/altair/light-client/p2p-interface.md)
 
 ### Merge
 

setup.py

@@ -608,7 +608,7 @@ def combine_constants(old_constants: Dict[str, str], new_constants: Dict[str, st
     'Bytes1', 'Bytes4', 'Bytes20', 'Bytes32', 'Bytes48', 'Bytes96', 'Bitlist', 'Bitvector',
     'uint8', 'uint16', 'uint32', 'uint64', 'uint128', 'uint256',
     'bytes', 'byte', 'ByteList', 'ByteVector',
-    'Dict', 'dict', 'field', 'ceillog2', 'floorlog2', 'Set',
+    'Dict', 'dict', 'field', 'ceillog2', 'floorlog2', 'Set', 'set'
 ]
 
 
@@ -728,7 +728,7 @@ def finalize_options(self):
                     specs/altair/fork.md
                     specs/altair/validator.md
                     specs/altair/p2p-interface.md
-                    specs/altair/sync-protocol.md
+                    specs/altair/light-client/sync-protocol.md
                 """
             elif self.spec_fork == MERGE:
                 self.md_doc_paths = """

specs/altair/light-client/README.md

@@ -0,0 +1,26 @@
+## Table of contents
+
+<!-- TOC -->
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [Ethereum 2.0 Altair - Beacon chain light client](#ethereum-20-altair---beacon-chain-light-client)
+  - [Specs](#specs)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+<!-- /TOC -->
+
+# Ethereum 2.0 Altair - Beacon chain light client
+
+The beacon chain light client protocol is an extra protocol for light clients and servers to communicate.
+We expect the beacon nodes that fully sync and verify the latest beacon state to act as servers
+while the light clients only have to download a partial of the beacon state from the servers.
+
+In the current simple design, light client only sync to the latest finalized beacon chain head
+so there should be no reorganization.
+The reorganizable light client design is still in active R&D.
+
+## Specs
+
+- [P2P Networking](./p2p-interface.md): the Req/Resp message formats for light client communications
+- [Sync Protocol](./sync-protocol.md): the detailed sync protocol

specs/altair/light-client/p2p-interface.md

@@ -0,0 +1,61 @@
+# Ethereum Altair Light Client P2P Interface
+
+**Notice**: This document is a work-in-progress for researchers and implementers.
+
+This document contains the networking specification for [minimal light client](./sync-protocol.md).
+This document should be viewed as a patch to the [Altair networking specification](./../p2p-interface.md).
+
+## Table of contents
+
+<!-- TOC -->
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [The Req/Resp domain](#the-reqresp-domain)
+  - [Messages](#messages)
+    - [GetLightClientSnapshot](#getlightclientsnapshot)
+    - [LightClientUpdate](#lightclientupdate)
+- [Server discovery](#server-discovery)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+<!-- /TOC -->
+
+## The Req/Resp domain
+
+### Messages
+
+#### GetLightClientSnapshot
+
+**Protocol ID:** `/eth2/beacon_chain/req/get_light_client_snapshot/1/`
+
+No Request Content.
+
+Response Content:
+
+```
+(
+  LightClientSnapshot
+)
+```
+
+The `LightClientSnapshot` SSZ container defined in [light client sync protocol](./sync-protocol.md#lightclientsnapshot).
+
+#### LightClientUpdate
+
+**Protocol ID:** `/eth2/beacon_chain/req/light_client_update/1/`
+
+Request Content:
+
+```
+(
+  LightClientUpdate
+)
+```
+
+No Response Content.
+
+The `LightClientUpdate` SSZ container defined in [light client sync protocol](./sync-protocol.md#lightclientupdate).
+
+## Server discovery
+
+[TODO]

specs/altair/sync-protocol.md → specs/altair/light-client/sync-protocol.md

@@ -1,4 +1,4 @@
-# Minimal Light Client
+# Ethereum Altair Minimal Light Client
 
 **Notice**: This document is a work-in-progress for researchers and implementers.
 
@@ -9,6 +9,11 @@
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
 - [Introduction](#introduction)
+- [Sync protocol](#sync-protocol)
+  - [Initialization](#initialization)
+  - [Minimal light client update](#minimal-light-client-update)
+    - [Server side](#server-side)
+    - [Client side](#client-side)
 - [Constants](#constants)
 - [Configuration](#configuration)
   - [Misc](#misc)
@@ -18,10 +23,16 @@
   - [`LightClientStore`](#lightclientstore)
 - [Helper functions](#helper-functions)
   - [`get_subtree_index`](#get_subtree_index)
-- [Light client state updates](#light-client-state-updates)
-    - [`validate_light_client_update`](#validate_light_client_update)
-    - [`apply_light_client_update`](#apply_light_client_update)
-    - [`process_light_client_update`](#process_light_client_update)
+  - [`get_light_client_store`](#get_light_client_store)
+  - [`get_light_client_slots_since_genesis`](#get_light_client_slots_since_genesis)
+  - [`get_light_client_current_slot`](#get_light_client_current_slot)
+  - [`validate_light_client_update`](#validate_light_client_update)
+  - [`apply_light_client_update`](#apply_light_client_update)
+- [Client side handlers](#client-side-handlers)
+  - [`on_light_client_tick`](#on_light_client_tick)
+  - [`on_light_client_update`](#on_light_client_update)
+- [Server side handlers](#server-side-handlers)
+- [Reorg mechanism](#reorg-mechanism)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -30,11 +41,36 @@
 
 Eth2 is designed to be light client friendly for constrained environments to
 access Eth2 with reasonable safety and liveness.
-Such environments include resource-constrained devices (e.g. phones for trust-minimised wallets)
+Such environments include resource-constrained devices (e.g. phones for trust-minimized wallets)
 and metered VMs (e.g. blockchain VMs for cross-chain bridges).
 
 This document suggests a minimal light client design for the beacon chain that
-uses sync committees introduced in [this beacon chain extension](./beacon-chain.md).
+uses sync committees introduced in [this beacon chain extension](./../beacon-chain.md).
+
+## Sync protocol
+
+### Initialization
+
+1. The client sends [`Status` message](./../../phase0/p2p-interface.md#status) to the server to exchange the status information.
+2. Instead of sending [`BeaconBlocksByRange` request](./../../phase0/p2p-interface.md#beaconblocksbyrange) in the beacon chain syncing, the client sends [`GetLightClientSnapshot` request](./p2p-interface.md#getlightclientsnapshot) to the server.
+3. The server responds with the `LightClientSnapshot` object of the finalized beacon chain head.
+4. The client would:
+    1. check if the hash tree root of the given `header` matches the `finalized_root` in the server's `Status` message.
+    2. check if the given response is valid for client to call `get_light_client_store` function to get the initial `store: LightClientStore`.
+    - If it's invalid, disconnect from the server; otherwise, keep syncing.
+
+### Minimal light client update
+
+In this minimal light client design, the light client only follows finality updates.
+
+#### Server side
+
+- Whenever `state.finalized_checkpoint` is changed, call `get_light_client_update` to generate the `LightClientUpdate` and then send to its light clients.
+
+#### Client side
+
+- `on_light_client_tick(store, time)` whenever `time > store.time` where `time` is the current Unix time
+- `on_light_client_update(store, update)` whenever a block `update: LightClientUpdate` is received
 
 ## Constants
 
@@ -77,8 +113,7 @@ class LightClientUpdate(Container):
     finality_header: BeaconBlockHeader
     finality_branch: Vector[Bytes32, floorlog2(FINALIZED_ROOT_INDEX)]
     # Sync committee aggregate signature
-    sync_committee_bits: Bitvector[SYNC_COMMITTEE_SIZE]
-    sync_committee_signature: BLSSignature
+    sync_aggregate: SyncAggregate
     # Fork version for the aggregate signature
     fork_version: Version
 ```
@@ -87,9 +122,12 @@ class LightClientUpdate(Container):
 
 ```python
 @dataclass
-class LightClientStore(object):
+class LightClientStore:
+    time: uint64
+    genesis_time: uint64
+    genesis_validators_root: Root
     snapshot: LightClientSnapshot
-    valid_updates: Set[LightClientUpdate]
+    valid_updates: Set[LightClientUpdate] = field(default_factory=set)
 ```
 
 ## Helper functions
@@ -101,22 +139,52 @@ def get_subtree_index(generalized_index: GeneralizedIndex) -> uint64:
     return uint64(generalized_index % 2**(floorlog2(generalized_index)))
 ```
 
-## Light client state updates
+### `get_light_client_store`
+
+```python
+def get_light_client_store(snapshot: LightClientSnapshot,
+                           genesis_time: uint64, genesis_validators_root: Root) -> LightClientStore:
+    return LightClientStore(
+        time=uint64(genesis_time + SECONDS_PER_SLOT * snapshot.header.slot),
+        genesis_time=genesis_time,
+        genesis_validators_root=genesis_validators_root,
+        snapshot=snapshot,
+        valid_updates=set(),
+    )
+```
+
+### `get_light_client_slots_since_genesis`
+
+```python
+def get_light_client_slots_since_genesis(store: LightClientStore) -> int:
+    return (store.time - store.genesis_time) // SECONDS_PER_SLOT
+```
+
+### `get_light_client_current_slot`
 
-A light client maintains its state in a `store` object of type `LightClientStore` and receives `update` objects of type `LightClientUpdate`. Every `update` triggers `process_light_client_update(store, update, current_slot)` where `current_slot` is the current slot based on some local clock.
+```python
+def get_light_client_current_slot(store: LightClientStore) -> Slot:
+    return Slot(GENESIS_SLOT + get_light_client_slots_since_genesis(store))
+```
 
-#### `validate_light_client_update`
+### `validate_light_client_update`
 
 ```python
-def validate_light_client_update(snapshot: LightClientSnapshot,
-                                 update: LightClientUpdate,
-                                 genesis_validators_root: Root) -> None:
+def validate_light_client_update(store: LightClientStore, update: LightClientUpdate) -> None:
+    snapshot = store.snapshot
+
     # Verify update slot is larger than snapshot slot
     assert update.header.slot > snapshot.header.slot
 
+    # Verify time
+    update_time = uint64(store.genesis_time + SECONDS_PER_SLOT * update.header.slot)
+    assert store.time >= update_time
+
     # Verify update does not skip a sync committee period
-    snapshot_period = compute_epoch_at_slot(snapshot.header.slot) // EPOCHS_PER_SYNC_COMMITTEE_PERIOD
-    update_period = compute_epoch_at_slot(update.header.slot) // EPOCHS_PER_SYNC_COMMITTEE_PERIOD
+    snapshot_epoch = compute_epoch_at_slot(snapshot.header.slot)
+    update_epoch = compute_epoch_at_slot(update.header.slot)
+    snapshot_period = compute_sync_committee_period(snapshot_epoch)
+    update_period = compute_sync_committee_period(update_epoch)
     assert update_period in (snapshot_period, snapshot_period + 1)
 
     # Verify update header root is the finalized root of the finality header, if specified
@@ -148,48 +216,77 @@ def validate_light_client_update(snapshot: LightClientSnapshot,
         )
 
     # Verify sync committee has sufficient participants
-    assert sum(update.sync_committee_bits) >= MIN_SYNC_COMMITTEE_PARTICIPANTS
+    assert sum(update.sync_aggregate.sync_committee_bits) >= MIN_SYNC_COMMITTEE_PARTICIPANTS
 
     # Verify sync committee aggregate signature
-    participant_pubkeys = [pubkey for (bit, pubkey) in zip(update.sync_committee_bits, sync_committee.pubkeys) if bit]
-    domain = compute_domain(DOMAIN_SYNC_COMMITTEE, update.fork_version, genesis_validators_root)
+    participant_pubkeys = [pubkey for (bit, pubkey)
+                           in zip(update.sync_aggregate.sync_committee_bits, sync_committee.pubkeys) if bit]
+    domain = compute_domain(DOMAIN_SYNC_COMMITTEE, update.fork_version, store.genesis_validators_root)
     signing_root = compute_signing_root(signed_header, domain)
-    assert bls.FastAggregateVerify(participant_pubkeys, signing_root, update.sync_committee_signature)
+    assert bls.FastAggregateVerify(participant_pubkeys, signing_root, update.sync_aggregate.sync_committee_signature)
 ```
 
-#### `apply_light_client_update`
+### `apply_light_client_update`
 
 ```python
 def apply_light_client_update(snapshot: LightClientSnapshot, update: LightClientUpdate) -> None:
-    snapshot_period = compute_epoch_at_slot(snapshot.header.slot) // EPOCHS_PER_SYNC_COMMITTEE_PERIOD
-    update_period = compute_epoch_at_slot(update.header.slot) // EPOCHS_PER_SYNC_COMMITTEE_PERIOD
+    snapshot_epoch = compute_epoch_at_slot(snapshot.header.slot)
+    update_epoch = compute_epoch_at_slot(update.header.slot)
+    snapshot_period = compute_sync_committee_period(snapshot_epoch)
+    update_period = compute_sync_committee_period(update_epoch)
     if update_period == snapshot_period + 1:
         snapshot.current_sync_committee = snapshot.next_sync_committee
         snapshot.next_sync_committee = update.next_sync_committee
     snapshot.header = update.header
 ```
 
-#### `process_light_client_update`
+## Client side handlers
+
+### `on_light_client_tick`
 
 ```python
-def process_light_client_update(store: LightClientStore, update: LightClientUpdate, current_slot: Slot,
-                                genesis_validators_root: Root) -> None:
-    validate_light_client_update(store.snapshot, update, genesis_validators_root)
+def on_light_client_tick(store: LightClientStore, time: uint64) -> None:
+    # update store time
+    store.time = time
+```
+
+### `on_light_client_update`
+
+A light client maintains its state in a `store` object of type `LightClientStore` and receives `update` objects of type `LightClientUpdate`. Every `update` triggers `on_light_client_update(store, update)`.
+
+```python
+def on_light_client_update(store: LightClientStore, update: LightClientUpdate) -> None:
+    validate_light_client_update(store, update)
     store.valid_updates.add(update)
 
     update_timeout = SLOTS_PER_EPOCH * EPOCHS_PER_SYNC_COMMITTEE_PERIOD
     if (
-        sum(update.sync_committee_bits) * 3 >= len(update.sync_committee_bits) * 2
+        sum(update.sync_aggregate.sync_committee_bits) * 3 >= len(update.sync_aggregate.sync_committee_bits) * 2
         and update.finality_header != BeaconBlockHeader()
     ):
         # Apply update if (1) 2/3 quorum is reached and (2) we have a finality proof.
         # Note that (2) means that the current light client design needs finality.
         # It may be changed to re-organizable light client design. See the on-going issue eth2.0-specs#2182.
         apply_light_client_update(store.snapshot, update)
         store.valid_updates = set()
-    elif current_slot > store.snapshot.header.slot + update_timeout:
+    elif get_light_client_current_slot(store) > store.snapshot.header.slot + update_timeout:
         # Forced best update when the update timeout has elapsed
         apply_light_client_update(store.snapshot,
-                                  max(store.valid_updates, key=lambda update: sum(update.sync_committee_bits)))
+                                  max(store.valid_updates,
+                                      key=lambda update: sum(update.sync_aggregate.sync_committee_bits)))
         store.valid_updates = set()
 ```
+
+## Server side handlers
+
+[TODO]
+
+```python
+def get_light_client_update(state: BeaconState) -> LightClientUpdate:
+    # [TODO]
+    pass
+```
+
+## Reorg mechanism
+
+[TODO] PR#2182 discussion