Messaging System

Messaging System

The messaging system is an initial implementation of the Transport and associated interfaces. To work, the messaging system requires an active internet connection and is designed to be horizontally scalable.

Three services make up the messaging system: Messaging Client (or Client Instance), Messaging Server (or Server Instance) and Messaging Discovery.

Messaging Clients and Messaging Servers are represented by public/private key pairs. Messaging Clients deliver data to one another via Messaging Servers which act as relays.

The Messaging Discovery is responsible for allowing Messaging Clients to find other advertised Messaging Clients via their public keys. It is also responsible for finding appropriate Messaging Servers that either "advertise" them, or "advertise" other Messaging Clients.

           [D]

     S(1)        S(2)
   //   \\      //   \\
  //     \\    //     \\
 C(A)    C(B) C(C)    C(D)

Legend:

• [D] - Discovery Service
• S(X) - Messaging Server (Server Instance)
• C(X) - Messaging Client (Client Instance)

Messaging Procedures

This is a summary of the procedures that the Messaging System handles.

Advertising a client:

To be discoverable by other clients, a client needs to advertise itself.

1. Client queries the Discovery to find available Servers.
2. Client connects to some (or all) of the suggested Servers.
3. Client updates it's own record in Discovery to include it's delegated Servers.

Client creates a channel to another client:

In order for two clients to communicate, both clients need to be connected to the same messaging server, and create a channel to each other via the server.

1. Client queries discovery of the remote client's connected servers. The client will connect to one of these servers if it originally has no shared servers with the remote.
2. The client sends a OpenChannel frame to the remote client via the shared server.
3. If the remote client accepts, a ChannelOpened frame is sent back to the initiating client (also via the shared server). A channel is represented via two Channel IDs (between the initiating client and the server, and between the responding client and the server). The associated between the two channel IDs is defined within the server.
4. Once a channel is created, clients can communicate via one another via the channel.

Messaging Discovery

The Messaging Discovery acts like a DNS for messaging instances (Messaging Clients or Messaging Servers).

Instance Entry

An entry within the Messaging Discovery can either represent a Messaging Server or a Messaging Client. The Messaging Discovery is a key-value store, in which entries (of either server or client) use their public keys as their "key".

Endpoints

There are three endpoints:

• Get Entry
• Post Entry
• Get Available Servers

Further documentation can be found here.

Messaging Link

The link provides two Messaging Instances a means to establish a connection with one another, and also handle a pool of connections.

Using the Messaging Discovery, a Messaging Instance can discover other instances via only their public key. However, a Link requires both a public key and an address:port.

Data sent via a Link is encapsulated in Frames. A Link is implemented using a TCP connection.

Link Handshake Frames

When setting up a Link between two instances, the instance that initiates is called the Initiator and the instance that responds is called the Responder. Each instance is represented by a public key.

To set up a Link, the Initiator first dials a TCP connection to the listening Responder. Once the TCP connection is established, the Responder sends the first Frame. It is expected that the Initiator knows the public key of the Responder

Given a situation where instances 'A' and 'B' are to establish a link with one another (where 'A' is the initiator), the following Frames are delivered to perform a handshake.

Link Handshake Frames are to be in JSON format.

Link Handshake Frame 1 (A -> B):

{
    "version": "0.1",
    "initiator": "036b630436972743d4b3f5bb39cd451da29d222b5d30893684a40f34a66c692157",
    "responder": "02e18998f0174631710e47052927e13bddc712ca0c11289e0150fbf57570e31151",
    "nonce": "853ea8454d11bd3b59cb31f8572a3779"
}

The initiator is responsible for sending the first frame.

• "version" specifies the version of messaging protocol that the initiator is using ("0.1" for now).
• "initiator" should contain the hex representation of the public key of the initiator (the instance that is sending the first handshake frame).
• "responder" should contain the hex representation of the public key of the expected responder (the responder should disconnect TCP if this is not their public key).
• "nonce" is the hex-string representation of a 16-byte nonce that the responder should sign (alongside the initiator's public key) to check authenticity of the responder and whether the responder.

Link Handshake Frame 2 (B -> A):

{
    "version": "0.1",
    "initiator": "036b630436972743d4b3f5bb39cd451da29d222b5d30893684a40f34a66c692157",
    "responder": "02e18998f0174631710e47052927e13bddc712ca0c11289e0150fbf57570e31151",
    "nonce": "853ea8454d11bd3b59cb31f8572a3779",
    "sig1": "df8a978f0ea681e218cfd8127692dbe4190441567181b9057ab15da34b08ff610d9060e5195419e1744bb57d50373c1dd444b5c2753a80dba32b292fa306e9df01"
}

This frame allows the responder agree with the initiator and prove it's ownership of it's claimed public key.

The "sig1" field contains a hex representation of the result of signing the concatenation of the version, initiator, responder and nonce fields. Note that before concatenation, hex representations should be decoded and the concatenation result needs to be hashed before being signed. "sig1" should be signed by the responder.

Link Handshake Frame 3 (A -> B):

{
    "version": "0.1",
    "initiator": "036b630436972743d4b3f5bb39cd451da29d222b5d30893684a40f34a66c692157",
    "responder": "02e18998f0174631710e47052927e13bddc712ca0c11289e0150fbf57570e31151",
    "nonce": "853ea8454d11bd3b59cb31f8572a3779",
    "sig1": "df8a978f0ea681e218cfd8127692dbe4190441567181b9057ab15da34b08ff610d9060e5195419e1744bb57d50373c1dd444b5c2753a80dba32b292fa306e9df01",
    "sig2": "fc17928d5a3f7691434282fb3108d1603889f996e8e45adc2e35362e08009b8611abb9f45e511b9931f0f04b37ff1057fd69554befe534ad28c77ff0c44121ab00"
}

This frame allows the initiator to inform the responder that "sig1" is accepted, and to prove the initiator's ownership of it's public key.

"sig2" is the signature result of the concatenation of the version, initiator, responder, nonce and sig1 fields. Concatenation rules are the same as that of "sig1".

Link Handshake Frame 4 (B -> A):

{
    "version": "0.1",
    "initiator": "036b630436972743d4b3f5bb39cd451da29d222b5d30893684a40f34a66c692157",
    "responder": "02e18998f0174631710e47052927e13bddc712ca0c11289e0150fbf57570e31151",
    "nonce": "853ea8454d11bd3b59cb31f8572a3779",
    "sig1": "df8a978f0ea681e218cfd8127692dbe4190441567181b9057ab15da34b08ff610d9060e5195419e1744bb57d50373c1dd444b5c2753a80dba32b292fa306e9df01",
    "sig2": "fc17928d5a3f7691434282fb3108d1603889f996e8e45adc2e35362e08009b8611abb9f45e511b9931f0f04b37ff1057fd69554befe534ad28c77ff0c44121ab00",
    "accepted": true
}

Sent by the responder, this frame concludes the handshake if the value of "accepted" is true.

Messaging Frames

After the handshake phase, frames have a reoccurring format. These are the Messaging Frames of the messaging system.

| FrameType | PayloadSize | Payload |
| 1 byte    | 2 bytes     | ~ bytes |

• The Type specifies the frame type. Different frame types are used for opening and closing channels as well as sending packets via the channels.
• PayloadSize contains an encoded uint16 value that represents the Payload's length (the max size is 65535).
• Payload has a length determined by PayloadSize.

The following is a summary of the frame types.

FrameTypeValue	FrameTypeName	FrameBody
0x0	OpenChannel	ChannelID + RemoteStatic + NoiseMessage1
0x1	ChannelOpened	ChannelID + NoiseMessage2
0x2	CloseChannel	ChannelID
0x3	ChannelClosed	ChannelID
0x4	Send	ChannelID + CipherText

The FrameBody has the following sub-fields. A FrameBody with multiple sub-fields have the sub-fields concatenated.

• The ChannelID sub-field is represented by a single byte. This restricts a Client Instance to have at most 256 channels via a single Server Instance.
• The RemoteStatic sub-field is represented by 33 bytes. It contains a public key of a remote Client Instance.
• NoiseMessage1 and NoiseMessage2 are both represented by 49 bytes. It contains the noise handshake messages for establishing symmetric encryption between the two client instances of the channel. The noise handshake pattern used is KK.
• The CipherText sub-field is the only sub-field with a modular length. It contains size of the encrypted payload followed by payload that is to be delivered.

Noise Implementation in Channels

As stated above, a channel is established using the OpenChannel and ChannelOpened frames. Then, after a channel is established, the two Client Instances of the channel can communicate with each over via Send frames (which includes a CipherText component).

The protocol used to establish the symmetric encryption of the CipherText is the Noise Protocol.

The curve used will be secp256k1 for the key pair, and chacha20poly1305 will be used for the symmetric encryption itself.

Note that, the noise protocol requires the public key length and the ECDH result length (shared secret) to be equal. Because for secp256k1, public keys have a length of 33, and the ECDH result has a length of 32, so an empty byte (0x0) should be appended to all generated ECDH results. Hence, the DHLEN constant for the noise protocol should be 33.

After the handshake, the CipherState object will be used by the Client Instances to encrypt and decrypt the CipherText contained within the Send frame.

Handshake pattern:

Only the KK interactive handshake pattern (fundamental) will be supported.

-> s
<- s
...
-> e, es, ss
<- e, ee, se

The -> e, es, ss message is the NoiseMessage1 of a OpenChannel frame, while the <- e, ee, se message is the NoiseMessage2 of a ChannelOpened frame.

Channel Management

Both the client and server instances needs to manage channels. Channels are associated with a channel ID and also the public key(s) of the remote instances that the channel interacts with. Channels are hence identified by Link + Channel ID.

From the perspective of a Client Instance, the assignment of Channel IDs are unique to a given link with a Server Instance. For example, let's say client 'A' is connected with server 'B' and server 'C', hence we have links 'AB' and 'AC'. We can have 'AB' and 'AC' share the same Channel ID, but because the channel itself is associated with a different link, they are considered different channels.

From the perspective of a Server Instance, the assignment of Channel IDs are unique to a given link with a Client Instance.

TODO: Account for changes here

Opening a Channel

A channel in it's entirety handles the communication between two client instances via a server instance (which acts as a relay). Within the link between a single client instance and the server instance, a channel is represented using a Channel ID. The Channel ID of the two links of the same "channel" can be different, and the Server Instance is responsible for recording this association of the Channel IDs (coupled with the client instance's public key).

When a Client Instance wishes to communicate with another Client Instance, it is responsible for initiating the creation of a channel. To do so, t sends a OpenChannel frame to the Server Instance in which:

• ChannelID contains a ChannelID that the client wishes to associate with the channel.
• RemoteStatic contains the public key of the remote Client Instance that the local client wishes to communicate via this channel.
• NoiseMessage1 is the first noise handshake message (the handshake pattern used is KK).

If the Server Instance wishes to reject the request to open channel, it can send a ChannelClosed frame back to the initiating client with the ChannelID sub-field containing the value of the channel ID suggested by the initiating client.

If the Server Instance wishes to go forward with opening of a channel, it sends a OpenChannel frame to the second Client Instance, in which ChannelID is an ID that's unique between the server and the second client and public key of the first client.

If the second Client Instance wishes to reject the request, it can send a ChannelClosed frame back to the server, and the server can subsequently send a ChannelClosed frame to the initiating client (the ChannelID sub-fields of these ChannelClosed frames should be the unique channel IDs of the associated links).

If the second Client Instance accepts the request, it sends a ChannelOpened back to the Server Instance (with the NoiseMessage2). Subsequently, the Server Instance sends a ChannelOpened back to the initiating client (the ChannelID sub-fields of these ChannelOpened frames should be the unique channel IDs of the associated links).

Closing a Channel

A Client Instance can safely close any of it's channels by sending a CloseChannel (with the associated ChannelID) to the Server Instance.

After a Client Instance sends a CloseChannel, no more frames are to be sent by that instance. However, the remote instance can still send frames until it receives the CloseChannel to it. The "close-responding" client then sends a ChannelClosed instance back to the "close-initiating" client. Once the ChannelClosed channel is sent by the "close-responding" client, it will no longer send or receive frames. Once the "close-initiator" receives the ChannelClosed frame. it will no longer receive frames.

In summary,

• When a client instance sends a CloseChannel frame, the channel is "partially-closed" and the client instance will only receive and not send via the channel. If a ChannelClosed frame is not received after a given timeout, the channel sends a ChannelClosed itself and the channel is "fully-closed".
• When a client instance receives a CloseChannel frame, it delivers a ChannelClosed frame and the channel is "fully-closed" and the client will no longer receive or send via the channel.
• When a client instance receives a ChannelClosed frame, the channel is "fully-closed".

Handling Disconnections

In any given situation, there may be a possibility that the Server Instance unexpectedly disconnects with a Client Instance, or that a Client Instance unexpectedly disconnects with a Server Instance. This should directly affect the channels associated with the Link in question.

When a Client Instance detects that a Server Instance has disconnected from it. All associated channels with that Server Instance should be closed. When a channel closes, the associated Transport should also be closed.

When a Server Instance detects a disconnection from a Client Instance, it should send a ChannelClosed frame to all the other Client Instances that shares a channel with the disconnected client. After so, the Server Instance should dissociate all relations with the closed channels.