Host-to-host communication

[elfenpiff]

Brief feature description

We require a tunnel or gateway to connect multiple iceoryx2 instances on different machines with each other.

A Gateway utilizes another host-to-host-capable communication mechanism to connect iceoryx2 instances on different hosts. Applications using this mechanism will also be able to hook into the traffic.

A tunnel writes iceoryx2 payloads directly to the transport, which requires more configuration and does not enable hooking into the network traffic from non-iceoryx2 applications, but the reduced overhead improves latency.

Detailed information

Requirements

• The gateway shall be configurable
  • it comes with a allow/deny list - services can automatically be shared with the world or are explicitly excluded
  • the configuration is in TOML - like the rest of the iceoryx2 configuration
  • a gateway config shall also contain protocol specific configurations
  • the config shall contain options for peer2peer as well as multicast communications
    • peer2peer example: provide a list of all iceoryx2 endpoints with ip/port
• If possible, the gateway shall not come with extra background threads. The user shall be able to handle everything in the main thread.
  • Certain protocol enforce background threads, but our implementation shall not
• The gateway shall be traffic efficient
  • only publish things that are required from an endpoint in the network
• Gateway shall be usable right out-of-the-box
  • Sane default configuration
  • iceoryx2 provides a service discovery service that is provided by every gateway. this allows other gateways to discover local services
• Service Discovery is written so that it does not require a background thread
  • it shall be split into two parts: 1. the library that performs the service discovery, 2. a binary that can be started and provides the service discovery service
  • the library shall be usable from within an arbitrary application. in the main loop one can call the loop callback of the service discovery to update the internal state and send out updates
  • publish-subscribe: service diff details like: service X, new publisher
  • request-response: to provide full service state
  • event: to notify others about service diff details
  • requires a user side struct for convenience to handle the service diffs(pubsub), state(rpc), events with one convenenient API
• so that samples are not forwarded in circle we need to be able to determine the machine origin of a sample
  • (if possible) find a way to generate a machine unique id automatically and use it in the gateway, otherwise the user must provide a unique id
  • the pubsub header must be extended so that the id can be stored there
• gateways and service discovery must be able to support multiple service styles:
  • ipc::Service, local::Service and future gpu::Service or hypervisor::Service
  • gateways shall support different iceoryx2 configs, see: https://github.com/eclipse-iceoryx/iceoryx2/tree/main/config
    and https://docs.rs/iceoryx2/latest/iceoryx2/config/index.html

Next Steps

• implement service discovery
  • use Service::list() to acquire all static service infos, see: https://docs.rs/iceoryx2/latest/iceoryx2/service/trait.Service.html#method.list
  • connect to service to acquire dynamic service info, see:

// event
let service = self.node.service_builder(element.static_details.name()).event().open().unwrap();
let dynamic_config = service.dynamic_config();

// pubsub
ipc::Service::list(Config::global_config(), |element| {
match element.static_details.messaging_pattern() {
iceoryx2::service::static_config::messaging_pattern::MessagingPattern::PublishSubscribe(config) => {
if let Ok(service) = unsafe { self.node.service_builder(element.static_details.name())
                        .publish_subscribe::<[CustomPayloadMarker]>()
                        .user_header::<CustomHeaderMarker>()
                        .__internal_set_payload_type_details(&config.message_type_details().payload)
                        .__internal_set_user_header_type_details(&config.message_type_details().user_header)
                        .open() } {
                        let dynamic_config = service.dynamic_config();

• event, use: https://docs.rs/iceoryx2/latest/iceoryx2/service/dynamic_config/event/struct.DynamicConfig.html
• pubsub, use: https://docs.rs/iceoryx2/latest/iceoryx2/service/dynamic_config/publish_subscribe/index.html
• The internal but public methods acquire more details:
  • pubsub: dynamic_config.__internal_list_subscribers(), dynamic_config.__internal_list_publishers()
  • event: dynamic_config.__internal_list_listeners(), dynamic_config.__internal_list_notifiers()