Status Update: 2022-11-23

[phil-opp]

In this discussion, we describe our recent progress and the main topics that we would like to discuss. We use a separate post for each topic to keep the discussion focused. Feel free to create new posts for topics that are not mentioned yet.

[phil-opp]

Dora v0.1 Release

• We released dora v0.1 last week
  • precompiled versions for Linux, Windows, and macOS
  • includes the dora cli, the dora-coordinator control plane, the dora-runtime for operators, and the iceoryx daemon (on UNIX only)
• A Getting Started guide is available in our README
• Please try it out and report any problems

Known Issues

• PyO3/Python dynamic link errors, especially on macOS: Errors when follow README for 0.1.0 release #147
  • PyO3 links the Python runtime dynamically, but it expects a certain path
  • This causes "Library not loaded" link errors when trying to run a Python operator on a machine with a different Python path. For details, see: Failed to run with miniconda Python on macOS PyO3/pyo3#2110
  • Static linking would solve the issue, but it's not officially supported by PyO3
  • Workaround: Package dora-runtime as a Python library and start it with the python executable that is in the PATH
    • See the draft PR at Fixing Python linking error #149
    • There are still issues with managing threads with pyo3/python GIL
• Iceoryx build errors when trying to build Rust node on macOS
  • the automatic CMake-based build of the C++ iceoryx code fails
  • For details, see Rust example build error when using 0.1.0 instruction  #150

We originally planned to create a blog post to promote the v0.1 release. However, because of the above issues, we decided to postpone this to the v0.2 release.

[haixuanTao]

See: https://pyo3.rs/latest/building_and_distribution.html#python-version

[phil-opp]

Plans for v0.2.0?

• Replacement for iceoryx
  • Building and distributing the C++-based iceoryx is painful
  • We plan to create our own dora-daemon as an alternative for shared memory communication
• Use the dora-daemon as a broker for custom node messages
  • Instead of directly sending the messages from the Rust/C/C++/Python nodes
  • This way, only the dora-daemon depends on the communication layer, so the node API does not need to build all the backends (iceoryx, zenoh, etc) anymore
  • Design overview:
    
• Maybe: Unify nodes and operators again
  • Shared memory communication through the dora-daemon might allow a fast, one-process-per-operator architecture
  • Spawning each operator as a separate process would remove the complexity of linking python, shared libraries and the dora-runtime, into a single process.
  • We already spawn each Python operator as a separate process -> no performance difference for Python
  • Open questions:
    • Is the communication through shared memory as fast as mpsc channels?
    • Is there any other problematic overhead when launching each operator as a separate process?

[phil-opp]

Comment from the meeting: We need to make sure that the dora-daemon does not become the single point of failure. We should think about ways to restart the daemon quickly, and also about ways to prevent data loss.

[phil-opp]

Idea: Make the communication part of dora-daemon general enough to make it usable in other projects as well, e.g. as a replacement for iceoryx. Ideally, we could also create C/C++ bindings.

[phil-opp]

"dora-rover"

• Agreement in principle with CentraleSupelec Tech Club to build a dora rover as the one we did with Bob.
• Instagram

https://www.instagram.com/reel/CkquyYJJb4y/

[haixuanTao]

robots available on the market:
-> F 1 Tenth
-> Xmax

[haixuanTao]

-> Maybe, Ten Robot: https://www.tianbot.com (this seems to be not open source)
-> MIT Racecar, https://mit-racecar.github.io/

[haixuanTao]

Embedded side:

• Compiled .so or raw file? Future plan we might compile Python .py to .so