dora-rs
Dora CLI #81
Replies: 4 comments 9 replies
-
|
@phil-opp @haixuanTao please think about this CLI proposal, we can further discuss |
Beta Was this translation helpful? Give feedback.
-
|
We may also add two more command: |
Beta Was this translation helpful? Give feedback.
-
|
For This is also valid for the dashboard. This is where tools like Grafana shine, as it has a lot of integrations. |
Beta Was this translation helpful? Give feedback.
-
|
I am thinking this is more for dora internal storage, and let the community to integrate/export back to whatever backend solution they choose. Of course, when we implement dora dashboard, we will provide some default backend such as Grafana I think this will apply to operators/nodes as well, in case dora stats need to query some info |
Beta Was this translation helpful? Give feedback.
-
|
As discussed, we'll start with a simple DB such as InfluxDB, Prometheus, ... |
Beta Was this translation helpful? Give feedback.
-
|
Hello everyone, I just stumpled upon dora and am pretty impressed with all of this! The one thing that is missing for me to seriously consider using dora for our applications is the logging infrastructure (similar to rosbag). I see that @haixuanTao put in some thoughts here, but I was curious if there is any progress or more thoughts on this? What is missing to make this work, and how could the community potentially help in pushing this forward? Thanks for clarifying, |
Beta Was this translation helpful? Give feedback.
-
|
Hi Jan, Thanks! So, we are investigating very deeply about data logging! Currently, we are building a bridge from dora to ROS2, that way you can use rosbag for your legacy data already logged or if you want to keep on using rosbag because it's a tool you know well. And this is our main focus at the moment! We want to learn as much about data logging with rosbag before building our own. We hope to bring a new tool to do data logging however! With the hope that the data logged is easily reusable. For instance, using the arrow file format to do zero-copy reading and provide very fast dataset in the likes of: https://huggingface.co/docs/datasets/about_arrow . This will make integrating DL dataset building for training and then directly using it in robot easier. We also hope to integrate the logs files and other metadata as well in the format to enable better debugging! Do you mind sharing what features you would like to see in a logging infrastructure (similar to rosbag)? Or potentially in which case would this be helpful to you? |
Beta Was this translation helpful? Give feedback.
-
|
That's great to hear! I don't know much about arrow, but I like the idea of being able to directly loading my logged data inside a Pytorch dataloader, without having to do any preprocessing and without any dependencies on something like ros! A lot of my current workflow right now is a) collecting data on the robot b) process the rosbag (this may even involve multiple steps, but the first is always from rosbag to simple files e.g. json/yaml and images) and c) further processing and loading. Obviously it is very important to be able to record data at a certain rate and then be able to replay it at similar rates, so that the downstream modules are not able to tell wether the data is coming from the robot or from the recording. I think those are the most important use cases for me. |
Beta Was this translation helpful? Give feedback.
-
|
My thoughts on this are as follows:
Sounds good to me!
In order to build the nodes/operators, we need to know the necessary build steps. We could add support for a simple build command to the YAML file, but I think we should be careful about potential complexity. We don't want to re-invent Also, this should be fully optional as users might want to use their own build system.
Could you clarify what you mean here? Are you thinking about a way to automatically generate necessary boilerplate? In that case, we could also add support for a
Sounds nice!
These sound good to me too.
Do you mean the standard output? Or should we provide some sort of opinionated logging callback? Maybe it makes sense to provide such callbacks as extensions that we stay as flexible as possible. For example, in some uses cases open telemetry logging might make sense, but in other cases users might want to use only file-based logging or no logging at all.
What's the difference between metrics and stats? These sound quite similar to me. They also have some overlap with the
I think the name "list" too generic. It's not clear if it lists the running dataflows, the available machines, etc. Maybe it makes sense to introduce another subcommand level. For example, we could use
Could you clarify what you mean with "entity". The dora framework in its entirety or a single operator/node/machine etc.?
That's certainly nice to have, but I fear that it might be difficult to perform such an upgrade automatically. We would need to update multiple executables, potentially across different machines, and also update all the nodes and operators to the new API crate versions (which might require code adjustments). |
Beta Was this translation helpful? Give feedback.
-
|
@phil-opp template -- exactly as you described log -- agree with your comments metrics/stats -- these indeed overlaps; for stats, I was thinking more of the traffic statistics list -- agree with you on this with sub command. We do potentially have multiple dataflow instance running. get -- Dora entities/resources are dataflow, nodes, operators, edges etc (just easy for user/developer to get internal info on dora). I am thinking how to manage these resources upgrade -- this is indeed complex / long term goal. At this stage, we just need to pay attention to these fault tolerance/redundancy aspects in our design |
Beta Was this translation helpful? Give feedback.
-
|
When there are multiple operators in a dataflow, if an operator reports an error, using dora will output the dora logs uuid op_name of all operators. Is it possible to accurately locate the error operator here, or highlight the error operator? It is convenient to quickly locate the problem operator. (dora3.7) jarvis@jia:~/coding/rust_home/github.com/meua/dora-drives$ RUST_LOG=info dora start graphs/oasis/oasis_agent.yaml --attach --hot-reload
bf70dd14-a2a5-41ba-a6be-986a752ceb44
2023-06-29T08:20:27.619314Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/oasis_agent failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 oasis_agent
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:27.619333Z WARN dora_daemon::pending: node `oasis_agent` exited before initializing dora connection
at binaries/daemon/src/pending.rs:76
2023-06-29T08:20:27.804293Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/pid_control_op failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 pid_control_op
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:27.809381Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/fot_op failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 fot_op
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:27.810257Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/obstacle_location_op failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 obstacle_location_op
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:27.817554Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/plot failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 plot
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:27.836609Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/carla_gps_op failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 carla_gps_op
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:28.199109Z ERROR dora_daemon:
bf70dd14-a2a5-41ba-a6be-986a752ceb44/yolov5 failed with exit code 1.
Check logs using: dora logs bf70dd14-a2a5-41ba-a6be-986a752ceb44 yolov5
at binaries/daemon/src/lib.rs:906
2023-06-29T08:20:28.441932Z INFO dora_cli::attach: dataflow bf70dd14-a2a5-41ba-a6be-986a752ceb44 stopped
at binaries/cli/src/attach.rs:132
(dora3.7) jarvis@jia:~/coding/rust_home/github.com/meua/dora-drives$
|
Beta Was this translation helpful? Give feedback.
-
Discussion on Dora Command Line Interface (CLI)
In order to give Dora user/developer good experience, let's discuss ergonomic way of Dora CLI (this is dora user interface, let's explore:).
Below is a initial suggestion for Dora CLI for open discussion:
Usage:
Beta Was this translation helpful? Give feedback.
All reactions