Fix CI build errors

[phil-opp]

We noticed some CI build errors in #489.

[phil-opp]

@haixuanTao I feel like I'm close to some possible solution here. With the latest commit I'm getting the following error on Linux/macOS:

 📦 Built wheel for abi3 Python ≥ 3.7 to /home/runner/work/dora/dora/target/wheels/dora_rs-0.3.3-cp37-abi3-manylinux_2_34_x86_64.whl
Processing ./target/wheels/dora_rs-0.3.3-cp37-abi3-manylinux_2_34_x86_64.whl
Collecting pyarrow
  Downloading pyarrow-16.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (40.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 MB 53.4 MB/s eta 0:00:00
Collecting numpy>=1.16.6
  Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 90.6 MB/s eta 0:00:00
Installing collected packages: numpy, pyarrow, dora-rs
Successfully installed dora-rs-0.3.3 numpy-1.26.4 pyarrow-16.0.0
Created new Python operator `op_1` at ./test_python_project/op_1
Created new Python operator `op_2` at ./test_python_project/op_2
Created new Python node `node_1` at ./test_python_project/node_1
Created new yaml dataflow `test_python_project` at ./test_python_project
Traceback (most recent call last):
  File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'dora'
Dataflow could not be validated.

Do you know what could cause this ModuleNotFoundError?

[haixuanTao]

The python being called is not the one from the virtual env where dora-rs is installed. By default, it is using the one from the dora-daemon which has been spawned before activating the virtual env. I would recommend either:

dora destroy 
dora up

Or changing the order in which the installation is done.

This is actually very annoying as it can be a bit confusing. But I don't know if there is much we can do now.

I think in the future we should probably do something like

dora run

With ephemeral daemon, and only use

dora up

When we're in a distributed environemnt

[phil-opp]

Thanks a lot for the explanation!

I split up the CLI job into a Rust job and a Python job and used separate dora up commands.

I think in the future we should probably do something like

dora run

Good idea!