BmiClientSingularity gets stuck

[sverhoeven]

Doing on Cartesius compute node

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
client = BmiClientSingularity('ewatercycle-lisflood-grpc4bmi.sif')

The ipython shell does not return to prompt. When pressing CTRL-C it gives following stacktrace

ipython-input-2-26f470878315> in <module>
----> 1 client = BmiClientSingularity('ewatercycle-lisflood-grpc4bmi.sif')

~/conda/envs/comparison/lib/python3.8/site-packages/grpc4bmi/bmi_client_singularity.py in __init__(self, image, input_dir, output_dir, timeout, extra_volumes)
     84         logging.info(f'Running {image} singularity container on port {port}')
     85         self.container = subprocess.Popen(args, env=env, preexec_fn=os.setsid)
---> 86         super(BmiClientSingularity, self).__init__(BmiClient.create_grpc_channel(port=port, host=host), timeout=timeout)
     87 
     88     def __del__(self):

~/conda/envs/comparison/lib/python3.8/site-packages/grpc4bmi/bmi_grpc_client.py in __init__(self, channel, timeout, stub)
     33             self.stub = bmi_pb2_grpc.BmiServiceStub(c)
     34             future = grpc.channel_ready_future(c)
---> 35             future.result(timeout=timeout)
     36         else:
     37             self.stub = stub
...

It works when starting bmi server using

singularity run ewatercycle-lisflood-grpc4bmi.sif

and connecting to it with

import grpc
from grpc4bmi.bmi_grpc_client import BmiClient

mymodel = BmiClient(grpc.insecure_channel("localhost:55555"))
mymodel.get_output_var_names()
('Discharge',)

[sverhoeven]

This feels similar to issues with Docker image which was resolved by adding a sleep between starting the container and connecting to it.

[sverhoeven]

Released 0.2.9 on pypi with delay, but did not fix issue.

On further investigation when BMI_PORT env var is set to 46113 the server is still running on port 55555.