How to run deepspeech on an existing tensorflow installation

[JRMeyer]

>>> howkong
[February 25, 2021, 11:12am]

I am trying to build a software that runs Deepspeech inference along
with another Tensorflow-based network.

However, when I ran my software, I find that the two tasks are
simutaneously trying to allocate large chunks of memory on all 4 of my
GPUs, starving each other of memory and cause cuDNN runtime errors.

But run the two models on two separate processes, each having 2 visible
GPUs, they run just fine.

Since the deepspeech-gpu pip package doesn't depend on
tensorflow-gpu, I assume that deepspeech-gpu has Tensorflow runtimes
integrated, and the integrated Tensorflow runtime is conflicting with
the Tensorflow that I manually installed via pip.

Is my assumption correct? If it is, is there a way to run Deepspeech on
my existing Tensorflow installation?

[This is an archived TTS discussion thread from discourse.mozilla.org/t/how-to-run-deepspeech-on-an-existing-tensorflow-installation]

[JRMeyer]

>>> lissyx
[February 25, 2021, 11:32am]

> has Tensorflow runtimes integrated

Yes

> the integrated Tensorflow runtime is conflicting with the Tensorflow
> that I manually installed via pip

We have test coverage to avoid that

> starving each other of memory and cause cuDNN runtime errors.

Have you tried the allow growth parameter or passing GPU options to
limit the amount of memory required? Several people reported being able
to share GPUs this way, and this was unrelated to a side tensorflow
install.

But honestly, with that few infos, no log, no versions of what you have
running, we can't really do more.

[Archived Post]

[JRMeyer]

>>> howkong
[February 26, 2021, 5:56am]

As far as I know, allow growth can be only be configured for a specific
TF sesssion. Is there a way to access the TF session on which my
Deepspeech model is running on from the python interface? slash
I am running infrence with deepspeech-gpu 0.9.3 , using the official
deepspeech-0.9.3-models.pbmm model. slash
I ran the
https://github.com/mozilla/DeepSpeech/blob/master/native_client/python/client.py
script, with a few lines added to include my other model.

[Archived Post]

[JRMeyer]

>>> lissyx
[February 26, 2021, 8:51am]

Not when you use libdeepspeech.so, but if you switch to transcribe.py
you can. However on Windows you might be facing other issues.

I think tensorflow allows changing that from env variable, you should
look up in their docs.

[Archived Post]

[JRMeyer]

>>> howkong
[March 1, 2021, 6:24am]

Thank you. Problem solved. slash
I simply added os.environ['TF_FORCE_GPU_ALLOW_GROWTH'] = 'true' to
that client.py script and no more problems occured.

[Archived Post]