Is deepspeech nativeclient thread safe

[JRMeyer]

>>> eggonlea
[April 29, 2019, 8:42pm]

If I'm going to convert multiple WAV files simultaneously, there're
different ways to do it: multi-process and multi-thread.

The questions is:

1. Is DS native client thread-safe? E.g. Can I initialize (with LM)
once and then create multiple threads to decode multiple streams? Is
there anything I should lock among threads?

2. If it's not thread-safe, can I initialize once and then fork
multiple processes to decode multiple streams?

If both answers are negative, every new forked process has to perform
the redudant initialization before decoding a new stream. Thanks!

[This is an archived TTS discussion thread from discourse.mozilla.org/t/is-deepspeech-nativeclient-thread-safe]

[JRMeyer]

>>> reuben
[April 29, 2019, 11:08pm]

It should be thread-safe as long as you don't cross the streams (aka
StreamingState slash *). The only catch is that in the GPU-enabled clients,
TensorFlow will allocate the entire GPU RAM by default, so the second
instance will run into OOMs.

For batch decoding of multiple WAV files, take a look at evaluate.py,
which is written for that use case. Maybe it's a better fit for you.

[Archived Post]

[JRMeyer]

>>> eggonlea
[April 30, 2019, 1:13am]

Thanks, Reuben!

It's for something like a web server, which might receive quite a few
requests at the same time. So I'm going to maintain a thread/process
pool to decode multiple streams simultaneously.

Just took a look at evaluate.py, which decodes a list of files in batch
mode. It seems not easy to modify it to support the above use case.

[Archived Post]

[JRMeyer]

>>> lissyx
[May 1, 2019, 9:57am]

You might want to take a look at
mozilla/DeepSpeech#838

[Archived Post]