Can we use deepspeech for vietnamese speech to text

[JRMeyer]

>>> phanthanhlong7695
[February 8, 2018, 9:42am]

i have alphabet.txt, vocaburary.txt, using kenlm, create lm.binary. slash
But i can not create trie file.

[This is an archived TTS discussion thread from discourse.mozilla.org/t/can-we-use-deepspeech-for-vietnamese-speech-to-text]

[JRMeyer]

>>> lissyx
[February 8, 2018, 10:51am]

Thanks, but you are spamming everywhere with your question, and not
answering when I'm asking details:
mozilla/DeepSpeech#1202 (comment)

So please answer and stop spamming everywhere.

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 8, 2018, 3:03pm]

i am so so sory. slash
i am really need help. slash
this is my fault slash
i am build Vietnamese Language Model. slash
i'm using kenlm to create lm.binary. it's ok. but i can not create trie
file. slash
can u help me? slash
and i am very very sory about that.

[Archived Post]

[JRMeyer]

>>> lissyx
[February 8, 2018, 3:17pm]

Stop being sorry and just document what issue exactly you have with
./generate_trie. In the issue you opened, it was obviously not being
built correctly. Please document exactly your steps so we can help you
![:slight_smile:](

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 9, 2018, 2:43am]

first, i dowloaded generate_trie from
https://github.com/mozilla/DeepSpeech/blob/master/native_client/README.md. slash
second, i use kenlm to create lm.binary . slash
then i run command: slash
./generate_trie .data/alphabet.txt .data/lm.binary .data/vocabulary.txt
.data/trie slash
errror: bash: ./generate_trie: cannot execute binary file: Exec format
error slash
i do not fix that. so can you help me with that?

[Archived Post]

[JRMeyer]

>>> lissyx
[February 9, 2018, 7:24am]

You need to download the binaries:
https://index.taskcluster.net/v1/task/project.deepspeech.deepspeech.native_client.v0.1.1.cpu/artifacts/public/native_client.tar.xz

It contains generate_trie

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 9, 2018, 9:12am]

./generate_trie data/alphabet.txt data/lm.binary data/vocabulary.txt
data/trie

error: Invalid label  slash
Aborted (core dumped)

i think maybe DEEPSPEECH do not support Vietnamese Language
![:cold_sweat:](

[Archived Post]

[JRMeyer]

>>> lissyx
[February 9, 2018, 11:12am]

Invalid label means you lack something. Did you made any change to
alphabet.txt ?

[
[[elpimous_robot
[February 9, 2018, 9:46pm]

Hello, slash
Perhaps you tried to simplify alphabet.txt !! Bad idea !! slash
an example of alphabet.txt :

alphabet.txt :

# Each line in this file represents the Unicode codepoint (UTF-8 encoded)
# associated with a numeric label.
# A line that starts with # is a comment. You can escape it with slash # if you wish
# to use '#' as a label.
# FOR FRENCH LIMITED CORPUS - JUST WORKING IN SOUND PERCEPTION - A BOT WILL ANALYSE RESULTS

a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v

Don't forget blank line... slash
Good luck

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 10, 2018, 3:06am]

My alphabet here

a slash
ă slash
â slash
b slash
c slash
d slash
đ slash
e slash
ê slash
g slash
h slash
i slash
k slash
l slash
m slash
n slash
o slash
ô slash
ơ slash
p slash
q slash
r slash
s slash
t slash
u slash
ư slash
v slash
x slash
y slash
à slash
á slash
ả slash
ã slash
ạ slash
ằ slash
ắ slash
ẳ slash
ẵ slash
ặ slash
ầ slash
ấ slash
ẩ slash
ẫ slash
ậ slash
è slash
é slash
ẻ slash
ẽ slash
ẹ slash
ề slash
ế slash
ể slash
ễ slash
ệ slash
ì slash
í slash
ỉ slash
ĩ slash
ị slash
ò slash
ó slash
ỏ slash
õ slash
ọ slash
ồ slash
ố slash
ổ slash
ỗ slash
ộ slash
ờ slash
ớ slash
ở slash
ỡ slash
ợ slash
ù slash
ú slash
ủ slash
ũ slash
ụ slash
ừ slash
ứ slash
ử slash
ữ slash
ự slash
ỳ slash
ý slash
ỷ slash
ỹ slash
ỵ

[Archived Post]

[JRMeyer]

>>> lissyx
[February 10, 2018, 10:12am]

Okay, so it seems like you have properly added vietnamese characters.
Can you ensure you have all those used in vocab ?

I remember similar issues because of characters encoded in UTF-8 on one
side, and in simple ASCII on the other, yet representing the same symbol
![:slight_smile:](

[Archived Post]

[JRMeyer]

>>> lissyx
[February 10, 2018, 11:16am]

See that
issue mozilla/DeepSpeech#1107, that kind of
tooling would find / help in your case for example.

[
[[elpimous_robot
[February 12, 2018, 8:03pm]

, slash
Hi. slash
Be sure to save your file as utf8 one. slash
ex : gedit - slash > save as - slash > bottom left : encoding UTF-8 (and not
iso-xxx) slash
I had problems with auto save (bad encoding)

[Archived Post]

[JRMeyer]

>>> jaredoptimus1
[February 12, 2018, 8:38pm]

I am having the same issue. I am trying to train Deep Speech for Bangla
language. slash
Invalid label ও slash
Aborted (core dumped) slash
It seems that this error is occurring while reading the vocabulary.txt
file. Because character ও is the first character of vocabulary.txt.

[Archived Post]

[JRMeyer]

>>> lissyx
[February 12, 2018, 8:45pm]

Please check issue linked above, and what I already explained earlier:
you need to find what is missing in your vocabulary.txt

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 21, 2018, 3:35am]

do you fix
that. slash
i See that issue mozilla/DeepSpeech#1107 but
nothing in there

[Archived Post]

[JRMeyer]

>>> lissyx
[February 21, 2018, 10:05am]

At the expense of being repetitive, this issue is just mismatching
between characters used in the training data and in the alphabet. You
should just add the missing ones.

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 21, 2018, 10:31am]

slash
for example: ma, mà, má, mả, mã, mạ slash
slash `, slash ~, ?, . i think this characters can't stand alone, it must be go
with a word? slash
i think so

[Archived Post]

[JRMeyer]

>>> lissyx
[February 21, 2018, 10:42am]

As long as you add those combined characters using proper unicode, it
should work. Make sure you use the same unicode between alphabet and
transcriptions. If you have any error raising about invalid label, it
means we might have something bogus somewhere in our UTF-8 handling, and
thus we will need more infos on how to reproduce.

[Archived Post]

[JRMeyer]

>>> phanthanhlong7695
[February 22, 2018, 2:48am]

slash
i save all file UTF-8. it still error invalid label

[next page
→

[Archived Post]