Releases: techiaith/docker-deepspeech-cy
DeepSpeech 0.9.3 (Mawrth / March 2021)
Dyma ein sgriptiau hyfforddi a gwerthuso adnabod lleferydd Cymraeg i gyd fynd a Mozilla DeepSpeech 0.9.3 ynghyd â modelau ar sail y data ar gael erbyn Mawrth 2021 (21.03).
These are our scripts for training and evaluating Welsh speech recognition to coincide with Mozilla DeepSpeech 0.9.3 as well as models trained on the data available by March 2021 (21.03)
Sgriptiau / Scripts
Mae'r sgriptiau yn hwyluso hyfforddi DeepSpeech gyda data o CommonVoice Mozilla Cymraeg. Er bod maint y data Cymraeg yn CommonVoice yn annigonol ar gyfer hyfforddi adnabod lleferydd Cymraeg cyflawn, mae'r sgriptiau yn llwyddo i ddarparu modelau defnyddiol ac ymarferol drwy ddefnyddio dulliau dysgu trosglwyddol i addasu modelau Saesneg DeepSpeech gyda data Cymraeg.
These scripts enable training DeepSpeech with the Welsh language Mozilla CommonVoice data set. Since the amount of Welsh data in CommonVoice is insufficient for training a complete Welsh language speech recognition engine, these scripts can provide feasible and practical models by using transfer learning methods to adapt Mozilla's English DeepSpeech models with Welsh data
Modelau / Models
Rydym hefyd yn cyhoeddi modelau sydd wedi'u hyfforddi gyda data Mozilla CommonVoice Cymraeg, a chyhoeddwyd ym mis Rhagfyr 2020 gyda 117 awr o recordiadau sain, yn ogystal a maint bach o ddata ychwanegol sydd wedi ei cyfrannu gan ddefnyddwyr gwefan Trawsgrifiwr Ar-lein yr uned technolegau iaith : https://trawsgrifiwr.techiaith.cymru.
This release also contains models trained with the Welsh dataset from Mozilla CommonVoice that was published in December 2020 and contains 117 hours of speech recordings as well with a small additional dataset of validated recordings donated by the first users of Bangor University's Language Technology Unit's online automatic transcription website service (Trawsgrifiwr Ar-lein); https://trawsgrifiwr.techiaith.cymru.
- model acwstig / acoustic model: techiaith_bangor_21.03.pbmm
- model iaith (parth trawsgrifio) / language model (transcription domain): techiaith_bangor_transcribe_21.03.scorer
- model iaith (parth macsen) / language model (macsen/voice assistant domain): techiaith_bangor_macsen_21.03.scorer
Mewn arbrofion syml, mae’r model trawsgrifio yn camddeall tua 11% o eiriau mewn brawddeg, tra bo'r modelau ar gyfer cynorthwyydd personol Macsen yn camddeall 13%. Yn anffodus rydym yn amau bod y gwir ganrannau camgymeriadau yn lawer uwch petai ddata profi gwell ar gael.
In simple evaluations, the models for transcribing exhibit a word error rate of 11%, while models for a Welsh language voice assistant called Macsen have an error rate of 13%. Unfortunately we suspect the true error percentages are a lot higher if better test data were available.
Enghreifftiau Defnydd / Example Usage
Mae enghraifft o sut ddefnyddir y ffeiliau modelau o fewn cynnyrch feddalwedd yn ein projectau Trawsgrifiwr.
Our Trawsgrifiwr (Transcriber) provides an example of how the models can be used in a software project
Yn ogystal a'n project gwasanaeth API adnabod lleferydd ar-lein
As well as our online speech recognition API project
https://github.com/techiaith/docker-deepspeech-cy-server/blob/v20.06/server/wsgi.py
Mae rhagor o enghreifftiau ar gael o ddogfennaeth Mozilla / Further examples are available from the Mozilla website
Python: https://deepspeech.readthedocs.io/en/latest/Python-Examples.html
C/C++: https://deepspeech.readthedocs.io/en/latest/C-Examples.html
Diolch / Thank you
Rydym yn ddiolchgar iawn i Rhoslyn Prys (meddal.com) a ymgymerodd â nifer o ymgyrchoedd torfoli ar sail wirfoddol, i'r Mentrau Iaith, Cyngor Gwynedd, Llyfrgell Genedlaethol Cymru a weithiodd gyda Rhoslyn ar rai o'r ymgyrchoedd hyn, hefyd i Lywodraeth Cymru, ac i liaws o gyfranogwyr ar draws Cymru a thu hwnt sydd wedi cyfrannu eu lleisiau i Common Voice Cymraeg.
We are very grateful to Rhoslyn Prys (meddal.com) who undertook on a voluntary basis a number of crowdsourcing campaigns, to the Mentrau Iaith, Gwynedd Council, the National Library of Wales who worked with Rhoslyn on some of these campaigns, to the Welsh Government, and to the many participants across Wales and beyond who have contributed their voices to the Welsh Common Voice datasets.
DeepSpeech 0.9.1 (Ionawr / January 2021)
Dyma'r sgriptiau a ddefnyddiwyd i hyfforddi a gwerthuso modelau Cymraeg DeepSpeech Mozilla ym mis Ionawr 2021 (21.01) ac i gyd-fynd â DeepSpeech fersiwn 0.9.1
These are the scripts used to train the Mozilla DeepSpeech Welsh models in January 2021 (21.01) and to coincide with DeepSpeech 0.9.1
Mae'r sgriptiau yn defnyddio data CommonVoice Mozilla Cymraeg a chyhoeddwyd ym mis Rhagfyr 2020 sydd yn cynnwys 117 awr o recordiadau sain.
The scripts use Welsh language data from Mozilla CommonVoice that was published in December 2020 and contains 117 hours of speech recordings.
Er bod maint y data Cymraeg yn CommonVoice yn annigonol ar gyfer hyfforddi adnabod lleferydd Cymraeg cyflawn, mae'r sgriptiau yn llwyddo i ddarparu modelau defnyddiol ac ymarferol drwy ddefnyddio dulliau dysgu trosglwyddol i addasu modelau Saesneg DeepSpeech gyda data Cymraeg. Mewn arbrofion syml a chynhelir gan y sgriptiau, mae’r model trawsgrifio yn camddeall tua 11% o eiriau mewn brawddeg, tra bo'r modelau ar gyfer cynorthwyydd personol Macsen yn camddeall 13%. Yn anffodus rydym yn amau bod wir ganrannau camgymeriadau yn lawer uwch petai ddull a data profi gwell ar gael.
Although the amount of Welsh data in CommonVoice is insufficient for training a complete Welsh language speech recognition engine, these scripts still succeed in providing a practical capability by using transfer learning methods to adapt Mozilla DeepSpeech project's English models with Welsh data. In simple evaluations that can be repeated with these scripts, the models for transcribing exhibit a word error rate of 11%, while models for a Welsh language voice assistant called Macsen have an error rate of 13%. Unfortunately we suspect that the true error percentages are a lot higher if better test data and methods were available
Modelau / Models
Mae'r modelau sydd wedi eu hyfforddi gyda'r sgriptiau hyn ac ar gael isod, sef / The trained models with these scripts are available below are
- model acwstig / acoustic model: techiaith_bangor_21.01.pbmm
- model iaith (parth trawsgrifio) / language model (transcription domain): techiaith_bangor_transcribe_21.01.scorer
- model iaith (parth macsen) / language model (macsen/voice assistant domain): techiaith_bangor_macsen_21.01.scorer
Enghreifftiau Defnydd / Example Usage
Mae enghraifft o sut ddefnyddir y ffeiliau modelau o fewn cynnyrch feddalwedd yn ein projectau Trawsgrifiwr.
Our Trawsgrifiwr (Transcriber) provides an example of how the models can be used in a software project
Yn ogystal a'n project gwasanaeth API adnabod lleferydd ar-lein
As well as our online speech recognition API project
https://github.com/techiaith/docker-deepspeech-cy-server/blob/v20.06/server/wsgi.py
Mae rhagor o enghreifftiau ar gael o ddogfennaeth Mozilla / Further examples are available from the Mozilla website
Python: https://deepspeech.readthedocs.io/en/latest/Python-Examples.html
C/C++: https://deepspeech.readthedocs.io/en/latest/C-Examples.html
Diolch / Thank you
Rydym yn ddiolchgar iawn i Rhoslyn Prys (meddal.com) a ymgymerodd â nifer o ymgyrchoedd torfoli ar sail wirfoddol, i'r Mentrau Iaith, Cyngor Gwynedd, Llyfrgell Genedlaethol Cymru a weithiodd gyda Rhoslyn ar rai o'r ymgyrchoedd hyn, hefyd i Lywodraeth Cymru, ac i liaws o gyfranogwyr ar draws Cymru a thu hwnt sydd wedi cyfrannu eu lleisiau i Common Voice Cymraeg.
We are very grateful to Rhoslyn Prys (meddal.com) who undertook on a voluntary basis a number of crowdsourcing campaigns, to the Mentrau Iaith, Gwynedd Council, the National Library of Wales who worked with Rhoslyn on some of these campaigns, to the Welsh Government, and to the many participants across Wales and beyond who have contributed their voices to the Welsh Common Voice datasets.
DeepSpeech 0.8.x (Medi / September 2020)
Dyma'r sgriptiau a ddefnyddiwyd i hyfforddi a gwerthuso modelau DeepSpeech Mozilla a ddefnyddir ar gyfer adnabod lleferydd Cymraeg o fewn y rhaglenni cod agored Macsen a'r Trawsgrifiwr Cymraeg ym mis Medi 2020 (20.09) ac i gydfynd â DeepSpeech fersiwn 0.8.x
These are the scripts used to train the Mozilla DeepSpeech models used for Welsh language speech recognition in the open source Macsen - Welsh Language Voice Assistant App and Welsh Transcriber applications in September 2020 (20.09) and to coincide with DeepSpeech 0.8.x
Mae'r sgriptiau yn defnyddio data CommonVoice Mozilla Cymraeg a chyhoeddwyd ym mis Mehefin 2020 sydd yn cynnwys 91 awr o recordiadau sain.
The scripts use Welsh language data from Mozilla CommonVoice that was published in June 2020 and contains 91 hours of speech recordings.
Er bod maint y data Cymraeg yn CommonVoice yn annigonol ar gyfer hyfforddi adnabod lleferydd Cymraeg cyflawn, mae'r sgriptiau yn llwyddo i ddarparu modelau defnyddiol ac ymarferol drwy ddefnyddio dulliau dysgu trosglwyddol i addasu modelau Saesneg DeepSpeech gyda data Cymraeg. Mewn arbrofion syml a chynhelir gan y sgriptiau, mae’r model trawsgrifio yn camddeall tua 24% o eiriau mewn brawddeg, tra bo'r modelau ar gyfer cynorthwyydd Macsen yn camddeall 13%.
Although the amount of Welsh data in CommonVoice is insufficient for training a complete Welsh language speech recognition engine, these scripts still succeed in providing a practical capability by using transfer learning methods to adapt DeepSpeech project's English models with Welsh data. In simple evaluations that can be repeated with these scripts, the models for transcribing exhibit a word error rate of 24%, while our voice assistant Macsen models have an error rate of 13%.
Modelau / Models
Mae'r modelau sydd wedi eu hyfforddi gyda'r sgriptiau hyn ac ar gael isod, sef / The trained models with these scripts are available below are
- model acwstig / acoustic model: techiaith_bangor_20.09.pbmm
- model iaith (parth trawsgrifio) / language model (transcription domain): techiaith_bangor_transcribe_20.09.scorer
- model iaith (parth macsen) / language model (macsen/voice assistant domain): techiaith_bangor_macsen_20.09.scorer
Enghreifftiau Defnydd / Example Usage
Mae enghraifft o sut ddefnyddir y ffeiliau modelau o fewn cynnyrch feddalwedd yn ein projectau Trawsgrifiwr.
Our Trawsgrifiwr (Transcriber) provides an example of how the models can be used in a software project
Yn ogystal a'n project gwasanaeth API adnabod lleferydd ar-lein
As well as our online speech recognition API project
https://github.com/techiaith/docker-deepspeech-cy-server/blob/v20.06/server/wsgi.py
Mae rhagor o enghreifftiau ar gael o ddogfennaeth Mozilla / Further examples are available from the Mozilla website
Python: https://deepspeech.readthedocs.io/en/latest/Python-Examples.html
C/C++: https://deepspeech.readthedocs.io/en/latest/C-Examples.html
Diolch / Thank you
Rydym yn ddiolchgar iawn i Rhoslyn Prys (meddal.com) a ymgymerodd â nifer o ymgyrchoedd torfoli ar sail wirfoddol, i'r Mentrau Iaith, Cyngor Gwynedd, Llyfrgell Genedlaethol Cymru a weithiodd gyda Rhoslyn ar rai o'r ymgyrchoedd hyn, hefyd i Lywodraeth Cymru, ac i liaws o gyfranogwyr ar draws Cymru a thu hwnt sydd wedi cyfrannu eu lleisiau i Common Voice Cymraeg.
We are very grateful to Rhoslyn Prys (meddal.com) who undertook on a voluntary basis a number of crowdsourcing campaigns, to the Mentrau Iaith, Gwynedd Council, the National Library of Wales who worked with Rhoslyn on some of these campaigns, to the Welsh Government, and to the many participants across Wales and beyond who have contributed their voices to the Welsh Common Voice datasets.
DeepSpeech 0.7.x (Gorffennaf / July 2020)
Dyma'r sgriptiau a ddefnyddiwyd i hyfforddi a gwerthuso modelau DeepSpeech Mozilla a ddefnyddir ar gyfer adnabod lleferydd Cymraeg o fewn y rhaglenni cod agored Macsen a'r Trawsgrifiwr Cymraeg ym mis Gorffennaf 2020 (20.07) ac i gydfynd â DeepSpeech fersiwn 0.7.x
These are the scripts used to train the Mozilla DeepSpeech models used for Welsh language speech recognition in the open source Macsen - Welsh Language Voice Assistant App and Welsh Transcriber applications in July 2020 (20.07) and to coincide with DeepSpeech 0.7.x
Mae'r sgriptiau yn defnyddio data CommonVoice Mozilla Cymraeg a chyhoeddwyd ym mis Mehefin 2020 sydd yn cynnwys 91 awr o recordiadau sain.
The scripts use Welsh language data from Mozilla CommonVoice that was published in June 2020 and contains 91 hours of speech recordings.
Er bod maint y data Cymraeg yn CommonVoice yn annigonol ar gyfer hyfforddi adnabod lleferydd Cymraeg cyflawn, mae'r sgriptiau yn llwyddo i ddarparu modelau defnyddiol ac ymarferol drwy ddefnyddio dulliau dysgu trosglwyddol i addasu modelau Saesneg DeepSpeech gyda data Cymraeg. Mewn arbrofion syml a chynhelir gan y sgriptiau, mae’r model trawsgrifio yn camddeall tua 24% o eiriau mewn brawddeg, tra bo'r modelau ar gyfer cynorthwyydd Macsen yn camddeall 13%.
Although the amount of Welsh data in CommonVoice is insufficient for training a complete Welsh language speech recognition engine, these scripts still succeed in providing a practical capability by using transfer learning methods to adapt DeepSpeech project's English models with Welsh data. In simple evaluations that can be repeated with these scripts, the models for transcribing exhibit a word error rate of 24%, while our voice assistant Macsen models have an error rate of 13%.
Modelau / Models
Mae'r modelau sydd wedi eu hyfforddi gyda'r sgriptiau hyn ac ar gael isod, sef / The trained models with these scripts are available below are
- model acwstig / acoustic model: techiaith_bangor_20.07.pbmm
- model iaith (parth trawsgrifio) / language model (transcription domain): techiaith_bangor_transcribe_20.07.scorer
- model iaith (parth macsen) / language model (macsen/voice assistant domain): techiaith_bangor_macsen_20.07.scorer
Enghreifftiau Defnydd / Example Usage
Mae enghraifft o sut ddefnyddir y ffeiliau modelau o fewn cynnyrch feddalwedd yn ein projectau Trawsgrifiwr.
Our Trawsgrifiwr (Transcriber) provides an example of how the models can be used in a software project
Yn ogystal a'n project gwasanaeth API adnabod lleferydd ar-lein
As well as our online speech recognition API project
https://github.com/techiaith/docker-deepspeech-cy-server/blob/v20.06/server/wsgi.py
Mae rhagor o enghreifftiau ar gael o ddogfennaeth Mozilla / Further examples are available from the Mozilla website
Python: https://deepspeech.readthedocs.io/en/latest/Python-Examples.html
C/C++: https://deepspeech.readthedocs.io/en/latest/C-Examples.html
Diolch / Thank you
Rydym yn ddiolchgar iawn i Rhoslyn Prys (meddal.com) a ymgymerodd â nifer o ymgyrchoedd torfoli ar sail wirfoddol, i'r Mentrau Iaith, Cyngor Gwynedd, Llyfrgell Genedlaethol Cymru a weithiodd gyda Rhoslyn ar rai o'r ymgyrchoedd hyn, hefyd i Lywodraeth Cymru, ac i liaws o gyfranogwyr ar draws Cymru a thu hwnt sydd wedi cyfrannu eu lleisiau i Common Voice Cymraeg.
We are very grateful to Rhoslyn Prys (meddal.com) who undertook on a voluntary basis a number of crowdsourcing campaigns, to the Mentrau Iaith, Gwynedd Council, the National Library of Wales who worked with Rhoslyn on some of these campaigns, to the Welsh Government, and to the many participants across Wales and beyond who have contributed their voices to the Welsh Common Voice datasets.
DeepSpeech 0.7.x
Dyma'r sgriptiau a ddefnyddiwyd i hyfforddi a gwerthuso modelau DeepSpeech Mozilla a ddefnyddir ar gyfer adnabod lleferydd Cymraeg o fewn y rhaglenni cod agored Macsen a'r Trawsgrifiwr Cymraeg ym mis Mehefin 2020 (20.06) ac i gydfynd â DeepSpeech fersiwn 0.7.x
These are the scripts used to train the Mozilla DeepSpeech models used for Welsh language speech recognition in the open source Macsen - Welsh Language Voice Assistant App and Welsh Transcriber applications in June 2020 (20.06) and to coincide with DeepSpeech 0.7.x
Mae'r sgriptiau yn defnyddio data CommonVoice Mozilla Cymraeg a chyhoeddwyd ym mis Rhagfyr 2019 sydd yn cynnwys 77 awr o recordiadau sain. Er bod maint y data Cymraeg yn CommonVoice yn annigonol ar gyfer hyfforddi adnabod lleferydd Cymraeg cyflawn, mae'r sgriptiau yn llwyddo i ddarparu modelau defnyddiol ac ymarferol drwy ddefnyddio dulliau dysgu trosglwyddol i addasu modelau Saesneg DeepSpeech gyda data Cymraeg. Mewn arbrofion syml a chynhelir gan y sgriptiau, mae’r model trawsgrifio yn camddeall tua 27% o eiriau mewn brawddeg, tra bo'r modelau ar gyfer cynorthwyydd Macsen yn camddeall 14%.
The scripts use Welsh language data from Mozilla CommonVoice that was published in December 2019 and contains 77 hours of speech recordings. Although the amount of Welsh data in CommonVoice is insufficient for training a complete Welsh language speech recognition engine, these scripts still succeed in providing a practical capability by using transfer learning methods to adapt DeepSpeech project's English models with Welsh data. In simple evaluations that can be repeated with these scripts, the models for transcribing exhibit a word error rate of 27%, while our voice assistant Macsen models have an error rate of 14%.
Modelau / Models
Mae'r modelau sydd wedi eu hyfforddi gyda'r sgriptiau hyn ac ar gael isod, sef / The trained models with these scripts are available below are
- model acwstig / acoustic model: techiaith_bangor_20.06.pbmm
- model iaith (parth trawsgrifio) / language model (transcription domain): techiaith_bangor_transcribe_20.06.scorer
- model iaith (parth macsen) / language model (macsen/voice assistant domain): techiaith_bangor_macsen_20.06.scorer
Enghreifftiau Defnydd / Example Usage
Mae enghraifft o sut ddefnyddir y ffeiliau modelau o fenw cynnyrch feddalwedd yn ein projectau Trawsgrifiwr.
Our Trawsgrifiwr (Transcriber) provides an example of how the models can be used in a software project
Yn ogystal a'n project gwasanaeth API adnabod lleferydd ar-lein
As well as our online speech recognition API project
https://github.com/techiaith/docker-deepspeech-cy-server/blob/v20.06/server/wsgi.py
Mae rhagor o enghreifftiau ar gael o ddogfennaeth Mozilla / Further examples are available from the Mozilla website
Python: https://deepspeech.readthedocs.io/en/latest/Python-Examples.html
C/C++: https://deepspeech.readthedocs.io/en/latest/C-Examples.html
Diolch / Thank you
Rydym yn ddiolchgar iawn i Rhoslyn Prys (meddal.com) a ymgymerodd â nifer o ymgyrchoedd torfoli ar sail wirfoddol, i'r Mentrau Iaith, Cyngor Gwynedd, Llyfrgell Genedlaethol Cymru a weithiodd gyda Rhoslyn ar rai o'r ymgyrchoedd hyn, hefyd i Lywodraeth Cymru, ac i liaws o gyfranogwyr ar draws Cymru a thu hwnt sydd wedi cyfrannu eu lleisiau i Common Voice Cymraeg.
We are very grateful to Rhoslyn Prys (meddal.com) who undertook on a voluntary basis a number of crowdsourcing campaigns, to the Mentrau Iaith, Gwynedd Council, the National Library of Wales who worked with Rhoslyn on some of these campaigns, to the Welsh Government, and to the many participants across Wales and beyond who have contributed their voices to the Welsh Common Voice datasets.
Arddweud a Macsen
Transcription and Macsen
Dyma'r sgriptiau a ddefnyddiwyd i hyfforddi'r modelau DeepSpeech Mozilla a ddefnyddir ar gyfer adnabod lleferydd Cymraeg o fewn y rhaglenni cod agored Macsen a'r Trawsgrifiwr Cymraeg ym mis Mawrth 2020 (20.03)
These are the scripts used to train the Mozilla DeepSpeech models used for Welsh language speech recognition in the open source Macsen - Welsh Language Voice Assistant App and Welsh Transcriber applications in March 2020 (20.03)
Mae'r angen llwytho i lawr y ddata hyfforddi a phrofi ar wahân o CommonVoice ac o'r porth technolegau iaith. Y data dylid llwytho i lawr o'r porth technolegau iaith yw:
-
set profi Macsen Macsen test set:
http://techiaith.cymru/deepspeech/macsen/datasets/macsen_200121.tar.gz -
set profi Arddweud Transcription test set :
http://techiaith.cymru/deepspeech/arddweud/datasets/arddweud_testset_200303.tar.gz
Er bod maint y data Cymraeg yn CommonVoice yn annigonol ar gyfer hyfforddi adnabod lleferydd Cymraeg cyflawn, mae'r sgriptiau yn llwyddo i ddarparu modelau defnyddiol ac ymarferol drwy ddefnyddio dulliau dysgu trosglwyddol i addasu modelau Saesneg DeepSpeech gyda data Cymraeg.
Although the amount of Welsh data in CommonVoice is insufficient for training a complete Welsh language speech recognition engine, these scripts still succeed in providing a practical capability by using transfer learning methods to adapt DeepSpeech project's English models with Welsh data
Mae'r modelau wedi eu hyfforddi ar gael o / The trained models are available from