poonehmousavi commited on
Commit
e8520ec
1 Parent(s): 269ef58

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -36
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
  language:
3
- - fr
4
  thumbnail: null
5
  pipeline_tag: automatic-speech-recognition
6
  tags:
@@ -8,39 +8,38 @@ tags:
8
  - pytorch
9
  - speechbrain
10
  - Transformer
11
- - hf-asr-leaderboard
12
  license: apache-2.0
13
  datasets:
14
- - commonvoice
15
  metrics:
16
  - wer
17
  - cer
18
  model-index:
19
- - name: asr-wav2vec2-commonvoice-fr
20
  results:
21
  - task:
22
  name: Automatic Speech Recognition
23
  type: automatic-speech-recognition
24
  dataset:
25
- name: CommonVoice 6.1 (French)
26
- type: mozilla-foundation/common_voice_6_1
27
- config: fr
28
  split: test
29
  args:
30
- language: fr
31
  metrics:
32
  - name: Test WER
33
  type: wer
34
- value: '9.96'
35
  ---
36
 
37
  <iframe src="https://ghbtns.com/github-btn.html?user=speechbrain&repo=speechbrain&type=star&count=true&size=large&v=2" frameborder="0" scrolling="0" width="170" height="30" title="GitHub"></iframe>
38
  <br/><br/>
39
 
40
- # wav2vec 2.0 with CTC/Attention trained on CommonVoice French (No LM)
41
 
42
  This repository provides all the necessary tools to perform automatic speech
43
- recognition from an end-to-end system pretrained on CommonVoice (French Language) within
44
  SpeechBrain. For a better experience, we encourage you to learn more about
45
  [SpeechBrain](https://speechbrain.github.io).
46
 
@@ -48,15 +47,15 @@ The performance of the model is the following:
48
 
49
  | Release | Test CER | Test WER | GPUs |
50
  |:-------------:|:--------------:|:--------------:| :--------:|
51
- | 24-08-21 | 3.19 | 9.96 | 2xV100 32GB |
52
 
53
  ## Pipeline description
54
 
55
  This ASR system is composed of 2 different but linked blocks:
56
- - Tokenizer (unigram) that transforms words into subword units and trained with
57
- the train transcriptions (train.tsv) of CommonVoice (FR).
58
- - Acoustic model (wav2vec2.0 + CTC). A pretrained wav2vec 2.0 model ([LeBenchmark/wav2vec2-FR-7K-large](https://huggingface.co/LeBenchmark/wav2vec2-FR-7K-large)) is combined with two DNN layers and finetuned on CommonVoice FR.
59
- The obtained final acoustic representation is given to the CTC greedy decoder.
60
 
61
  The system is trained with recordings sampled at 16kHz (single channel).
62
  The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling *transcribe_file* if needed.
@@ -72,18 +71,21 @@ pip install speechbrain transformers
72
  Please notice that we encourage you to read our tutorials and learn more about
73
  [SpeechBrain](https://speechbrain.github.io).
74
 
75
- ### Transcribing your own audio files (in French)
76
 
77
  ```python
78
  from speechbrain.pretrained import EncoderASR
79
 
80
- asr_model = EncoderASR.from_hparams(source="speechbrain/asr-wav2vec2-commonvoice-fr", savedir="pretrained_models/asr-wav2vec2-commonvoice-fr")
81
- asr_model.transcribe_file('speechbrain/asr-wav2vec2-commonvoice-fr/example-fr.wav')
82
 
83
  ```
84
  ### Inference on GPU
85
  To perform inference on the GPU, add `run_opts={"device":"cuda"}` when calling the `from_hparams` method.
86
 
 
 
 
87
  ### Training
88
  The model was trained with SpeechBrain.
89
  To train it from scratch follow these steps:
@@ -101,30 +103,32 @@ pip install -e .
101
  3. Run Training:
102
  ```bash
103
  cd recipes/CommonVoice/ASR/CTC/
104
- python train_with_wav2vec.py hparams/train_fr_with_wav2vec.yaml --data_folder=your_data_folder
105
  ```
106
 
107
- You can find our training results (models, logs, etc) [here](https://drive.google.com/drive/folders/1T9DfdZwcNI9CURxhLCi8GA5JVz8adiY8?usp=sharing).
108
 
109
  ### Limitations
110
  The SpeechBrain team does not provide any warranty on the performance achieved by this model when used on other datasets.
111
 
112
- #### Referencing SpeechBrain
113
 
114
- ```
115
- @misc{SB2021,
116
- author = {Ravanelli, Mirco and Parcollet, Titouan and Rouhe, Aku and Plantinga, Peter and Rastorgueva, Elena and Lugosch, Loren and Dawalatabad, Nauman and Ju-Chieh, Chou and Heba, Abdel and Grondin, Francois and Aris, William and Liao, Chien-Feng and Cornell, Samuele and Yeh, Sung-Lin and Na, Hwidong and Gao, Yan and Fu, Szu-Wei and Subakan, Cem and De Mori, Renato and Bengio, Yoshua },
117
- title = {SpeechBrain},
118
- year = {2021},
119
- publisher = {GitHub},
120
- journal = {GitHub repository},
121
- howpublished = {\\\\url{https://github.com/speechbrain/speechbrain}},
122
- }
123
- ```
124
 
125
- #### About SpeechBrain
126
- SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.
127
 
128
- Website: https://speechbrain.github.io/
 
129
 
130
- GitHub: https://github.com/speechbrain/speechbrain
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  language:
3
+ - en
4
  thumbnail: null
5
  pipeline_tag: automatic-speech-recognition
6
  tags:
 
8
  - pytorch
9
  - speechbrain
10
  - Transformer
 
11
  license: apache-2.0
12
  datasets:
13
+ - commonvoice.14.0
14
  metrics:
15
  - wer
16
  - cer
17
  model-index:
18
+ - name: asr-wav2vec2-commonvoice-14-en
19
  results:
20
  - task:
21
  name: Automatic Speech Recognition
22
  type: automatic-speech-recognition
23
  dataset:
24
+ name: CommonVoice Corpus 14.0 (English)
25
+ type: mozilla-foundation/common_voice_14.0
26
+ config: en
27
  split: test
28
  args:
29
+ language: en
30
  metrics:
31
  - name: Test WER
32
  type: wer
33
+ value: '16.68'
34
  ---
35
 
36
  <iframe src="https://ghbtns.com/github-btn.html?user=speechbrain&repo=speechbrain&type=star&count=true&size=large&v=2" frameborder="0" scrolling="0" width="170" height="30" title="GitHub"></iframe>
37
  <br/><br/>
38
 
39
+ # wav2vec 2.0 with CTC trained on CommonVoice English (No LM)
40
 
41
  This repository provides all the necessary tools to perform automatic speech
42
+ recognition from an end-to-end system pretrained on CommonVoice (English Language) within
43
  SpeechBrain. For a better experience, we encourage you to learn more about
44
  [SpeechBrain](https://speechbrain.github.io).
45
 
 
47
 
48
  | Release | Test CER | Test WER | GPUs |
49
  |:-------------:|:--------------:|:--------------:| :--------:|
50
+ | 15-08-23 | 7.92 | 16.86 | 1xV100 32GB |
51
 
52
  ## Pipeline description
53
 
54
  This ASR system is composed of 2 different but linked blocks:
55
+ - Tokenizer (unigram) that transforms words into unigrams and trained with
56
+ the train transcriptions (train.tsv) of CommonVoice (en).
57
+ - Acoustic model (wav2vec2.0 + CTC). A pretrained wav2vec 2.0 model ([wav2vec2-large-lv60](https://huggingface.co/facebook/wav2vec2-large-lv60)) is combined with two DNN layers and finetuned on CommonVoice DE.
58
+ The obtained final acoustic representation is given to the CTC decoder.
59
 
60
  The system is trained with recordings sampled at 16kHz (single channel).
61
  The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling *transcribe_file* if needed.
 
71
  Please notice that we encourage you to read our tutorials and learn more about
72
  [SpeechBrain](https://speechbrain.github.io).
73
 
74
+ ### Transcribing your own audio files (in English)
75
 
76
  ```python
77
  from speechbrain.pretrained import EncoderASR
78
 
79
+ asr_model = EncoderASR.from_hparams(source="speechbrain/asr-wav2vec2-commonvoice-14-en", savedir="pretrained_models/asr-wav2vec2-commonvoice-14-en")
80
+ asr_model.transcribe_file("speechbrain/asr-wav2vec2-commonvoice-14-en/example-en.wav")
81
 
82
  ```
83
  ### Inference on GPU
84
  To perform inference on the GPU, add `run_opts={"device":"cuda"}` when calling the `from_hparams` method.
85
 
86
+ ## Parallel Inference on a Batch
87
+ Please, [see this Colab notebook](https://colab.research.google.com/drive/1hX5ZI9S4jHIjahFCZnhwwQmFoGAi3tmu?usp=sharing) to figure out how to transcribe in parallel a batch of input sentences using a pre-trained model.
88
+
89
  ### Training
90
  The model was trained with SpeechBrain.
91
  To train it from scratch follow these steps:
 
103
  3. Run Training:
104
  ```bash
105
  cd recipes/CommonVoice/ASR/CTC/
106
+ python train_with_wav2vec.py hparams/train_en_with_wav2vec.yaml --data_folder=your_data_folder
107
  ```
108
 
109
+ You can find our training results (models, logs, etc) [here](https://www.dropbox.com/sh/ch10cnbhf1faz3w/AACdHFG65LC6582H0Tet_glTa?dl=0).
110
 
111
  ### Limitations
112
  The SpeechBrain team does not provide any warranty on the performance achieved by this model when used on other datasets.
113
 
 
114
 
115
+ # **About SpeechBrain**
116
+ - Website: https://speechbrain.github.io/
117
+ - Code: https://github.com/speechbrain/speechbrain/
118
+ - HuggingFace: https://huggingface.co/speechbrain/
 
 
 
 
 
 
119
 
 
 
120
 
121
+ # **Citing SpeechBrain**
122
+ Please, cite SpeechBrain if you use it for your research or business.
123
 
124
+ ```bibtex
125
+ @misc{speechbrain,
126
+ title={{SpeechBrain}: A General-Purpose Speech Toolkit},
127
+ author={Mirco Ravanelli and Titouan Parcollet and Peter Plantinga and Aku Rouhe and Samuele Cornell and Loren Lugosch and Cem Subakan and Nauman Dawalatabad and Abdelwahab Heba and Jianyuan Zhong and Ju-Chieh Chou and Sung-Lin Yeh and Szu-Wei Fu and Chien-Feng Liao and Elena Rastorgueva and François Grondin and William Aris and Hwidong Na and Yan Gao and Renato De Mori and Yoshua Bengio},
128
+ year={2021},
129
+ eprint={2106.04624},
130
+ archivePrefix={arXiv},
131
+ primaryClass={eess.AS},
132
+ note={arXiv:2106.04624}
133
+ }
134
+ ```