Hugging Face's logo

meiiny00
/
w2v-bert-2.0-15red

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
wav2vec2-bert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
w2v-bert-2.0-15red / README.md
meiiny00's picture
meiiny00
Upload tokenizer
2afa447 verified
preview code
|
raw
history blame contribute delete
3.6 kB
metadata
base_model: facebook/w2v-bert-2.0
library_name: transformers
license: mit
metrics:
  - wer
tags:
  - generated_from_trainer
model-index:
  - name: w2v-bert-2.0-15red
    results: []

w2v-bert-2.0-15red

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2430
  • Wer: 0.1037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • training_steps: 17000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
0.6691 0.4261 500 0.6233 0.5383
0.45 0.8522 1000 0.4130 0.3675
0.3758 1.2782 1500 0.3631 0.3105
0.2521 1.7043 2000 0.3245 0.2777
0.2897 2.1304 2500 0.3024 0.2466
0.23 2.5565 3000 0.2813 0.2389
0.2937 2.9825 3500 0.2713 0.2236
0.1833 3.4086 4000 0.2600 0.2055
0.1375 3.8347 4500 0.2424 0.1910
0.2097 4.2608 5000 0.2376 0.1856
0.1676 4.6868 5500 0.2304 0.1839
0.1268 5.1129 6000 0.2328 0.1687
0.1229 5.5390 6500 0.2274 0.1646
0.1116 5.9651 7000 0.2103 0.1562
0.2322 6.3911 7500 0.2080 0.1540
0.1592 6.8172 8000 0.2151 0.1496
0.0796 7.2433 8500 0.2065 0.1401
0.0774 7.6694 9000 0.2036 0.1373
0.0979 8.0954 9500 0.2109 0.1361
0.0916 8.5215 10000 0.2082 0.1320
0.1057 8.9476 10500 0.2080 0.1294
0.0642 9.3737 11000 0.2032 0.1245
0.0585 9.7997 11500 0.1974 0.1232
0.0531 10.2258 12000 0.2108 0.1203
0.049 10.6519 12500 0.2027 0.1155
0.0431 11.0780 13000 0.2065 0.1152
0.0454 11.5040 13500 0.2167 0.1122
0.0236 11.9301 14000 0.2195 0.1113
0.0313 12.3562 14500 0.2314 0.1080
0.0452 12.7823 15000 0.2231 0.1063
0.0159 13.2084 15500 0.2259 0.1057
0.0166 13.6344 16000 0.2355 0.1043
0.0175 14.0605 16500 0.2340 0.1040
0.0169 14.4866 17000 0.2430 0.1037

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.4.1+cu124
  • Datasets 2.20.0
  • Tokenizers 0.19.1