README.md

metadata

license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v2-base-pretrained_lr1e-4_at0.8_da0.3
    results: []

w2v2-base-pretrained_lr1e-4_at0.8_da0.3

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.3227
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 200
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
18.4732	7.14	100	3.5073	1.0
3.2701	14.29	200	3.3516	1.0
3.234	21.43	300	3.3874	1.0
3.2196	28.57	400	3.3154	1.0
3.2146	35.71	500	3.3741	1.0
3.1916	42.86	600	3.3402	1.0
3.1749	50.0	700	3.3768	1.0
3.2124	57.14	800	3.2787	1.0
3.2133	64.29	900	3.3424	1.0
3.2101	71.43	1000	3.2885	1.0
3.2047	78.57	1100	3.3397	1.0
3.2137	85.71	1200	3.3420	1.0
3.2107	92.86	1300	3.3670	1.0
3.214	100.0	1400	3.3011	1.0
3.2089	107.14	1500	3.3278	1.0
3.2095	114.29	1600	3.3363	1.0
3.2053	121.43	1700	3.3086	1.0
3.2009	128.57	1800	3.3105	1.0
3.2037	135.71	1900	3.3275	1.0
3.2074	142.86	2000	3.3092	1.0
3.2134	150.0	2100	3.3099	1.0
3.2042	157.14	2200	3.3025	1.0
3.2006	164.29	2300	3.3110	1.0
3.2052	171.43	2400	3.3393	1.0
3.2052	178.57	2500	3.3191	1.0
3.2026	185.71	2600	3.3303	1.0
3.2113	192.86	2700	3.3261	1.0
3.2048	200.0	2800	3.3227	1.0

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1