Using the `WANDB_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none).
06/10/2024 22:36:58 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: False
06/10/2024 22:36:58 - INFO - __main__ - Training/evaluation parameters Seq2SeqTrainingArguments(
_n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=False,
bf16_full_eval=False,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=0,
dataloader_persistent_workers=False,
dataloader_pin_memory=True,
ddp_backend=None,
ddp_broadcast_buffers=None,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=None,
ddp_timeout=1800,
debug=[],
deepspeed=None,
disable_tqdm=False,
dispatch_batches=None,
do_eval=False,
do_predict=True,
do_train=False,
eval_accumulation_steps=None,
eval_delay=0,
eval_steps=None,
evaluation_strategy=no,
fp16=False,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
generation_config=None,
generation_max_length=None,
generation_num_beams=2,
gradient_accumulation_steps=1,
gradient_checkpointing=False,
gradient_checkpointing_kwargs=None,
greater_is_better=None,
group_by_length=False,
half_precision_backend=auto,
hub_always_push=False,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
include_num_input_tokens_seen=False,
include_tokens_per_second=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=5e-05,
length_column_name=input_length,
load_best_model_at_end=False,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=/beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407/eval/NLU/runs/Jun10_22-36-57_tholus-7.int.europe.naverlabs.com,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=500,
logging_strategy=steps,
lr_scheduler_kwargs={},
lr_scheduler_type=linear,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
neftune_noise_alpha=None,
no_cuda=False,
num_train_epochs=3.0,
optim=adamw_torch,
optim_args=None,
output_dir=/beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407/eval/NLU,
overwrite_output_dir=False,
past_index=-1,
per_device_eval_batch_size=32,
per_device_train_batch_size=8,
predict_with_generate=True,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=/beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407/eval/NLU,
save_on_each_node=False,
save_only_model=False,
save_safetensors=True,
save_steps=500,
save_strategy=steps,
save_total_limit=None,
seed=42,
skip_memory_metrics=True,
sortish_sampler=False,
split_batches=False,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_cpu=False,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.0,
warmup_steps=0,
weight_decay=0.0,
)
Loading Dataset Infos from /beegfs/scratch/user/blee/hugging-face/models/modules/datasets_modules/datasets/massive_slu/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617
06/10/2024 22:36:58 - INFO - datasets.info - Loading Dataset Infos from /beegfs/scratch/user/blee/hugging-face/models/modules/datasets_modules/datasets/massive_slu/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617
Overwrite dataset info from restored data version if exists.
06/10/2024 22:36:58 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
Loading Dataset info from /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617
06/10/2024 22:36:58 - INFO - datasets.info - Loading Dataset info from /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617
Found cached dataset massive_slu (/beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617)
06/10/2024 22:36:58 - INFO - datasets.builder - Found cached dataset massive_slu (/beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617)
Loading Dataset info from /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617
06/10/2024 22:36:58 - INFO - datasets.info - Loading Dataset info from /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617
[INFO|configuration_utils.py:737] 2024-06-10 22:36:58,618 >> loading configuration file /beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407/config.json
[INFO|configuration_utils.py:802] 2024-06-10 22:36:58,633 >> Model config MT5Config {
  "_name_or_path": "/beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407",
  "architectures": [
    "MT5ForConditionalGeneration"
  ],
  "classifier_dropout": 0.0,
  "d_ff": 2048,
  "d_kv": 64,
  "d_model": 768,
  "decoder_start_token_id": 0,
  "dense_act_fn": "gelu_new",
  "dropout": 0.2,
  "dropout_rate": 0.1,
  "eos_token_id": 1,
  "feed_forward_proj": "gated-gelu",
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "is_gated_act": true,
  "layer_norm_epsilon": 1e-06,
  "model_type": "mt5",
  "num_decoder_layers": 12,
  "num_heads": 12,
  "num_layers": 12,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_max_distance": 128,
  "relative_attention_num_buckets": 32,
  "tie_word_embeddings": false,
  "tokenizer_class": "T5Tokenizer",
  "torch_dtype": "float32",
  "transformers_version": "4.37.0.dev0",
  "use_cache": true,
  "vocab_size": 250112
}

[INFO|tokenization_utils_base.py:2024] 2024-06-10 22:36:58,656 >> loading file spiece.model
[INFO|tokenization_utils_base.py:2024] 2024-06-10 22:36:58,658 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2024] 2024-06-10 22:36:58,660 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2024] 2024-06-10 22:36:58,662 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2024] 2024-06-10 22:36:58,665 >> loading file tokenizer_config.json
[INFO|modeling_utils.py:3373] 2024-06-10 22:36:59,181 >> loading weights file /beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407/model.safetensors
[INFO|configuration_utils.py:826] 2024-06-10 22:36:59,353 >> Generate config GenerationConfig {
  "decoder_start_token_id": 0,
  "eos_token_id": 1,
  "pad_token_id": 0
}

[INFO|modeling_utils.py:4224] 2024-06-10 22:37:04,608 >> All model checkpoint weights were used when initializing MT5ForConditionalGeneration.

[INFO|modeling_utils.py:4232] 2024-06-10 22:37:04,610 >> All the weights of MT5ForConditionalGeneration were initialized from the model checkpoint at /beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407.
If your task is similar to the task the model of the checkpoint was trained on, you can already use MT5ForConditionalGeneration for predictions without further training.
[INFO|configuration_utils.py:779] 2024-06-10 22:37:04,625 >> loading configuration file /beegfs/scratch/user/blee/project_3/models/NLU.mt5-base.task_type-1.fine_tune.gpu_a100-40g+.node-1x1.bsz-64.epochs-22.metric-ema.metric_lang-all/checkpoint-30407/generation_config.json
[INFO|configuration_utils.py:826] 2024-06-10 22:37:04,627 >> Generate config GenerationConfig {
  "decoder_start_token_id": 0,
  "eos_token_id": 1,
  "pad_token_id": 0
}


Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-525dddbb93e613dc.arrow
06/10/2024 22:37:04 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-525dddbb93e613dc.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 10152.18 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 9881.17 examples/s] 

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-0ff2f782f6ce550d.arrow
06/10/2024 22:37:05 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-0ff2f782f6ce550d.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 23379.87 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 22426.85 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-db517c5fbde3f93f.arrow
06/10/2024 22:37:05 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-db517c5fbde3f93f.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 24733.67 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 23464.22 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-b1a23ce3c266ed7e.arrow
06/10/2024 22:37:05 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-b1a23ce3c266ed7e.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 11672.89 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 11022.17 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-d14f4edcd32d40f9.arrow
06/10/2024 22:37:05 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-d14f4edcd32d40f9.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 12128.52 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 11483.51 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-c32b0d4edcd745d1.arrow
06/10/2024 22:37:06 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-c32b0d4edcd745d1.arrow

Running tokenizer on prediction dataset:  34%|███▎      | 1000/2974 [00:00<00:00, 7372.97 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 13353.47 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-65b7a58a8f4b62ba.arrow
06/10/2024 22:37:06 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-65b7a58a8f4b62ba.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 23271.39 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 21331.30 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-8571e8bc453b7162.arrow
06/10/2024 22:37:06 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-8571e8bc453b7162.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 23459.63 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 18122.89 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-c74febb30a64318a.arrow
06/10/2024 22:37:06 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-c74febb30a64318a.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 11319.17 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 10715.31 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-4c2b08ae5bc14226.arrow
06/10/2024 22:37:06 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-4c2b08ae5bc14226.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 24163.00 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 20604.87 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-5a36e19b80c96113.arrow
06/10/2024 22:37:07 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-5a36e19b80c96113.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 20147.34 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 12138.36 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-b7c60de80d7c7840.arrow
06/10/2024 22:37:07 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-b7c60de80d7c7840.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 25152.92 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 22962.73 examples/s]

Running tokenizer on prediction dataset:   0%|          | 0/2974 [00:00<?, ? examples/s]Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-ae561d58d8bc6128.arrow
06/10/2024 22:37:07 - INFO - datasets.arrow_dataset - Caching processed dataset at /beegfs/scratch/user/blee/hugging-face/models/datasets/massive_slu/multilingual-test/1.0.0/f9c095e36aa8a498aff90ba642b0b428b56191e41f2c80e78e378689cdb36617/cache-ae561d58d8bc6128.arrow

Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 26368.84 examples/s]
Running tokenizer on prediction dataset: 100%|██████████| 2974/2974 [00:00<00:00, 23629.43 examples/s]
06/10/2024 22:37:09 - WARNING - accelerate.utils.other - Detected kernel version 4.18.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
06/10/2024 22:37:10 - INFO - __main__ - *** Predict ***
06/10/2024 22:37:10 - INFO - __main__ - *** test_en_US ***
[INFO|trainer.py:718] 2024-06-10 22:37:10,687 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:37:10,695 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:37:10,696 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:37:10,697 >>   Batch size = 32
[WARNING|logging.py:314] 2024-06-10 22:37:10,704 >> You're using a T5TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:20,  4.34it/s]
  3%|▎         | 3/93 [00:00<00:25,  3.51it/s]
  4%|▍         | 4/93 [00:01<00:27,  3.20it/s]
  5%|▌         | 5/93 [00:01<00:32,  2.72it/s]
  6%|▋         | 6/93 [00:02<00:32,  2.71it/s]
  8%|▊         | 7/93 [00:02<00:31,  2.69it/s]
  9%|▊         | 8/93 [00:02<00:32,  2.61it/s]
 10%|▉         | 9/93 [00:03<00:33,  2.52it/s]
 11%|█         | 10/93 [00:03<00:33,  2.51it/s]
 12%|█▏        | 11/93 [00:04<00:33,  2.44it/s]
 13%|█▎        | 12/93 [00:04<00:32,  2.46it/s]
 14%|█▍        | 13/93 [00:04<00:33,  2.41it/s]
 15%|█▌        | 14/93 [00:05<00:37,  2.12it/s]
 16%|█▌        | 15/93 [00:05<00:36,  2.11it/s]
 17%|█▋        | 16/93 [00:06<00:34,  2.25it/s]
 18%|█▊        | 17/93 [00:06<00:34,  2.18it/s]
 19%|█▉        | 18/93 [00:07<00:33,  2.26it/s]
 20%|██        | 19/93 [00:07<00:33,  2.19it/s]
 22%|██▏       | 20/93 [00:08<00:32,  2.23it/s]
 23%|██▎       | 21/93 [00:08<00:32,  2.21it/s]
 24%|██▎       | 22/93 [00:09<00:33,  2.14it/s]
 25%|██▍       | 23/93 [00:09<00:33,  2.10it/s]
 26%|██▌       | 24/93 [00:10<00:31,  2.17it/s]
 27%|██▋       | 25/93 [00:10<00:30,  2.19it/s]
 28%|██▊       | 26/93 [00:10<00:30,  2.19it/s]
 29%|██▉       | 27/93 [00:11<00:28,  2.30it/s]
 30%|███       | 28/93 [00:11<00:26,  2.41it/s]
 31%|███       | 29/93 [00:12<00:26,  2.44it/s]
 32%|███▏      | 30/93 [00:12<00:25,  2.43it/s]
 33%|███▎      | 31/93 [00:13<00:27,  2.25it/s]
 34%|███▍      | 32/93 [00:13<00:26,  2.33it/s]
 35%|███▌      | 33/93 [00:13<00:26,  2.26it/s]
 37%|███▋      | 34/93 [00:14<00:27,  2.15it/s]
 38%|███▊      | 35/93 [00:14<00:25,  2.26it/s]
 39%|███▊      | 36/93 [00:15<00:23,  2.45it/s]
 40%|███▉      | 37/93 [00:15<00:22,  2.52it/s]
 41%|████      | 38/93 [00:16<00:23,  2.32it/s]
 42%|████▏     | 39/93 [00:16<00:22,  2.37it/s]
 43%|████▎     | 40/93 [00:16<00:22,  2.39it/s]
 44%|████▍     | 41/93 [00:17<00:22,  2.32it/s]
 45%|████▌     | 42/93 [00:17<00:22,  2.25it/s]
 46%|████▌     | 43/93 [00:18<00:23,  2.10it/s]
 47%|████▋     | 44/93 [00:18<00:23,  2.10it/s]
 48%|████▊     | 45/93 [00:19<00:21,  2.22it/s]
 49%|████▉     | 46/93 [00:19<00:20,  2.27it/s]
 51%|█████     | 47/93 [00:20<00:20,  2.20it/s]
 52%|█████▏    | 48/93 [00:20<00:19,  2.28it/s]
 53%|█████▎    | 49/93 [00:20<00:18,  2.41it/s]
 54%|█████▍    | 50/93 [00:21<00:20,  2.13it/s]
 55%|█████▍    | 51/93 [00:22<00:20,  2.02it/s]
 56%|█████▌    | 52/93 [00:22<00:21,  1.95it/s]
 57%|█████▋    | 53/93 [00:23<00:19,  2.00it/s]
 58%|█████▊    | 54/93 [00:23<00:20,  1.95it/s]
 59%|█████▉    | 55/93 [00:23<00:18,  2.10it/s]
 60%|██████    | 56/93 [00:24<00:15,  2.33it/s]
 61%|██████▏   | 57/93 [00:24<00:16,  2.22it/s]
 62%|██████▏   | 58/93 [00:25<00:14,  2.35it/s]
 63%|██████▎   | 59/93 [00:25<00:14,  2.28it/s]
 65%|██████▍   | 60/93 [00:26<00:15,  2.17it/s]
 66%|██████▌   | 61/93 [00:26<00:13,  2.34it/s]
 67%|██████▋   | 62/93 [00:27<00:14,  2.19it/s]
 68%|██████▊   | 63/93 [00:27<00:13,  2.27it/s]
 69%|██████▉   | 64/93 [00:29<00:26,  1.10it/s]
 70%|██████▉   | 65/93 [00:29<00:21,  1.33it/s]
 71%|███████   | 66/93 [00:30<00:17,  1.55it/s]
 72%|███████▏  | 67/93 [00:30<00:15,  1.73it/s]
 73%|███████▎  | 68/93 [00:31<00:15,  1.66it/s]
 74%|███████▍  | 69/93 [00:31<00:13,  1.80it/s]
 75%|███████▌  | 70/93 [00:32<00:11,  1.93it/s]
 76%|███████▋  | 71/93 [00:32<00:10,  2.04it/s]
 77%|███████▋  | 72/93 [00:33<00:10,  2.03it/s]
 78%|███████▊  | 73/93 [00:33<00:09,  2.15it/s]
 80%|███████▉  | 74/93 [00:33<00:08,  2.11it/s]
 81%|████████  | 75/93 [00:34<00:07,  2.31it/s]
 82%|████████▏ | 76/93 [00:34<00:07,  2.16it/s]
 83%|████████▎ | 77/93 [00:35<00:07,  2.19it/s]
 84%|████████▍ | 78/93 [00:35<00:07,  2.11it/s]
 85%|████████▍ | 79/93 [00:36<00:07,  1.91it/s]
 86%|████████▌ | 80/93 [00:37<00:07,  1.80it/s]
 87%|████████▋ | 81/93 [00:37<00:06,  1.97it/s]
 88%|████████▊ | 82/93 [00:37<00:05,  2.16it/s]
 89%|████████▉ | 83/93 [00:38<00:04,  2.20it/s]
 90%|█████████ | 84/93 [00:38<00:04,  2.06it/s]
 91%|█████████▏| 85/93 [00:39<00:03,  2.03it/s]
 92%|█████████▏| 86/93 [00:39<00:03,  2.16it/s]
 94%|█████████▎| 87/93 [00:40<00:02,  2.17it/s]
 95%|█████████▍| 88/93 [00:40<00:02,  2.13it/s]
 96%|█████████▌| 89/93 [00:41<00:01,  2.16it/s]
 97%|█████████▋| 90/93 [00:41<00:01,  2.20it/s]
 98%|█████████▊| 91/93 [00:42<00:00,  2.08it/s]
 99%|█████████▉| 92/93 [00:42<00:00,  2.17it/s]
100%|██████████| 93/93 [00:42<00:00,  2.18it/s]
100%|██████████| 93/93 [00:43<00:00,  2.14it/s]
***** predict_test_en_US metrics *****
  predict_ex_match_acc         =     0.7317
  predict_ex_match_acc_stderr  =     0.0081
  predict_intent_acc           =     0.8894
  predict_intent_acc_stderr    =     0.0058
  predict_loss                 =     0.1316
  predict_runtime              = 0:00:44.50
  predict_samples              =       2974
  predict_samples_per_second   =     66.819
  predict_slot_micro_f1        =     0.8224
  predict_slot_micro_f1_stderr =     0.0027
  predict_steps_per_second     =      2.089
06/10/2024 22:37:55 - INFO - __main__ - *** test_es_ES ***
[INFO|trainer.py:718] 2024-06-10 22:37:55,406 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:37:55,409 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:37:55,409 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:37:55,410 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:22,  3.96it/s]
  3%|▎         | 3/93 [00:00<00:27,  3.25it/s]
  4%|▍         | 4/93 [00:01<00:29,  3.01it/s]
  5%|▌         | 5/93 [00:01<00:35,  2.46it/s]
  6%|▋         | 6/93 [00:02<00:36,  2.41it/s]
  8%|▊         | 7/93 [00:02<00:35,  2.44it/s]
  9%|▊         | 8/93 [00:03<00:35,  2.43it/s]
 10%|▉         | 9/93 [00:03<00:37,  2.24it/s]
 11%|█         | 10/93 [00:04<00:36,  2.27it/s]
 12%|█▏        | 11/93 [00:04<00:35,  2.32it/s]
 13%|█▎        | 12/93 [00:04<00:34,  2.33it/s]
 14%|█▍        | 13/93 [00:05<00:35,  2.27it/s]
 15%|█▌        | 14/93 [00:06<00:40,  1.93it/s]
 16%|█▌        | 15/93 [00:06<00:39,  1.96it/s]
 17%|█▋        | 16/93 [00:07<00:39,  1.97it/s]
 18%|█▊        | 17/93 [00:07<00:40,  1.90it/s]
 19%|█▉        | 18/93 [00:08<00:37,  1.98it/s]
 20%|██        | 19/93 [00:08<00:38,  1.94it/s]
 22%|██▏       | 20/93 [00:09<00:36,  2.02it/s]
 23%|██▎       | 21/93 [00:09<00:37,  1.93it/s]
 24%|██▎       | 22/93 [00:10<00:36,  1.93it/s]
 25%|██▍       | 23/93 [00:10<00:35,  1.99it/s]
 26%|██▌       | 24/93 [00:10<00:32,  2.12it/s]
 27%|██▋       | 25/93 [00:12<01:01,  1.11it/s]
 28%|██▊       | 26/93 [00:13<00:51,  1.30it/s]
 29%|██▉       | 27/93 [00:13<00:42,  1.55it/s]
 30%|███       | 28/93 [00:14<00:37,  1.73it/s]
 31%|███       | 29/93 [00:14<00:33,  1.89it/s]
 32%|███▏      | 30/93 [00:14<00:31,  1.98it/s]
 33%|███▎      | 31/93 [00:15<00:33,  1.86it/s]
 34%|███▍      | 32/93 [00:16<00:30,  2.01it/s]
 35%|███▌      | 33/93 [00:16<00:30,  1.98it/s]
 37%|███▋      | 34/93 [00:17<00:30,  1.93it/s]
 38%|███▊      | 35/93 [00:17<00:28,  2.04it/s]
 39%|███▊      | 36/93 [00:17<00:25,  2.24it/s]
 40%|███▉      | 37/93 [00:18<00:22,  2.47it/s]
 41%|████      | 38/93 [00:18<00:24,  2.23it/s]
 42%|████▏     | 39/93 [00:19<00:23,  2.28it/s]
 43%|████▎     | 40/93 [00:19<00:24,  2.14it/s]
 44%|████▍     | 41/93 [00:20<00:26,  1.97it/s]
 45%|████▌     | 42/93 [00:20<00:27,  1.83it/s]
 46%|████▌     | 43/93 [00:21<00:28,  1.78it/s]
 47%|████▋     | 44/93 [00:22<00:26,  1.82it/s]
 48%|████▊     | 45/93 [00:22<00:24,  1.95it/s]
 49%|████▉     | 46/93 [00:22<00:24,  1.94it/s]
 51%|█████     | 47/93 [00:23<00:25,  1.81it/s]
 52%|█████▏    | 48/93 [00:24<00:23,  1.92it/s]
 53%|█████▎    | 49/93 [00:24<00:21,  2.03it/s]
 54%|█████▍    | 50/93 [00:24<00:20,  2.06it/s]
 55%|█████▍    | 51/93 [00:25<00:21,  2.00it/s]
 56%|█████▌    | 52/93 [00:26<00:22,  1.86it/s]
 57%|█████▋    | 53/93 [00:26<00:20,  1.91it/s]
 58%|█████▊    | 54/93 [00:27<00:20,  1.92it/s]
 59%|█████▉    | 55/93 [00:27<00:19,  1.98it/s]
 60%|██████    | 56/93 [00:27<00:17,  2.15it/s]
 61%|██████▏   | 57/93 [00:28<00:17,  2.03it/s]
 62%|██████▏   | 58/93 [00:28<00:16,  2.07it/s]
 63%|██████▎   | 59/93 [00:29<00:15,  2.14it/s]
 65%|██████▍   | 60/93 [00:29<00:16,  2.01it/s]
 66%|██████▌   | 61/93 [00:30<00:15,  2.08it/s]
 67%|██████▋   | 62/93 [00:30<00:14,  2.11it/s]
 68%|██████▊   | 63/93 [00:31<00:14,  2.10it/s]
 69%|██████▉   | 64/93 [00:33<00:27,  1.06it/s]
 70%|██████▉   | 65/93 [00:33<00:22,  1.23it/s]
 71%|███████   | 66/93 [00:34<00:20,  1.34it/s]
 72%|███████▏  | 67/93 [00:34<00:17,  1.52it/s]
 73%|███████▎  | 68/93 [00:35<00:16,  1.49it/s]
 74%|███████▍  | 69/93 [00:36<00:14,  1.62it/s]
 75%|███████▌  | 70/93 [00:36<00:13,  1.72it/s]
 76%|███████▋  | 71/93 [00:37<00:12,  1.80it/s]
 77%|███████▋  | 72/93 [00:37<00:11,  1.86it/s]
 78%|███████▊  | 73/93 [00:38<00:10,  1.91it/s]
 80%|███████▉  | 74/93 [00:38<00:09,  1.91it/s]
 81%|████████  | 75/93 [00:39<00:08,  2.03it/s]
 82%|████████▏ | 76/93 [00:39<00:08,  2.07it/s]
 83%|████████▎ | 77/93 [00:39<00:07,  2.20it/s]
 84%|████████▍ | 78/93 [00:40<00:06,  2.29it/s]
 85%|████████▍ | 79/93 [00:40<00:07,  1.95it/s]
 86%|████████▌ | 80/93 [00:41<00:07,  1.86it/s]
 87%|████████▋ | 81/93 [00:42<00:06,  1.93it/s]
 88%|████████▊ | 82/93 [00:42<00:05,  2.11it/s]
 89%|████████▉ | 83/93 [00:42<00:04,  2.06it/s]
 90%|█████████ | 84/93 [00:43<00:04,  1.88it/s]
 91%|█████████▏| 85/93 [00:44<00:04,  1.99it/s]
 92%|█████████▏| 86/93 [00:44<00:03,  2.09it/s]
 94%|█████████▎| 87/93 [00:44<00:02,  2.01it/s]
 95%|█████████▍| 88/93 [00:45<00:02,  2.00it/s]
 96%|█████████▌| 89/93 [00:45<00:01,  2.04it/s]
 97%|█████████▋| 90/93 [00:46<00:01,  2.09it/s]
 98%|█████████▊| 91/93 [00:46<00:01,  2.00it/s]
 99%|█████████▉| 92/93 [00:47<00:00,  2.03it/s]
100%|██████████| 93/93 [00:47<00:00,  2.13it/s]
100%|██████████| 93/93 [00:48<00:00,  1.94it/s]
***** predict_test_es_ES metrics *****
  predict_ex_match_acc         =     0.6722
  predict_ex_match_acc_stderr  =     0.0086
  predict_intent_acc           =     0.8692
  predict_intent_acc_stderr    =     0.0062
  predict_loss                 =      0.159
  predict_runtime              = 0:00:48.47
  predict_samples              =       2974
  predict_samples_per_second   =     61.347
  predict_slot_micro_f1        =       0.76
  predict_slot_micro_f1_stderr =     0.0029
  predict_steps_per_second     =      1.918
06/10/2024 22:38:44 - INFO - __main__ - *** test_de_DE ***
[INFO|trainer.py:718] 2024-06-10 22:38:44,125 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:38:44,127 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:38:44,128 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:38:44,128 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:20,  4.47it/s]
  3%|▎         | 3/93 [00:00<00:24,  3.64it/s]
  4%|▍         | 4/93 [00:01<00:26,  3.31it/s]
  5%|▌         | 5/93 [00:01<00:30,  2.91it/s]
  6%|▋         | 6/93 [00:01<00:31,  2.74it/s]
  8%|▊         | 7/93 [00:02<00:31,  2.69it/s]
  9%|▊         | 8/93 [00:02<00:31,  2.66it/s]
 10%|▉         | 9/93 [00:03<00:33,  2.54it/s]
 11%|█         | 10/93 [00:03<00:32,  2.53it/s]
 12%|█▏        | 11/93 [00:03<00:32,  2.55it/s]
 13%|█▎        | 12/93 [00:04<00:31,  2.56it/s]
 14%|█▍        | 13/93 [00:04<00:32,  2.49it/s]
 15%|█▌        | 14/93 [00:05<00:36,  2.14it/s]
 16%|█▌        | 15/93 [00:05<00:36,  2.16it/s]
 17%|█▋        | 16/93 [00:06<00:32,  2.34it/s]
 18%|█▊        | 17/93 [00:06<00:34,  2.22it/s]
 19%|█▉        | 18/93 [00:07<00:32,  2.29it/s]
 20%|██        | 19/93 [00:07<00:32,  2.29it/s]
 22%|██▏       | 20/93 [00:07<00:30,  2.38it/s]
 23%|██▎       | 21/93 [00:08<00:29,  2.46it/s]
 24%|██▎       | 22/93 [00:08<00:30,  2.29it/s]
 25%|██▍       | 23/93 [00:09<00:31,  2.26it/s]
 26%|██▌       | 24/93 [00:09<00:29,  2.35it/s]
 27%|██▋       | 25/93 [00:10<00:30,  2.25it/s]
 28%|██▊       | 26/93 [00:10<00:30,  2.22it/s]
 29%|██▉       | 27/93 [00:10<00:28,  2.33it/s]
 30%|███       | 28/93 [00:11<00:26,  2.47it/s]
 31%|███       | 29/93 [00:11<00:25,  2.51it/s]
 32%|███▏      | 30/93 [00:12<00:25,  2.49it/s]
 33%|███▎      | 31/93 [00:12<00:25,  2.40it/s]
 34%|███▍      | 32/93 [00:12<00:25,  2.39it/s]
 35%|███▌      | 33/93 [00:13<00:27,  2.22it/s]
 37%|███▋      | 34/93 [00:13<00:25,  2.27it/s]
 38%|███▊      | 35/93 [00:14<00:24,  2.32it/s]
 39%|███▊      | 36/93 [00:14<00:22,  2.48it/s]
 40%|███▉      | 37/93 [00:14<00:21,  2.64it/s]
 41%|████      | 38/93 [00:15<00:22,  2.49it/s]
 42%|████▏     | 39/93 [00:15<00:21,  2.49it/s]
 43%|████▎     | 40/93 [00:16<00:21,  2.45it/s]
 44%|████▍     | 41/93 [00:16<00:22,  2.33it/s]
 45%|████▌     | 42/93 [00:17<00:24,  2.11it/s]
 46%|████▌     | 43/93 [00:17<00:24,  2.03it/s]
 47%|████▋     | 44/93 [00:18<00:23,  2.13it/s]
 48%|████▊     | 45/93 [00:18<00:21,  2.25it/s]
 49%|████▉     | 46/93 [00:19<00:20,  2.31it/s]
 51%|█████     | 47/93 [00:19<00:21,  2.19it/s]
 52%|█████▏    | 48/93 [00:20<00:19,  2.26it/s]
 53%|█████▎    | 49/93 [00:20<00:19,  2.30it/s]
 54%|█████▍    | 50/93 [00:20<00:18,  2.31it/s]
 55%|█████▍    | 51/93 [00:21<00:17,  2.42it/s]
 56%|█████▌    | 52/93 [00:21<00:19,  2.15it/s]
 57%|█████▋    | 53/93 [00:22<00:18,  2.14it/s]
 58%|█████▊    | 54/93 [00:22<00:18,  2.06it/s]
 59%|█████▉    | 55/93 [00:23<00:17,  2.18it/s]
 60%|██████    | 56/93 [00:23<00:16,  2.30it/s]
 61%|██████▏   | 57/93 [00:24<00:16,  2.20it/s]
 62%|██████▏   | 58/93 [00:24<00:15,  2.28it/s]
 63%|██████▎   | 59/93 [00:24<00:15,  2.22it/s]
 65%|██████▍   | 60/93 [00:25<00:15,  2.13it/s]
 66%|██████▌   | 61/93 [00:25<00:14,  2.27it/s]
 67%|██████▋   | 62/93 [00:26<00:14,  2.14it/s]
 68%|██████▊   | 63/93 [00:26<00:13,  2.29it/s]
 69%|██████▉   | 64/93 [00:28<00:26,  1.10it/s]
 70%|██████▉   | 65/93 [00:29<00:20,  1.35it/s]
 71%|███████   | 66/93 [00:29<00:17,  1.57it/s]
 72%|███████▏  | 67/93 [00:29<00:14,  1.77it/s]
 73%|███████▎  | 68/93 [00:30<00:13,  1.92it/s]
 74%|███████▍  | 69/93 [00:30<00:12,  1.91it/s]
 75%|███████▌  | 70/93 [00:31<00:11,  2.05it/s]
 76%|███████▋  | 71/93 [00:31<00:10,  2.15it/s]
 77%|███████▋  | 72/93 [00:32<00:10,  2.08it/s]
 78%|███████▊  | 73/93 [00:32<00:08,  2.27it/s]
 80%|███████▉  | 74/93 [00:32<00:08,  2.22it/s]
 81%|████████  | 75/93 [00:33<00:08,  2.21it/s]
 82%|████████▏ | 76/93 [00:33<00:07,  2.17it/s]
 83%|████████▎ | 77/93 [00:34<00:07,  2.20it/s]
 84%|████████▍ | 78/93 [00:34<00:06,  2.23it/s]
 85%|████████▍ | 79/93 [00:35<00:07,  1.95it/s]
 86%|████████▌ | 80/93 [00:35<00:06,  1.95it/s]
 87%|████████▋ | 81/93 [00:36<00:05,  2.04it/s]
 88%|████████▊ | 82/93 [00:36<00:04,  2.22it/s]
 89%|████████▉ | 83/93 [00:37<00:04,  2.22it/s]
 90%|█████████ | 84/93 [00:37<00:04,  2.20it/s]
 91%|█████████▏| 85/93 [00:38<00:03,  2.24it/s]
 92%|█████████▏| 86/93 [00:38<00:03,  2.31it/s]
 94%|█████████▎| 87/93 [00:38<00:02,  2.23it/s]
 95%|█████████▍| 88/93 [00:39<00:02,  2.24it/s]
 96%|█████████▌| 89/93 [00:39<00:01,  2.19it/s]
 97%|█████████▋| 90/93 [00:40<00:01,  2.33it/s]
 98%|█████████▊| 91/93 [00:40<00:00,  2.16it/s]
 99%|█████████▉| 92/93 [00:41<00:00,  2.23it/s]
100%|██████████| 93/93 [00:41<00:00,  2.23it/s]
100%|██████████| 93/93 [00:41<00:00,  2.22it/s]
***** predict_test_de_DE metrics *****
  predict_ex_match_acc         =      0.696
  predict_ex_match_acc_stderr  =     0.0084
  predict_intent_acc           =     0.8665
  predict_intent_acc_stderr    =     0.0062
  predict_loss                 =     0.1518
  predict_runtime              = 0:00:42.34
  predict_samples              =       2974
  predict_samples_per_second   =     70.231
  predict_slot_micro_f1        =     0.7999
  predict_slot_micro_f1_stderr =     0.0029
  predict_steps_per_second     =      2.196
06/10/2024 22:39:26 - INFO - __main__ - *** test_fr_FR ***
[INFO|trainer.py:718] 2024-06-10 22:39:26,669 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:39:26,671 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:39:26,672 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:39:26,672 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:22,  4.11it/s]
  3%|▎         | 3/93 [00:00<00:30,  2.95it/s]
  4%|▍         | 4/93 [00:01<00:33,  2.65it/s]
  5%|▌         | 5/93 [00:01<00:38,  2.30it/s]
  6%|▋         | 6/93 [00:02<00:36,  2.37it/s]
  8%|▊         | 7/93 [00:02<00:38,  2.23it/s]
  9%|▊         | 8/93 [00:03<00:37,  2.27it/s]
 10%|▉         | 9/93 [00:03<00:37,  2.26it/s]
 11%|█         | 10/93 [00:04<00:38,  2.17it/s]
 12%|█▏        | 11/93 [00:04<00:37,  2.20it/s]
 13%|█▎        | 12/93 [00:05<00:36,  2.22it/s]
 14%|█▍        | 13/93 [00:05<00:36,  2.19it/s]
 15%|█▌        | 14/93 [00:06<00:41,  1.89it/s]
 16%|█▌        | 15/93 [00:06<00:40,  1.91it/s]
 17%|█▋        | 16/93 [00:07<00:37,  2.03it/s]
 18%|█▊        | 17/93 [00:07<00:39,  1.91it/s]
 19%|█▉        | 18/93 [00:08<00:37,  1.98it/s]
 20%|██        | 19/93 [00:08<00:37,  1.98it/s]
 22%|██▏       | 20/93 [00:09<00:36,  2.03it/s]
 23%|██▎       | 21/93 [00:09<00:35,  2.04it/s]
 24%|██▎       | 22/93 [00:10<00:35,  1.99it/s]
 25%|██▍       | 23/93 [00:10<00:33,  2.06it/s]
 26%|██▌       | 24/93 [00:11<00:33,  2.09it/s]
 27%|██▋       | 25/93 [00:11<00:35,  1.92it/s]
 28%|██▊       | 26/93 [00:12<00:36,  1.85it/s]
 29%|██▉       | 27/93 [00:12<00:32,  2.05it/s]
 30%|███       | 28/93 [00:13<00:30,  2.13it/s]
 31%|███       | 29/93 [00:13<00:30,  2.11it/s]
 32%|███▏      | 30/93 [00:14<00:28,  2.21it/s]
 33%|███▎      | 31/93 [00:14<00:28,  2.19it/s]
 34%|███▍      | 32/93 [00:14<00:28,  2.13it/s]
 35%|███▌      | 33/93 [00:15<00:28,  2.08it/s]
 37%|███▋      | 34/93 [00:15<00:28,  2.09it/s]
 38%|███▊      | 35/93 [00:16<00:26,  2.20it/s]
 39%|███▊      | 36/93 [00:16<00:25,  2.25it/s]
 40%|███▉      | 37/93 [00:17<00:23,  2.39it/s]
 41%|████      | 38/93 [00:17<00:26,  2.06it/s]
 42%|████▏     | 39/93 [00:18<00:24,  2.17it/s]
 43%|████▎     | 40/93 [00:18<00:24,  2.18it/s]
 44%|████▍     | 41/93 [00:19<00:24,  2.10it/s]
 45%|████▌     | 42/93 [00:19<00:26,  1.89it/s]
 46%|████▌     | 43/93 [00:20<00:27,  1.82it/s]
 47%|████▋     | 44/93 [00:20<00:26,  1.86it/s]
 48%|████▊     | 45/93 [00:21<00:23,  2.00it/s]
 49%|████▉     | 46/93 [00:21<00:23,  1.98it/s]
 51%|█████     | 47/93 [00:22<00:23,  1.99it/s]
 52%|█████▏    | 48/93 [00:22<00:21,  2.10it/s]
 53%|█████▎    | 49/93 [00:23<00:20,  2.14it/s]
 54%|█████▍    | 50/93 [00:23<00:20,  2.12it/s]
 55%|█████▍    | 51/93 [00:24<00:20,  2.04it/s]
 56%|█████▌    | 52/93 [00:24<00:19,  2.08it/s]
 57%|█████▋    | 53/93 [00:25<00:19,  2.07it/s]
 58%|█████▊    | 54/93 [00:25<00:19,  2.03it/s]
 59%|█████▉    | 55/93 [00:26<00:19,  1.95it/s]
 60%|██████    | 56/93 [00:26<00:17,  2.15it/s]
 61%|██████▏   | 57/93 [00:26<00:15,  2.28it/s]
 62%|██████▏   | 58/93 [00:27<00:15,  2.27it/s]
 63%|██████▎   | 59/93 [00:27<00:15,  2.18it/s]
 65%|██████▍   | 60/93 [00:28<00:15,  2.16it/s]
 66%|██████▌   | 61/93 [00:28<00:13,  2.31it/s]
 67%|██████▋   | 62/93 [00:29<00:14,  2.14it/s]
 68%|██████▊   | 63/93 [00:29<00:14,  2.06it/s]
 69%|██████▉   | 64/93 [00:31<00:27,  1.04it/s]
 70%|██████▉   | 65/93 [00:32<00:23,  1.21it/s]
 71%|███████   | 66/93 [00:32<00:19,  1.39it/s]
 72%|███████▏  | 67/93 [00:33<00:16,  1.56it/s]
 73%|███████▎  | 68/93 [00:33<00:16,  1.56it/s]
 74%|███████▍  | 69/93 [00:34<00:13,  1.74it/s]
 75%|███████▌  | 70/93 [00:34<00:12,  1.88it/s]
 76%|███████▋  | 71/93 [00:35<00:11,  1.96it/s]
 77%|███████▋  | 72/93 [00:35<00:11,  1.90it/s]
 78%|███████▊  | 73/93 [00:36<00:10,  1.98it/s]
 80%|███████▉  | 74/93 [00:36<00:09,  2.08it/s]
 81%|████████  | 75/93 [00:37<00:07,  2.25it/s]
 82%|████████▏ | 76/93 [00:37<00:07,  2.16it/s]
 83%|████████▎ | 77/93 [00:38<00:07,  2.17it/s]
 84%|████████▍ | 78/93 [00:38<00:07,  2.08it/s]
 85%|████████▍ | 79/93 [00:39<00:06,  2.12it/s]
 86%|████████▌ | 80/93 [00:40<00:11,  1.11it/s]
 87%|████████▋ | 81/93 [00:41<00:09,  1.29it/s]
 88%|████████▊ | 82/93 [00:41<00:07,  1.50it/s]
 89%|████████▉ | 83/93 [00:42<00:06,  1.64it/s]
 90%|█████████ | 84/93 [00:42<00:05,  1.72it/s]
 91%|█████████▏| 85/93 [00:43<00:04,  1.82it/s]
 92%|█████████▏| 86/93 [00:43<00:03,  1.95it/s]
 94%|█████████▎| 87/93 [00:44<00:03,  1.89it/s]
 95%|█████████▍| 88/93 [00:44<00:02,  1.92it/s]
 96%|█████████▌| 89/93 [00:45<00:02,  1.98it/s]
 97%|█████████▋| 90/93 [00:45<00:01,  2.10it/s]
 98%|█████████▊| 91/93 [00:46<00:00,  2.00it/s]
 99%|█████████▉| 92/93 [00:46<00:00,  2.07it/s]
100%|██████████| 93/93 [00:47<00:00,  2.07it/s]
100%|██████████| 93/93 [00:47<00:00,  1.96it/s]
***** predict_test_fr_FR metrics *****
  predict_ex_match_acc         =     0.6725
  predict_ex_match_acc_stderr  =     0.0086
  predict_intent_acc           =     0.8716
  predict_intent_acc_stderr    =     0.0061
  predict_loss                 =     0.1494
  predict_runtime              = 0:00:47.79
  predict_samples              =       2974
  predict_samples_per_second   =     62.224
  predict_slot_micro_f1        =       0.76
  predict_slot_micro_f1_stderr =     0.0029
  predict_steps_per_second     =      1.946
06/10/2024 22:40:14 - INFO - __main__ - *** test_pt_PT ***
[INFO|trainer.py:718] 2024-06-10 22:40:14,689 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:40:14,691 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:40:14,692 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:40:14,692 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:24,  3.77it/s]
  3%|▎         | 3/93 [00:00<00:28,  3.12it/s]
  4%|▍         | 4/93 [00:01<00:29,  3.00it/s]
  5%|▌         | 5/93 [00:01<00:34,  2.52it/s]
  6%|▋         | 6/93 [00:02<00:35,  2.42it/s]
  8%|▊         | 7/93 [00:02<00:39,  2.17it/s]
  9%|▊         | 8/93 [00:03<00:37,  2.25it/s]
 10%|▉         | 9/93 [00:03<00:38,  2.18it/s]
 11%|█         | 10/93 [00:04<00:38,  2.17it/s]
 12%|█▏        | 11/93 [00:04<00:38,  2.15it/s]
 13%|█▎        | 12/93 [00:05<00:36,  2.20it/s]
 14%|█▍        | 13/93 [00:05<00:36,  2.17it/s]
 15%|█▌        | 14/93 [00:06<00:40,  1.96it/s]
 16%|█▌        | 15/93 [00:06<00:38,  2.03it/s]
 17%|█▋        | 16/93 [00:07<00:36,  2.13it/s]
 18%|█▊        | 17/93 [00:07<00:37,  2.04it/s]
 19%|█▉        | 18/93 [00:07<00:34,  2.18it/s]
 20%|██        | 19/93 [00:08<00:34,  2.15it/s]
 22%|██▏       | 20/93 [00:08<00:32,  2.24it/s]
 23%|██▎       | 21/93 [00:09<00:31,  2.30it/s]
 24%|██▎       | 22/93 [00:09<00:32,  2.17it/s]
 25%|██▍       | 23/93 [00:10<00:32,  2.17it/s]
 26%|██▌       | 24/93 [00:10<00:30,  2.25it/s]
 27%|██▋       | 25/93 [00:11<00:31,  2.19it/s]
 28%|██▊       | 26/93 [00:11<00:30,  2.21it/s]
 29%|██▉       | 27/93 [00:12<00:29,  2.24it/s]
 30%|███       | 28/93 [00:12<00:28,  2.32it/s]
 31%|███       | 29/93 [00:12<00:27,  2.31it/s]
 32%|███▏      | 30/93 [00:13<00:28,  2.24it/s]
 33%|███▎      | 31/93 [00:13<00:27,  2.29it/s]
 34%|███▍      | 32/93 [00:14<00:27,  2.26it/s]
 35%|███▌      | 33/93 [00:14<00:26,  2.23it/s]
 37%|███▋      | 34/93 [00:15<00:29,  2.02it/s]
 38%|███▊      | 35/93 [00:15<00:27,  2.11it/s]
 39%|███▊      | 36/93 [00:15<00:24,  2.33it/s]
 40%|███▉      | 37/93 [00:16<00:23,  2.39it/s]
 41%|████      | 38/93 [00:16<00:25,  2.18it/s]
 42%|████▏     | 39/93 [00:17<00:23,  2.25it/s]
 43%|████▎     | 40/93 [00:17<00:24,  2.19it/s]
 44%|████▍     | 41/93 [00:18<00:26,  1.99it/s]
 45%|████▌     | 42/93 [00:19<00:26,  1.92it/s]
 46%|████▌     | 43/93 [00:19<00:27,  1.81it/s]
 47%|████▋     | 44/93 [00:20<00:25,  1.91it/s]
 48%|████▊     | 45/93 [00:20<00:23,  2.07it/s]
 49%|████▉     | 46/93 [00:21<00:23,  2.01it/s]
 51%|█████     | 47/93 [00:21<00:24,  1.86it/s]
 52%|█████▏    | 48/93 [00:22<00:23,  1.91it/s]
 53%|█████▎    | 49/93 [00:22<00:20,  2.14it/s]
 54%|█████▍    | 50/93 [00:23<00:21,  2.00it/s]
 55%|█████▍    | 51/93 [00:23<00:20,  2.02it/s]
 56%|█████▌    | 52/93 [00:24<00:20,  1.99it/s]
 57%|█████▋    | 53/93 [00:24<00:20,  1.99it/s]
 58%|█████▊    | 54/93 [00:25<00:20,  1.87it/s]
 59%|█████▉    | 55/93 [00:25<00:20,  1.88it/s]
 60%|██████    | 56/93 [00:26<00:17,  2.12it/s]
 61%|██████▏   | 57/93 [00:26<00:16,  2.15it/s]
 62%|██████▏   | 58/93 [00:26<00:15,  2.24it/s]
 63%|██████▎   | 59/93 [00:27<00:15,  2.23it/s]
 65%|██████▍   | 60/93 [00:27<00:15,  2.11it/s]
 66%|██████▌   | 61/93 [00:28<00:14,  2.29it/s]
 67%|██████▋   | 62/93 [00:28<00:14,  2.19it/s]
 68%|██████▊   | 63/93 [00:29<00:13,  2.19it/s]
 69%|██████▉   | 64/93 [00:31<00:27,  1.07it/s]
 70%|██████▉   | 65/93 [00:31<00:22,  1.27it/s]
 71%|███████   | 66/93 [00:32<00:20,  1.35it/s]
 72%|███████▏  | 67/93 [00:32<00:17,  1.52it/s]
 73%|███████▎  | 68/93 [00:33<00:17,  1.47it/s]
 74%|███████▍  | 69/93 [00:34<00:15,  1.59it/s]
 75%|███████▌  | 70/93 [00:34<00:14,  1.63it/s]
 76%|███████▋  | 71/93 [00:35<00:12,  1.78it/s]
 77%|███████▋  | 72/93 [00:35<00:11,  1.86it/s]
 78%|███████▊  | 73/93 [00:35<00:09,  2.02it/s]
 80%|███████▉  | 74/93 [00:36<00:09,  2.00it/s]
 81%|████████  | 75/93 [00:36<00:08,  2.09it/s]
 82%|████████▏ | 76/93 [00:37<00:08,  2.08it/s]
 83%|████████▎ | 77/93 [00:37<00:07,  2.13it/s]
 84%|████████▍ | 78/93 [00:38<00:07,  2.12it/s]
 85%|████████▍ | 79/93 [00:38<00:06,  2.02it/s]
 86%|████████▌ | 80/93 [00:39<00:06,  1.87it/s]
 87%|████████▋ | 81/93 [00:39<00:06,  1.87it/s]
 88%|████████▊ | 82/93 [00:40<00:05,  2.03it/s]
 89%|████████▉ | 83/93 [00:40<00:04,  2.04it/s]
 90%|█████████ | 84/93 [00:41<00:04,  1.94it/s]
 91%|█████████▏| 85/93 [00:41<00:03,  2.03it/s]
 92%|█████████▏| 86/93 [00:42<00:03,  2.15it/s]
 94%|█████████▎| 87/93 [00:42<00:02,  2.11it/s]
 95%|█████████▍| 88/93 [00:43<00:02,  2.05it/s]
 96%|█████████▌| 89/93 [00:43<00:01,  2.00it/s]
 97%|█████████▋| 90/93 [00:44<00:01,  2.06it/s]
 98%|█████████▊| 91/93 [00:44<00:01,  2.00it/s]
 99%|█████████▉| 92/93 [00:45<00:00,  2.08it/s]
100%|██████████| 93/93 [00:45<00:00,  2.19it/s]
100%|██████████| 93/93 [00:45<00:00,  2.03it/s]
***** predict_test_pt_PT metrics *****
  predict_ex_match_acc         =      0.688
  predict_ex_match_acc_stderr  =     0.0085
  predict_intent_acc           =     0.8742
  predict_intent_acc_stderr    =     0.0061
  predict_loss                 =     0.1455
  predict_runtime              = 0:00:46.29
  predict_samples              =       2974
  predict_samples_per_second   =      64.24
  predict_slot_micro_f1        =      0.777
  predict_slot_micro_f1_stderr =     0.0029
  predict_steps_per_second     =      2.009
06/10/2024 22:41:01 - INFO - __main__ - *** test_pl_PL ***
[INFO|trainer.py:718] 2024-06-10 22:41:01,217 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:41:01,222 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:41:01,222 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:41:01,223 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:14,  6.19it/s]
  3%|▎         | 3/93 [00:00<00:22,  4.04it/s]
  4%|▍         | 4/93 [00:01<00:24,  3.63it/s]
  5%|▌         | 5/93 [00:01<00:32,  2.71it/s]
  6%|▋         | 6/93 [00:01<00:32,  2.67it/s]
  8%|▊         | 7/93 [00:02<00:32,  2.64it/s]
  9%|▊         | 8/93 [00:02<00:32,  2.60it/s]
 10%|▉         | 9/93 [00:03<00:32,  2.62it/s]
 11%|█         | 10/93 [00:03<00:31,  2.62it/s]
 12%|█▏        | 11/93 [00:03<00:33,  2.44it/s]
 13%|█▎        | 12/93 [00:04<00:33,  2.44it/s]
 14%|█▍        | 13/93 [00:04<00:32,  2.45it/s]
 15%|█▌        | 14/93 [00:05<00:35,  2.26it/s]
 16%|█▌        | 15/93 [00:05<00:34,  2.26it/s]
 17%|█▋        | 16/93 [00:06<00:32,  2.37it/s]
 18%|█▊        | 17/93 [00:06<00:33,  2.28it/s]
 19%|█▉        | 18/93 [00:06<00:31,  2.37it/s]
 20%|██        | 19/93 [00:07<00:31,  2.34it/s]
 22%|██▏       | 20/93 [00:07<00:29,  2.49it/s]
 23%|██▎       | 21/93 [00:08<00:28,  2.52it/s]
 24%|██▎       | 22/93 [00:08<00:30,  2.33it/s]
 25%|██▍       | 23/93 [00:09<00:28,  2.42it/s]
 26%|██▌       | 24/93 [00:09<00:29,  2.36it/s]
 27%|██▋       | 25/93 [00:10<00:33,  2.05it/s]
 28%|██▊       | 26/93 [00:10<00:33,  2.00it/s]
 29%|██▉       | 27/93 [00:11<00:30,  2.16it/s]
 30%|███       | 28/93 [00:11<00:28,  2.31it/s]
 31%|███       | 29/93 [00:11<00:26,  2.44it/s]
 32%|███▏      | 30/93 [00:12<00:27,  2.32it/s]
 33%|███▎      | 31/93 [00:12<00:26,  2.37it/s]
 34%|███▍      | 32/93 [00:12<00:24,  2.44it/s]
 35%|███▌      | 33/93 [00:13<00:24,  2.40it/s]
 37%|███▋      | 34/93 [00:13<00:24,  2.44it/s]
 38%|███▊      | 35/93 [00:14<00:22,  2.55it/s]
 39%|███▊      | 36/93 [00:14<00:20,  2.76it/s]
 40%|███▉      | 37/93 [00:14<00:19,  2.89it/s]
 41%|████      | 38/93 [00:15<00:21,  2.57it/s]
 42%|████▏     | 39/93 [00:15<00:20,  2.66it/s]
 43%|████▎     | 40/93 [00:16<00:20,  2.58it/s]
 44%|████▍     | 41/93 [00:16<00:21,  2.42it/s]
 45%|████▌     | 42/93 [00:16<00:21,  2.41it/s]
 46%|████▌     | 43/93 [00:17<00:23,  2.13it/s]
 47%|████▋     | 44/93 [00:17<00:21,  2.26it/s]
 48%|████▊     | 45/93 [00:18<00:20,  2.39it/s]
 49%|████▉     | 46/93 [00:18<00:20,  2.28it/s]
 51%|█████     | 47/93 [00:19<00:20,  2.23it/s]
 52%|█████▏    | 48/93 [00:19<00:19,  2.29it/s]
 53%|█████▎    | 49/93 [00:19<00:18,  2.44it/s]
 54%|█████▍    | 50/93 [00:20<00:18,  2.32it/s]
 55%|█████▍    | 51/93 [00:20<00:18,  2.25it/s]
 56%|█████▌    | 52/93 [00:21<00:19,  2.14it/s]
 57%|█████▋    | 53/93 [00:21<00:18,  2.12it/s]
 58%|█████▊    | 54/93 [00:22<00:17,  2.21it/s]
 59%|█████▉    | 55/93 [00:22<00:15,  2.38it/s]
 60%|██████    | 56/93 [00:22<00:14,  2.57it/s]
 61%|██████▏   | 57/93 [00:23<00:14,  2.52it/s]
 62%|██████▏   | 58/93 [00:23<00:13,  2.61it/s]
 63%|██████▎   | 59/93 [00:24<00:13,  2.58it/s]
 65%|██████▍   | 60/93 [00:24<00:13,  2.48it/s]
 66%|██████▌   | 61/93 [00:24<00:12,  2.51it/s]
 67%|██████▋   | 62/93 [00:25<00:13,  2.31it/s]
 68%|██████▊   | 63/93 [00:25<00:12,  2.44it/s]
 69%|██████▉   | 64/93 [00:27<00:25,  1.12it/s]
 70%|██████▉   | 65/93 [00:28<00:20,  1.34it/s]
 71%|███████   | 66/93 [00:28<00:17,  1.57it/s]
 72%|███████▏  | 67/93 [00:29<00:14,  1.76it/s]
 73%|███████▎  | 68/93 [00:29<00:14,  1.72it/s]
 74%|███████▍  | 69/93 [00:30<00:12,  1.90it/s]
 75%|███████▌  | 70/93 [00:30<00:11,  2.04it/s]
 76%|███████▋  | 71/93 [00:30<00:10,  2.09it/s]
 77%|███████▋  | 72/93 [00:31<00:10,  2.09it/s]
 78%|███████▊  | 73/93 [00:31<00:08,  2.25it/s]
 80%|███████▉  | 74/93 [00:32<00:08,  2.21it/s]
 81%|████████  | 75/93 [00:32<00:07,  2.40it/s]
 82%|████████▏ | 76/93 [00:33<00:07,  2.36it/s]
 83%|████████▎ | 77/93 [00:33<00:06,  2.50it/s]
 84%|████████▍ | 78/93 [00:33<00:05,  2.68it/s]
 85%|████████▍ | 79/93 [00:34<00:06,  2.25it/s]
 86%|████████▌ | 80/93 [00:34<00:06,  2.01it/s]
 87%|████████▋ | 81/93 [00:35<00:05,  2.18it/s]
 88%|████████▊ | 82/93 [00:35<00:04,  2.33it/s]
 89%|████████▉ | 83/93 [00:36<00:04,  2.39it/s]
 90%|█████████ | 84/93 [00:36<00:04,  2.21it/s]
 91%|█████████▏| 85/93 [00:37<00:03,  2.19it/s]
 92%|█████████▏| 86/93 [00:37<00:03,  2.30it/s]
 94%|█████████▎| 87/93 [00:37<00:02,  2.36it/s]
 95%|█████████▍| 88/93 [00:38<00:02,  2.41it/s]
 96%|█████████▌| 89/93 [00:38<00:01,  2.35it/s]
 97%|█████████▋| 90/93 [00:39<00:01,  2.42it/s]
 98%|█████████▊| 91/93 [00:39<00:00,  2.30it/s]
 99%|█████████▉| 92/93 [00:39<00:00,  2.29it/s]
100%|██████████| 93/93 [00:40<00:00,  2.45it/s]
100%|██████████| 93/93 [00:40<00:00,  2.30it/s]
***** predict_test_pl_PL metrics *****
  predict_ex_match_acc         =     0.6577
  predict_ex_match_acc_stderr  =     0.0087
  predict_intent_acc           =     0.8753
  predict_intent_acc_stderr    =     0.0061
  predict_loss                 =     0.1657
  predict_runtime              = 0:00:41.03
  predict_samples              =       2974
  predict_samples_per_second   =     72.472
  predict_slot_micro_f1        =     0.7381
  predict_slot_micro_f1_stderr =     0.0034
  predict_steps_per_second     =      2.266
06/10/2024 22:41:42 - INFO - __main__ - *** test_nl_NL ***
[INFO|trainer.py:718] 2024-06-10 22:41:42,453 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:41:42,456 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:41:42,456 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:41:42,457 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:22,  4.13it/s]
  3%|▎         | 3/93 [00:00<00:26,  3.35it/s]
  4%|▍         | 4/93 [00:01<00:28,  3.15it/s]
  5%|▌         | 5/93 [00:01<00:33,  2.60it/s]
  6%|▋         | 6/93 [00:02<00:33,  2.60it/s]
  8%|▊         | 7/93 [00:02<00:32,  2.63it/s]
  9%|▊         | 8/93 [00:02<00:33,  2.57it/s]
 10%|▉         | 9/93 [00:03<00:34,  2.45it/s]
 11%|█         | 10/93 [00:03<00:33,  2.48it/s]
 12%|█▏        | 11/93 [00:04<00:32,  2.49it/s]
 13%|█▎        | 12/93 [00:04<00:32,  2.46it/s]
 14%|█▍        | 13/93 [00:04<00:33,  2.41it/s]
 15%|█▌        | 14/93 [00:05<00:35,  2.20it/s]
 16%|█▌        | 15/93 [00:05<00:35,  2.22it/s]
 17%|█▋        | 16/93 [00:06<00:35,  2.19it/s]
 18%|█▊        | 17/93 [00:06<00:35,  2.16it/s]
 19%|█▉        | 18/93 [00:07<00:35,  2.11it/s]
 20%|██        | 19/93 [00:07<00:34,  2.14it/s]
 22%|██▏       | 20/93 [00:08<00:33,  2.19it/s]
 23%|██▎       | 21/93 [00:08<00:31,  2.28it/s]
 24%|██▎       | 22/93 [00:09<00:33,  2.14it/s]
 25%|██▍       | 23/93 [00:09<00:31,  2.26it/s]
 26%|██▌       | 24/93 [00:09<00:29,  2.37it/s]
 27%|██▋       | 25/93 [00:10<00:29,  2.33it/s]
 28%|██▊       | 26/93 [00:10<00:29,  2.28it/s]
 29%|██▉       | 27/93 [00:11<00:28,  2.33it/s]
 30%|███       | 28/93 [00:11<00:27,  2.37it/s]
 31%|███       | 29/93 [00:12<00:26,  2.41it/s]
 32%|███▏      | 30/93 [00:12<00:27,  2.30it/s]
 33%|███▎      | 31/93 [00:13<00:29,  2.09it/s]
 34%|███▍      | 32/93 [00:13<00:29,  2.04it/s]
 35%|███▌      | 33/93 [00:14<00:29,  2.01it/s]
 37%|███▋      | 34/93 [00:14<00:28,  2.09it/s]
 38%|███▊      | 35/93 [00:15<00:27,  2.14it/s]
 39%|███▊      | 36/93 [00:15<00:24,  2.31it/s]
 40%|███▉      | 37/93 [00:15<00:23,  2.41it/s]
 41%|████      | 38/93 [00:16<00:24,  2.26it/s]
 42%|████▏     | 39/93 [00:16<00:24,  2.21it/s]
 43%|████▎     | 40/93 [00:17<00:23,  2.23it/s]
 44%|████▍     | 41/93 [00:17<00:23,  2.24it/s]
 45%|████▌     | 42/93 [00:18<00:24,  2.08it/s]
 46%|████▌     | 43/93 [00:18<00:26,  1.91it/s]
 47%|████▋     | 44/93 [00:19<00:23,  2.06it/s]
 48%|████▊     | 45/93 [00:19<00:22,  2.17it/s]
 49%|████▉     | 46/93 [00:20<00:22,  2.06it/s]
 51%|█████     | 47/93 [00:20<00:22,  2.03it/s]
 52%|█████▏    | 48/93 [00:21<00:21,  2.13it/s]
 53%|█████▎    | 49/93 [00:21<00:19,  2.22it/s]
 54%|█████▍    | 50/93 [00:22<00:19,  2.18it/s]
 55%|█████▍    | 51/93 [00:22<00:19,  2.17it/s]
 56%|█████▌    | 52/93 [00:22<00:18,  2.19it/s]
 57%|█████▋    | 53/93 [00:23<00:19,  2.06it/s]
 58%|█████▊    | 54/93 [00:23<00:18,  2.09it/s]
 59%|█████▉    | 55/93 [00:24<00:18,  2.08it/s]
 60%|██████    | 56/93 [00:24<00:17,  2.13it/s]
 61%|██████▏   | 57/93 [00:25<00:17,  2.08it/s]
 62%|██████▏   | 58/93 [00:25<00:15,  2.21it/s]
 63%|██████▎   | 59/93 [00:26<00:15,  2.19it/s]
 65%|██████▍   | 60/93 [00:26<00:15,  2.15it/s]
 66%|██████▌   | 61/93 [00:27<00:13,  2.29it/s]
 67%|██████▋   | 62/93 [00:27<00:14,  2.13it/s]
 68%|██████▊   | 63/93 [00:28<00:13,  2.16it/s]
 69%|██████▉   | 64/93 [00:30<00:26,  1.08it/s]
 70%|██████▉   | 65/93 [00:30<00:21,  1.29it/s]
 71%|███████   | 66/93 [00:30<00:18,  1.49it/s]
 72%|███████▏  | 67/93 [00:31<00:16,  1.60it/s]
 73%|███████▎  | 68/93 [00:31<00:14,  1.73it/s]
 74%|███████▍  | 69/93 [00:32<00:12,  1.85it/s]
 75%|███████▌  | 70/93 [00:32<00:12,  1.88it/s]
 76%|███████▋  | 71/93 [00:33<00:10,  2.02it/s]
 77%|███████▋  | 72/93 [00:33<00:10,  1.97it/s]
 78%|███████▊  | 73/93 [00:34<00:09,  2.12it/s]
 80%|███████▉  | 74/93 [00:34<00:09,  2.01it/s]
 81%|████████  | 75/93 [00:35<00:08,  2.13it/s]
 82%|████████▏ | 76/93 [00:35<00:08,  2.11it/s]
 83%|████████▎ | 77/93 [00:36<00:07,  2.17it/s]
 84%|████████▍ | 78/93 [00:36<00:07,  2.11it/s]
 85%|████████▍ | 79/93 [00:37<00:07,  1.87it/s]
 86%|████████▌ | 80/93 [00:37<00:07,  1.74it/s]
 87%|████████▋ | 81/93 [00:38<00:06,  1.87it/s]
 88%|████████▊ | 82/93 [00:38<00:05,  2.07it/s]
 89%|████████▉ | 83/93 [00:39<00:04,  2.15it/s]
 90%|█████████ | 84/93 [00:39<00:04,  1.99it/s]
 91%|█████████▏| 85/93 [00:40<00:03,  2.10it/s]
 92%|█████████▏| 86/93 [00:40<00:03,  2.19it/s]
 94%|█████████▎| 87/93 [00:41<00:02,  2.14it/s]
 95%|█████████▍| 88/93 [00:41<00:02,  2.22it/s]
 96%|█████████▌| 89/93 [00:41<00:01,  2.19it/s]
 97%|█████████▋| 90/93 [00:42<00:01,  2.26it/s]
 98%|█████████▊| 91/93 [00:42<00:00,  2.08it/s]
 99%|█████████▉| 92/93 [00:43<00:00,  2.13it/s]
100%|██████████| 93/93 [00:43<00:00,  2.17it/s]
100%|██████████| 93/93 [00:44<00:00,  2.11it/s]
***** predict_test_nl_NL metrics *****
  predict_ex_match_acc         =     0.6974
  predict_ex_match_acc_stderr  =     0.0084
  predict_intent_acc           =     0.8769
  predict_intent_acc_stderr    =      0.006
  predict_loss                 =     0.1494
  predict_runtime              = 0:00:44.48
  predict_samples              =       2974
  predict_samples_per_second   =     66.861
  predict_slot_micro_f1        =     0.7816
  predict_slot_micro_f1_stderr =     0.0029
  predict_steps_per_second     =      2.091
06/10/2024 22:42:27 - INFO - __main__ - *** test_hu_HU ***
[INFO|trainer.py:718] 2024-06-10 22:42:27,132 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:42:27,134 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:42:27,134 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:42:27,135 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:20,  4.36it/s]
  3%|▎         | 3/93 [00:00<00:25,  3.54it/s]
  4%|▍         | 4/93 [00:01<00:29,  3.05it/s]
  5%|▌         | 5/93 [00:01<00:32,  2.67it/s]
  6%|▋         | 6/93 [00:02<00:32,  2.64it/s]
  8%|▊         | 7/93 [00:02<00:32,  2.63it/s]
  9%|▊         | 8/93 [00:02<00:33,  2.53it/s]
 10%|▉         | 9/93 [00:03<00:34,  2.42it/s]
 11%|█         | 10/93 [00:03<00:33,  2.50it/s]
 12%|█▏        | 11/93 [00:04<00:35,  2.31it/s]
 13%|█▎        | 12/93 [00:04<00:33,  2.44it/s]
 14%|█▍        | 13/93 [00:04<00:32,  2.46it/s]
 15%|█▌        | 14/93 [00:05<00:35,  2.24it/s]
 16%|█▌        | 15/93 [00:05<00:33,  2.33it/s]
 17%|█▋        | 16/93 [00:06<00:31,  2.45it/s]
 18%|█▊        | 17/93 [00:06<00:33,  2.29it/s]
 19%|█▉        | 18/93 [00:07<00:31,  2.40it/s]
 20%|██        | 19/93 [00:07<00:30,  2.40it/s]
 22%|██▏       | 20/93 [00:07<00:30,  2.40it/s]
 23%|██▎       | 21/93 [00:08<00:28,  2.55it/s]
 24%|██▎       | 22/93 [00:08<00:29,  2.39it/s]
 25%|██▍       | 23/93 [00:09<00:28,  2.47it/s]
 26%|██▌       | 24/93 [00:09<00:26,  2.57it/s]
 27%|██▋       | 25/93 [00:10<00:30,  2.25it/s]
 28%|██▊       | 26/93 [00:10<00:27,  2.42it/s]
 29%|██▉       | 27/93 [00:10<00:26,  2.47it/s]
 30%|███       | 28/93 [00:11<00:25,  2.56it/s]
 31%|███       | 29/93 [00:11<00:24,  2.56it/s]
 32%|███▏      | 30/93 [00:12<00:29,  2.16it/s]
 33%|███▎      | 31/93 [00:12<00:28,  2.17it/s]
 34%|███▍      | 32/93 [00:13<00:26,  2.26it/s]
 35%|███▌      | 33/93 [00:13<00:27,  2.21it/s]
 37%|███▋      | 34/93 [00:13<00:26,  2.19it/s]
 38%|███▊      | 35/93 [00:14<00:25,  2.26it/s]
 39%|███▊      | 36/93 [00:14<00:23,  2.46it/s]
 40%|███▉      | 37/93 [00:14<00:20,  2.76it/s]
 41%|████      | 38/93 [00:15<00:21,  2.54it/s]
 42%|████▏     | 39/93 [00:15<00:20,  2.62it/s]
 43%|████▎     | 40/93 [00:16<00:20,  2.63it/s]
 44%|████▍     | 41/93 [00:16<00:20,  2.51it/s]
 45%|████▌     | 42/93 [00:17<00:21,  2.34it/s]
 46%|████▌     | 43/93 [00:17<00:21,  2.31it/s]
 47%|████▋     | 44/93 [00:17<00:20,  2.40it/s]
 48%|████▊     | 45/93 [00:18<00:19,  2.41it/s]
 49%|████▉     | 46/93 [00:18<00:19,  2.36it/s]
 51%|█████     | 47/93 [00:19<00:21,  2.17it/s]
 52%|█████▏    | 48/93 [00:19<00:19,  2.36it/s]
 53%|█████▎    | 49/93 [00:19<00:17,  2.54it/s]
 54%|█████▍    | 50/93 [00:20<00:17,  2.50it/s]
 55%|█████▍    | 51/93 [00:20<00:16,  2.52it/s]
 56%|█████▌    | 52/93 [00:21<00:16,  2.44it/s]
 57%|█████▋    | 53/93 [00:21<00:17,  2.33it/s]
 58%|█████▊    | 54/93 [00:22<00:17,  2.21it/s]
 59%|█████▉    | 55/93 [00:22<00:17,  2.19it/s]
 60%|██████    | 56/93 [00:22<00:15,  2.40it/s]
 61%|██████▏   | 57/93 [00:23<00:15,  2.36it/s]
 62%|██████▏   | 58/93 [00:23<00:13,  2.54it/s]
 63%|██████▎   | 59/93 [00:24<00:12,  2.69it/s]
 65%|██████▍   | 60/93 [00:24<00:12,  2.65it/s]
 66%|██████▌   | 61/93 [00:24<00:12,  2.64it/s]
 67%|██████▋   | 62/93 [00:25<00:12,  2.42it/s]
 68%|██████▊   | 63/93 [00:25<00:12,  2.50it/s]
 69%|██████▉   | 64/93 [00:27<00:25,  1.12it/s]
 70%|██████▉   | 65/93 [00:28<00:20,  1.35it/s]
 71%|███████   | 66/93 [00:28<00:17,  1.57it/s]
 72%|███████▏  | 67/93 [00:28<00:14,  1.77it/s]
 73%|███████▎  | 68/93 [00:29<00:13,  1.88it/s]
 74%|███████▍  | 69/93 [00:29<00:11,  2.12it/s]
 75%|███████▌  | 70/93 [00:30<00:09,  2.34it/s]
 76%|███████▋  | 71/93 [00:30<00:09,  2.40it/s]
 77%|███████▋  | 72/93 [00:30<00:09,  2.25it/s]
 78%|███████▊  | 73/93 [00:31<00:08,  2.46it/s]
 80%|███████▉  | 74/93 [00:31<00:08,  2.25it/s]
 81%|████████  | 75/93 [00:32<00:07,  2.42it/s]
 82%|████████▏ | 76/93 [00:32<00:07,  2.35it/s]
 83%|████████▎ | 77/93 [00:32<00:06,  2.45it/s]
 84%|████████▍ | 78/93 [00:33<00:05,  2.58it/s]
 85%|████████▍ | 79/93 [00:33<00:06,  2.12it/s]
 86%|████████▌ | 80/93 [00:34<00:06,  1.98it/s]
 87%|████████▋ | 81/93 [00:34<00:05,  2.18it/s]
 88%|████████▊ | 82/93 [00:35<00:04,  2.35it/s]
 89%|████████▉ | 83/93 [00:35<00:04,  2.35it/s]
 90%|█████████ | 84/93 [00:36<00:04,  2.19it/s]
 91%|█████████▏| 85/93 [00:36<00:03,  2.25it/s]
 92%|█████████▏| 86/93 [00:37<00:03,  2.30it/s]
 94%|█████████▎| 87/93 [00:37<00:02,  2.28it/s]
 95%|█████████▍| 88/93 [00:37<00:02,  2.28it/s]
 96%|█████████▌| 89/93 [00:38<00:01,  2.22it/s]
 97%|█████████▋| 90/93 [00:38<00:01,  2.25it/s]
 98%|█████████▊| 91/93 [00:39<00:00,  2.10it/s]
 99%|█████████▉| 92/93 [00:39<00:00,  2.18it/s]
100%|██████████| 93/93 [00:40<00:00,  2.29it/s]
100%|██████████| 93/93 [00:40<00:00,  2.29it/s]
***** predict_test_hu_HU metrics *****
  predict_ex_match_acc         =     0.6866
  predict_ex_match_acc_stderr  =     0.0085
  predict_intent_acc           =     0.8668
  predict_intent_acc_stderr    =     0.0062
  predict_loss                 =     0.1564
  predict_runtime              = 0:00:41.00
  predict_samples              =       2974
  predict_samples_per_second   =     72.526
  predict_slot_micro_f1        =     0.7878
  predict_slot_micro_f1_stderr =     0.0031
  predict_steps_per_second     =      2.268
06/10/2024 22:43:08 - INFO - __main__ - *** test_ru_RU ***
[INFO|trainer.py:718] 2024-06-10 22:43:08,343 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:43:08,345 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:43:08,345 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:43:08,346 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:18,  4.79it/s]
  3%|▎         | 3/93 [00:00<00:24,  3.73it/s]
  4%|▍         | 4/93 [00:01<00:26,  3.39it/s]
  5%|▌         | 5/93 [00:01<00:29,  2.95it/s]
  6%|▋         | 6/93 [00:01<00:30,  2.83it/s]
  8%|▊         | 7/93 [00:02<00:31,  2.70it/s]
  9%|▊         | 8/93 [00:02<00:32,  2.65it/s]
 10%|▉         | 9/93 [00:03<00:32,  2.58it/s]
 11%|█         | 10/93 [00:03<00:33,  2.49it/s]
 12%|█▏        | 11/93 [00:03<00:32,  2.53it/s]
 13%|█▎        | 12/93 [00:04<00:32,  2.52it/s]
 14%|█▍        | 13/93 [00:04<00:31,  2.52it/s]
 15%|█▌        | 14/93 [00:05<00:36,  2.19it/s]
 16%|█▌        | 15/93 [00:05<00:35,  2.22it/s]
 17%|█▋        | 16/93 [00:06<00:32,  2.35it/s]
 18%|█▊        | 17/93 [00:06<00:33,  2.26it/s]
 19%|█▉        | 18/93 [00:06<00:31,  2.35it/s]
 20%|██        | 19/93 [00:07<00:31,  2.38it/s]
 22%|██▏       | 20/93 [00:07<00:29,  2.44it/s]
 23%|██▎       | 21/93 [00:08<00:28,  2.50it/s]
 24%|██▎       | 22/93 [00:08<00:31,  2.28it/s]
 25%|██▍       | 23/93 [00:09<00:30,  2.33it/s]
 26%|██▌       | 24/93 [00:09<00:28,  2.43it/s]
 27%|██▋       | 25/93 [00:10<00:30,  2.22it/s]
 28%|██▊       | 26/93 [00:10<00:30,  2.21it/s]
 29%|██▉       | 27/93 [00:10<00:28,  2.32it/s]
 30%|███       | 28/93 [00:11<00:27,  2.40it/s]
 31%|███       | 29/93 [00:11<00:25,  2.48it/s]
 32%|███▏      | 30/93 [00:12<00:25,  2.45it/s]
 33%|███▎      | 31/93 [00:12<00:26,  2.35it/s]
 34%|███▍      | 32/93 [00:12<00:25,  2.42it/s]
 35%|███▌      | 33/93 [00:13<00:24,  2.47it/s]
 37%|███▋      | 34/93 [00:13<00:25,  2.34it/s]
 38%|███▊      | 35/93 [00:14<00:24,  2.39it/s]
 39%|███▊      | 36/93 [00:14<00:22,  2.54it/s]
 40%|███▉      | 37/93 [00:14<00:20,  2.76it/s]
 41%|████      | 38/93 [00:15<00:21,  2.56it/s]
 42%|████▏     | 39/93 [00:15<00:21,  2.52it/s]
 43%|████▎     | 40/93 [00:16<00:22,  2.36it/s]
 44%|████▍     | 41/93 [00:16<00:22,  2.30it/s]
 45%|████▌     | 42/93 [00:17<00:21,  2.32it/s]
 46%|████▌     | 43/93 [00:17<00:23,  2.12it/s]
 47%|████▋     | 44/93 [00:17<00:21,  2.27it/s]
 48%|████▊     | 45/93 [00:18<00:21,  2.27it/s]
 49%|████▉     | 46/93 [00:18<00:20,  2.28it/s]
 51%|█████     | 47/93 [00:19<00:24,  1.85it/s]
 52%|█████▏    | 48/93 [00:19<00:21,  2.06it/s]
 53%|█████▎    | 49/93 [00:20<00:19,  2.24it/s]
 54%|█████▍    | 50/93 [00:20<00:18,  2.27it/s]
 55%|█████▍    | 51/93 [00:21<00:19,  2.20it/s]
 56%|█████▌    | 52/93 [00:21<00:18,  2.17it/s]
 57%|█████▋    | 53/93 [00:22<00:19,  2.07it/s]
 58%|█████▊    | 54/93 [00:22<00:18,  2.06it/s]
 59%|█████▉    | 55/93 [00:23<00:17,  2.12it/s]
 60%|██████    | 56/93 [00:23<00:15,  2.35it/s]
 61%|██████▏   | 57/93 [00:23<00:15,  2.37it/s]
 62%|██████▏   | 58/93 [00:24<00:13,  2.52it/s]
 63%|██████▎   | 59/93 [00:24<00:14,  2.40it/s]
 65%|██████▍   | 60/93 [00:25<00:14,  2.36it/s]
 66%|██████▌   | 61/93 [00:25<00:13,  2.44it/s]
 67%|██████▋   | 62/93 [00:25<00:13,  2.31it/s]
 68%|██████▊   | 63/93 [00:26<00:12,  2.31it/s]
 69%|██████▉   | 64/93 [00:28<00:26,  1.10it/s]
 70%|██████▉   | 65/93 [00:28<00:21,  1.33it/s]
 71%|███████   | 66/93 [00:29<00:17,  1.56it/s]
 72%|███████▏  | 67/93 [00:29<00:14,  1.76it/s]
 73%|███████▎  | 68/93 [00:30<00:14,  1.76it/s]
 74%|███████▍  | 69/93 [00:30<00:13,  1.84it/s]
 75%|███████▌  | 70/93 [00:31<00:11,  1.92it/s]
 76%|███████▋  | 71/93 [00:31<00:10,  2.05it/s]
 77%|███████▋  | 72/93 [00:32<00:10,  2.06it/s]
 78%|███████▊  | 73/93 [00:32<00:08,  2.26it/s]
 80%|███████▉  | 74/93 [00:32<00:09,  2.06it/s]
 81%|████████  | 75/93 [00:33<00:08,  2.22it/s]
 82%|████████▏ | 76/93 [00:33<00:07,  2.15it/s]
 83%|████████▎ | 77/93 [00:34<00:07,  2.11it/s]
 84%|████████▍ | 78/93 [00:34<00:07,  2.02it/s]
 85%|████████▍ | 79/93 [00:35<00:07,  1.98it/s]
 86%|████████▌ | 80/93 [00:35<00:06,  1.95it/s]
 87%|████████▋ | 81/93 [00:36<00:06,  1.86it/s]
 88%|████████▊ | 82/93 [00:36<00:05,  2.08it/s]
 89%|████████▉ | 83/93 [00:37<00:04,  2.23it/s]
 90%|█████████ | 84/93 [00:37<00:04,  2.17it/s]
 91%|█████████▏| 85/93 [00:38<00:03,  2.16it/s]
 92%|█████████▏| 86/93 [00:38<00:03,  2.25it/s]
 94%|█████████▎| 87/93 [00:39<00:02,  2.25it/s]
 95%|█████████▍| 88/93 [00:39<00:02,  2.15it/s]
 96%|█████████▌| 89/93 [00:40<00:01,  2.12it/s]
 97%|█████████▋| 90/93 [00:40<00:01,  2.18it/s]
 98%|█████████▊| 91/93 [00:41<00:00,  2.05it/s]
 99%|█████████▉| 92/93 [00:41<00:00,  2.15it/s]
100%|██████████| 93/93 [00:41<00:00,  2.32it/s]
100%|██████████| 93/93 [00:41<00:00,  2.21it/s]
***** predict_test_ru_RU metrics *****
  predict_ex_match_acc         =     0.7028
  predict_ex_match_acc_stderr  =     0.0084
  predict_intent_acc           =     0.8769
  predict_intent_acc_stderr    =      0.006
  predict_loss                 =     0.1495
  predict_runtime              = 0:00:42.44
  predict_samples              =       2974
  predict_samples_per_second   =     70.064
  predict_slot_micro_f1        =     0.7874
  predict_slot_micro_f1_stderr =     0.0031
  predict_steps_per_second     =      2.191
06/10/2024 22:43:50 - INFO - __main__ - *** test_tr_TR ***
[INFO|trainer.py:718] 2024-06-10 22:43:50,993 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:43:50,995 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:43:50,996 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:43:50,996 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:17,  5.33it/s]
  3%|▎         | 3/93 [00:00<00:23,  3.85it/s]
  4%|▍         | 4/93 [00:01<00:24,  3.70it/s]
  5%|▌         | 5/93 [00:01<00:28,  3.10it/s]
  6%|▋         | 6/93 [00:01<00:29,  2.96it/s]
  8%|▊         | 7/93 [00:02<00:30,  2.80it/s]
  9%|▊         | 8/93 [00:02<00:29,  2.87it/s]
 10%|▉         | 9/93 [00:03<00:32,  2.55it/s]
 11%|█         | 10/93 [00:03<00:33,  2.47it/s]
 12%|█▏        | 11/93 [00:03<00:33,  2.48it/s]
 13%|█▎        | 12/93 [00:04<00:31,  2.55it/s]
 14%|█▍        | 13/93 [00:04<00:32,  2.49it/s]
 15%|█▌        | 14/93 [00:05<00:35,  2.23it/s]
 16%|█▌        | 15/93 [00:05<00:36,  2.12it/s]
 17%|█▋        | 16/93 [00:06<00:34,  2.25it/s]
 18%|█▊        | 17/93 [00:06<00:33,  2.30it/s]
 19%|█▉        | 18/93 [00:06<00:31,  2.39it/s]
 20%|██        | 19/93 [00:07<00:31,  2.37it/s]
 22%|██▏       | 20/93 [00:07<00:28,  2.53it/s]
 23%|██▎       | 21/93 [00:08<00:27,  2.59it/s]
 24%|██▎       | 22/93 [00:08<00:29,  2.41it/s]
 25%|██▍       | 23/93 [00:08<00:28,  2.47it/s]
 26%|██▌       | 24/93 [00:09<00:28,  2.43it/s]
 27%|██▋       | 25/93 [00:09<00:27,  2.49it/s]
 28%|██▊       | 26/93 [00:10<00:27,  2.42it/s]
 29%|██▉       | 27/93 [00:10<00:26,  2.45it/s]
 30%|███       | 28/93 [00:10<00:24,  2.65it/s]
 31%|███       | 29/93 [00:11<00:23,  2.75it/s]
 32%|███▏      | 30/93 [00:11<00:23,  2.70it/s]
 33%|███▎      | 31/93 [00:11<00:23,  2.59it/s]
 34%|███▍      | 32/93 [00:12<00:23,  2.63it/s]
 35%|███▌      | 33/93 [00:12<00:23,  2.56it/s]
 37%|███▋      | 34/93 [00:13<00:23,  2.54it/s]
 38%|███▊      | 35/93 [00:13<00:22,  2.61it/s]
 39%|███▊      | 36/93 [00:13<00:20,  2.79it/s]
 40%|███▉      | 37/93 [00:14<00:18,  3.07it/s]
 41%|████      | 38/93 [00:14<00:20,  2.71it/s]
 42%|████▏     | 39/93 [00:14<00:19,  2.74it/s]
 43%|████▎     | 40/93 [00:15<00:19,  2.67it/s]
 44%|████▍     | 41/93 [00:15<00:20,  2.58it/s]
 45%|████▌     | 42/93 [00:16<00:20,  2.48it/s]
 46%|████▌     | 43/93 [00:16<00:22,  2.24it/s]
 47%|████▋     | 44/93 [00:17<00:20,  2.34it/s]
 48%|████▊     | 45/93 [00:17<00:19,  2.49it/s]
 49%|████▉     | 46/93 [00:17<00:19,  2.45it/s]
 51%|█████     | 47/93 [00:18<00:19,  2.41it/s]
 52%|█████▏    | 48/93 [00:18<00:18,  2.41it/s]
 53%|█████▎    | 49/93 [00:19<00:17,  2.58it/s]
 54%|█████▍    | 50/93 [00:19<00:18,  2.32it/s]
 55%|█████▍    | 51/93 [00:19<00:17,  2.37it/s]
 56%|█████▌    | 52/93 [00:20<00:17,  2.35it/s]
 57%|█████▋    | 53/93 [00:20<00:18,  2.17it/s]
 58%|█████▊    | 54/93 [00:21<00:18,  2.10it/s]
 59%|█████▉    | 55/93 [00:21<00:17,  2.22it/s]
 60%|██████    | 56/93 [00:22<00:15,  2.41it/s]
 61%|██████▏   | 57/93 [00:22<00:14,  2.44it/s]
 62%|██████▏   | 58/93 [00:22<00:13,  2.60it/s]
 63%|██████▎   | 59/93 [00:23<00:12,  2.72it/s]
 65%|██████▍   | 60/93 [00:23<00:13,  2.52it/s]
 66%|██████▌   | 61/93 [00:24<00:12,  2.56it/s]
 67%|██████▋   | 62/93 [00:24<00:13,  2.36it/s]
 68%|██████▊   | 63/93 [00:25<00:13,  2.30it/s]
 69%|██████▉   | 64/93 [00:27<00:26,  1.11it/s]
 70%|██████▉   | 65/93 [00:27<00:20,  1.36it/s]
 71%|███████   | 66/93 [00:27<00:16,  1.59it/s]
 72%|███████▏  | 67/93 [00:28<00:14,  1.78it/s]
 73%|███████▎  | 68/93 [00:28<00:13,  1.81it/s]
 74%|███████▍  | 69/93 [00:29<00:12,  1.96it/s]
 75%|███████▌  | 70/93 [00:29<00:11,  2.09it/s]
 76%|███████▋  | 71/93 [00:29<00:09,  2.20it/s]
 77%|███████▋  | 72/93 [00:30<00:09,  2.22it/s]
 78%|███████▊  | 73/93 [00:30<00:08,  2.40it/s]
 80%|███████▉  | 74/93 [00:31<00:08,  2.33it/s]
 81%|████████  | 75/93 [00:31<00:07,  2.37it/s]
 82%|████████▏ | 76/93 [00:31<00:07,  2.39it/s]
 83%|████████▎ | 77/93 [00:32<00:06,  2.52it/s]
 84%|████████▍ | 78/93 [00:32<00:05,  2.57it/s]
 85%|████████▍ | 79/93 [00:33<00:06,  2.19it/s]
 86%|████████▌ | 80/93 [00:33<00:06,  2.01it/s]
 87%|████████▋ | 81/93 [00:34<00:05,  2.05it/s]
 88%|████████▊ | 82/93 [00:34<00:04,  2.23it/s]
 89%|████████▉ | 83/93 [00:35<00:04,  2.33it/s]
 90%|█████████ | 84/93 [00:35<00:03,  2.25it/s]
 91%|█████████▏| 85/93 [00:35<00:03,  2.27it/s]
 92%|█████████▏| 86/93 [00:36<00:02,  2.36it/s]
 94%|█████████▎| 87/93 [00:36<00:02,  2.32it/s]
 95%|█████████▍| 88/93 [00:37<00:02,  2.29it/s]
 96%|█████████▌| 89/93 [00:37<00:01,  2.33it/s]
 97%|█████████▋| 90/93 [00:38<00:01,  2.40it/s]
 98%|█████████▊| 91/93 [00:38<00:00,  2.26it/s]
 99%|█████████▉| 92/93 [00:38<00:00,  2.32it/s]
100%|██████████| 93/93 [00:39<00:00,  2.48it/s]
100%|██████████| 93/93 [00:39<00:00,  2.35it/s]
***** predict_test_tr_TR metrics *****
  predict_ex_match_acc         =     0.6856
  predict_ex_match_acc_stderr  =     0.0085
  predict_intent_acc           =     0.8675
  predict_intent_acc_stderr    =     0.0062
  predict_loss                 =     0.1582
  predict_runtime              = 0:00:39.89
  predict_samples              =       2974
  predict_samples_per_second   =      74.54
  predict_slot_micro_f1        =     0.7798
  predict_slot_micro_f1_stderr =     0.0033
  predict_steps_per_second     =      2.331
06/10/2024 22:44:31 - INFO - __main__ - *** test_vi_VN ***
[INFO|trainer.py:718] 2024-06-10 22:44:31,087 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:44:31,090 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:44:31,091 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:44:31,091 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:31,  2.86it/s]
  3%|▎         | 3/93 [00:01<00:36,  2.44it/s]
  4%|▍         | 4/93 [00:01<00:38,  2.33it/s]
  5%|▌         | 5/93 [00:02<00:42,  2.05it/s]
  6%|▋         | 6/93 [00:02<00:42,  2.06it/s]
  8%|▊         | 7/93 [00:03<00:43,  1.97it/s]
  9%|▊         | 8/93 [00:03<00:41,  2.03it/s]
 10%|▉         | 9/93 [00:04<00:40,  2.07it/s]
 11%|█         | 10/93 [00:04<00:41,  2.00it/s]
 12%|█▏        | 11/93 [00:05<00:43,  1.89it/s]
 13%|█▎        | 12/93 [00:05<00:43,  1.85it/s]
 14%|█▍        | 13/93 [00:06<00:41,  1.94it/s]
 15%|█▌        | 14/93 [00:07<00:45,  1.72it/s]
 16%|█▌        | 15/93 [00:07<00:44,  1.74it/s]
 17%|█▋        | 16/93 [00:08<00:44,  1.73it/s]
 18%|█▊        | 17/93 [00:08<00:45,  1.69it/s]
 19%|█▉        | 18/93 [00:09<00:42,  1.78it/s]
 20%|██        | 19/93 [00:09<00:42,  1.75it/s]
 22%|██▏       | 20/93 [00:10<00:40,  1.80it/s]
 23%|██▎       | 21/93 [00:11<00:39,  1.82it/s]
 24%|██▎       | 22/93 [00:11<00:38,  1.84it/s]
 25%|██▍       | 23/93 [00:12<00:37,  1.87it/s]
 26%|██▌       | 24/93 [00:12<00:38,  1.82it/s]
 27%|██▋       | 25/93 [00:14<00:54,  1.25it/s]
 28%|██▊       | 26/93 [00:14<00:48,  1.38it/s]
 29%|██▉       | 27/93 [00:15<00:42,  1.55it/s]
 30%|███       | 28/93 [00:15<00:37,  1.73it/s]
 31%|███       | 29/93 [00:15<00:34,  1.84it/s]
 32%|███▏      | 30/93 [00:16<00:33,  1.86it/s]
 33%|███▎      | 31/93 [00:17<00:33,  1.83it/s]
 34%|███▍      | 32/93 [00:17<00:33,  1.82it/s]
 35%|███▌      | 33/93 [00:18<00:33,  1.78it/s]
 37%|███▋      | 34/93 [00:18<00:36,  1.62it/s]
 38%|███▊      | 35/93 [00:19<00:33,  1.75it/s]
 39%|███▊      | 36/93 [00:19<00:33,  1.72it/s]
 40%|███▉      | 37/93 [00:20<00:31,  1.80it/s]
 41%|████      | 38/93 [00:21<00:35,  1.55it/s]
 42%|████▏     | 39/93 [00:21<00:32,  1.64it/s]
 43%|████▎     | 40/93 [00:22<00:32,  1.64it/s]
 44%|████▍     | 41/93 [00:23<00:33,  1.55it/s]
 45%|████▌     | 42/93 [00:23<00:33,  1.54it/s]
 46%|████▌     | 43/93 [00:24<00:33,  1.49it/s]
 47%|████▋     | 44/93 [00:25<00:32,  1.52it/s]
 48%|████▊     | 45/93 [00:25<00:30,  1.59it/s]
 49%|████▉     | 46/93 [00:26<00:30,  1.55it/s]
 51%|█████     | 47/93 [00:27<00:28,  1.61it/s]
 52%|█████▏    | 48/93 [00:27<00:27,  1.65it/s]
 53%|█████▎    | 49/93 [00:28<00:25,  1.72it/s]
 54%|█████▍    | 50/93 [00:28<00:25,  1.71it/s]
 55%|█████▍    | 51/93 [00:29<00:24,  1.75it/s]
 56%|█████▌    | 52/93 [00:29<00:23,  1.75it/s]
 57%|█████▋    | 53/93 [00:30<00:23,  1.69it/s]
 58%|█████▊    | 54/93 [00:31<00:24,  1.57it/s]
 59%|█████▉    | 55/93 [00:31<00:23,  1.65it/s]
 60%|██████    | 56/93 [00:32<00:20,  1.81it/s]
 61%|██████▏   | 57/93 [00:32<00:18,  1.90it/s]
 62%|██████▏   | 58/93 [00:33<00:18,  1.90it/s]
 63%|██████▎   | 59/93 [00:33<00:17,  1.92it/s]
 65%|██████▍   | 60/93 [00:34<00:16,  1.95it/s]
 66%|██████▌   | 61/93 [00:34<00:15,  2.06it/s]
 67%|██████▋   | 62/93 [00:35<00:18,  1.72it/s]
 68%|██████▊   | 63/93 [00:36<00:17,  1.67it/s]
 69%|██████▉   | 64/93 [00:38<00:30,  1.05s/it]
 70%|██████▉   | 65/93 [00:38<00:25,  1.10it/s]
 71%|███████   | 66/93 [00:39<00:21,  1.25it/s]
 72%|███████▏  | 67/93 [00:39<00:19,  1.35it/s]
 73%|███████▎  | 68/93 [00:40<00:18,  1.32it/s]
 74%|███████▍  | 69/93 [00:41<00:16,  1.42it/s]
 75%|███████▌  | 70/93 [00:41<00:15,  1.51it/s]
 76%|███████▋  | 71/93 [00:42<00:14,  1.51it/s]
 77%|███████▋  | 72/93 [00:43<00:14,  1.50it/s]
 78%|███████▊  | 73/93 [00:43<00:12,  1.63it/s]
 80%|███████▉  | 74/93 [00:44<00:11,  1.70it/s]
 81%|████████  | 75/93 [00:44<00:10,  1.74it/s]
 82%|████████▏ | 76/93 [00:46<00:16,  1.02it/s]
 83%|████████▎ | 77/93 [00:47<00:13,  1.16it/s]
 84%|████████▍ | 78/93 [00:47<00:11,  1.31it/s]
 85%|████████▍ | 79/93 [00:48<00:10,  1.38it/s]
 86%|████████▌ | 80/93 [00:50<00:14,  1.10s/it]
 87%|████████▋ | 81/93 [00:51<00:11,  1.01it/s]
 88%|████████▊ | 82/93 [00:51<00:09,  1.20it/s]
 89%|████████▉ | 83/93 [00:52<00:07,  1.31it/s]
 90%|█████████ | 84/93 [00:52<00:06,  1.41it/s]
 91%|█████████▏| 85/93 [00:53<00:05,  1.47it/s]
 92%|█████████▏| 86/93 [00:53<00:04,  1.61it/s]
 94%|█████████▎| 87/93 [00:54<00:03,  1.53it/s]
 95%|█████████▍| 88/93 [00:55<00:03,  1.61it/s]
 96%|█████████▌| 89/93 [00:55<00:02,  1.71it/s]
 97%|█████████▋| 90/93 [00:56<00:01,  1.73it/s]
 98%|█████████▊| 91/93 [00:56<00:01,  1.67it/s]
 99%|█████████▉| 92/93 [00:57<00:00,  1.73it/s]
100%|██████████| 93/93 [00:57<00:00,  1.82it/s]
100%|██████████| 93/93 [00:58<00:00,  1.60it/s]
***** predict_test_vi_VN metrics *****
  predict_ex_match_acc         =     0.6557
  predict_ex_match_acc_stderr  =     0.0087
  predict_intent_acc           =     0.8608
  predict_intent_acc_stderr    =     0.0063
  predict_loss                 =       0.13
  predict_runtime              = 0:00:58.58
  predict_samples              =       2974
  predict_samples_per_second   =     50.767
  predict_slot_micro_f1        =     0.7441
  predict_slot_micro_f1_stderr =     0.0027
  predict_steps_per_second     =      1.588
06/10/2024 22:45:29 - INFO - __main__ - *** test_ar_SA ***
[INFO|trainer.py:718] 2024-06-10 22:45:29,940 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:45:29,942 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:45:29,943 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:45:29,943 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:16,  5.41it/s]
  3%|▎         | 3/93 [00:00<00:24,  3.74it/s]
  4%|▍         | 4/93 [00:01<00:25,  3.47it/s]
  5%|▌         | 5/93 [00:01<00:30,  2.85it/s]
  6%|▋         | 6/93 [00:01<00:30,  2.87it/s]
  8%|▊         | 7/93 [00:02<00:30,  2.81it/s]
  9%|▊         | 8/93 [00:02<00:30,  2.79it/s]
 10%|▉         | 9/93 [00:03<00:33,  2.49it/s]
 11%|█         | 10/93 [00:03<00:33,  2.49it/s]
 12%|█▏        | 11/93 [00:03<00:33,  2.42it/s]
 13%|█▎        | 12/93 [00:04<00:32,  2.51it/s]
 14%|█▍        | 13/93 [00:04<00:32,  2.45it/s]
 15%|█▌        | 14/93 [00:05<00:36,  2.18it/s]
 16%|█▌        | 15/93 [00:05<00:36,  2.16it/s]
 17%|█▋        | 16/93 [00:06<00:34,  2.22it/s]
 18%|█▊        | 17/93 [00:06<00:33,  2.25it/s]
 19%|█▉        | 18/93 [00:07<00:33,  2.22it/s]
 20%|██        | 19/93 [00:07<00:33,  2.21it/s]
 22%|██▏       | 20/93 [00:07<00:30,  2.37it/s]
 23%|██▎       | 21/93 [00:08<00:29,  2.45it/s]
 24%|██▎       | 22/93 [00:08<00:29,  2.41it/s]
 25%|██▍       | 23/93 [00:09<00:29,  2.41it/s]
 26%|██▌       | 24/93 [00:09<00:29,  2.34it/s]
 27%|██▋       | 25/93 [00:09<00:26,  2.54it/s]
 28%|██▊       | 26/93 [00:10<00:26,  2.57it/s]
 29%|██▉       | 27/93 [00:10<00:24,  2.65it/s]
 30%|███       | 28/93 [00:10<00:23,  2.74it/s]
 31%|███       | 29/93 [00:11<00:24,  2.63it/s]
 32%|███▏      | 30/93 [00:11<00:24,  2.61it/s]
 33%|███▎      | 31/93 [00:12<00:24,  2.54it/s]
 34%|███▍      | 32/93 [00:12<00:24,  2.52it/s]
 35%|███▌      | 33/93 [00:13<00:26,  2.31it/s]
 37%|███▋      | 34/93 [00:13<00:24,  2.41it/s]
 38%|███▊      | 35/93 [00:13<00:23,  2.43it/s]
 39%|███▊      | 36/93 [00:14<00:22,  2.58it/s]
 40%|███▉      | 37/93 [00:14<00:20,  2.70it/s]
 41%|████      | 38/93 [00:14<00:21,  2.60it/s]
 42%|████▏     | 39/93 [00:15<00:19,  2.80it/s]
 43%|████▎     | 40/93 [00:15<00:20,  2.62it/s]
 44%|████▍     | 41/93 [00:16<00:20,  2.54it/s]
 45%|████▌     | 42/93 [00:16<00:20,  2.44it/s]
 46%|████▌     | 43/93 [00:17<00:22,  2.20it/s]
 47%|████▋     | 44/93 [00:17<00:20,  2.38it/s]
 48%|████▊     | 45/93 [00:17<00:18,  2.55it/s]
 49%|████▉     | 46/93 [00:18<00:18,  2.53it/s]
 51%|█████     | 47/93 [00:18<00:18,  2.43it/s]
 52%|█████▏    | 48/93 [00:18<00:17,  2.59it/s]
 53%|█████▎    | 49/93 [00:19<00:16,  2.74it/s]
 54%|█████▍    | 50/93 [00:19<00:17,  2.50it/s]
 55%|█████▍    | 51/93 [00:20<00:17,  2.38it/s]
 56%|█████▌    | 52/93 [00:20<00:18,  2.23it/s]
 57%|█████▋    | 53/93 [00:21<00:20,  1.98it/s]
 58%|█████▊    | 54/93 [00:21<00:19,  2.03it/s]
 59%|█████▉    | 55/93 [00:22<00:17,  2.17it/s]
 60%|██████    | 56/93 [00:22<00:16,  2.26it/s]
 61%|██████▏   | 57/93 [00:23<00:16,  2.12it/s]
 62%|██████▏   | 58/93 [00:23<00:15,  2.28it/s]
 63%|██████▎   | 59/93 [00:23<00:14,  2.33it/s]
 65%|██████▍   | 60/93 [00:24<00:14,  2.32it/s]
 66%|██████▌   | 61/93 [00:24<00:12,  2.48it/s]
 67%|██████▋   | 62/93 [00:25<00:13,  2.37it/s]
 68%|██████▊   | 63/93 [00:25<00:12,  2.40it/s]
 69%|██████▉   | 64/93 [00:27<00:25,  1.13it/s]
 70%|██████▉   | 65/93 [00:28<00:21,  1.31it/s]
 71%|███████   | 66/93 [00:28<00:18,  1.48it/s]
 72%|███████▏  | 67/93 [00:28<00:14,  1.75it/s]
 73%|███████▎  | 68/93 [00:29<00:13,  1.82it/s]
 74%|███████▍  | 69/93 [00:29<00:12,  2.00it/s]
 75%|███████▌  | 70/93 [00:30<00:10,  2.19it/s]
 76%|███████▋  | 71/93 [00:30<00:09,  2.30it/s]
 77%|███████▋  | 72/93 [00:31<00:09,  2.16it/s]
 78%|███████▊  | 73/93 [00:31<00:08,  2.35it/s]
 80%|███████▉  | 74/93 [00:31<00:08,  2.28it/s]
 81%|████████  | 75/93 [00:32<00:07,  2.39it/s]
 82%|████████▏ | 76/93 [00:32<00:07,  2.33it/s]
 83%|████████▎ | 77/93 [00:33<00:06,  2.32it/s]
 84%|████████▍ | 78/93 [00:33<00:06,  2.30it/s]
 85%|████████▍ | 79/93 [00:34<00:06,  2.03it/s]
 86%|████████▌ | 80/93 [00:34<00:06,  2.01it/s]
 87%|████████▋ | 81/93 [00:35<00:05,  2.04it/s]
 88%|████████▊ | 82/93 [00:35<00:04,  2.23it/s]
 89%|████████▉ | 83/93 [00:35<00:04,  2.22it/s]
 90%|█████████ | 84/93 [00:36<00:04,  2.16it/s]
 91%|█████████▏| 85/93 [00:36<00:03,  2.29it/s]
 92%|█████████▏| 86/93 [00:37<00:02,  2.35it/s]
 94%|█████████▎| 87/93 [00:37<00:02,  2.41it/s]
 95%|█████████▍| 88/93 [00:38<00:02,  2.42it/s]
 96%|█████████▌| 89/93 [00:38<00:01,  2.36it/s]
 97%|█████████▋| 90/93 [00:38<00:01,  2.39it/s]
 98%|█████████▊| 91/93 [00:39<00:00,  2.25it/s]
 99%|█████████▉| 92/93 [00:39<00:00,  2.28it/s]
100%|██████████| 93/93 [00:40<00:00,  2.31it/s]
100%|██████████| 93/93 [00:40<00:00,  2.30it/s]
***** predict_test_ar_SA metrics *****
  predict_ex_match_acc         =     0.6422
  predict_ex_match_acc_stderr  =     0.0088
  predict_intent_acc           =     0.8255
  predict_intent_acc_stderr    =      0.007
  predict_loss                 =     0.1855
  predict_runtime              = 0:00:40.74
  predict_samples              =       2974
  predict_samples_per_second   =     72.983
  predict_slot_micro_f1        =     0.7601
  predict_slot_micro_f1_stderr =     0.0034
  predict_steps_per_second     =      2.282
06/10/2024 22:46:10 - INFO - __main__ - *** test_ko_KR ***
[INFO|trainer.py:718] 2024-06-10 22:46:10,882 >> The following columns in the test set don't have a corresponding argument in `MT5ForConditionalGeneration.forward` and have been ignored: intent_str, annot_utt, locale, id. If intent_str, annot_utt, locale, id are not expected by `MT5ForConditionalGeneration.forward`,  you can safely ignore this message.
[INFO|trainer.py:3199] 2024-06-10 22:46:10,885 >> ***** Running Prediction *****
[INFO|trainer.py:3201] 2024-06-10 22:46:10,886 >>   Num examples = 2974
[INFO|trainer.py:3204] 2024-06-10 22:46:10,886 >>   Batch size = 32

  0%|          | 0/93 [00:00<?, ?it/s]
  2%|▏         | 2/93 [00:00<00:14,  6.13it/s]
  3%|▎         | 3/93 [00:00<00:21,  4.23it/s]
  4%|▍         | 4/93 [00:01<00:24,  3.56it/s]
  5%|▌         | 5/93 [00:01<00:30,  2.87it/s]
  6%|▋         | 6/93 [00:01<00:29,  2.96it/s]
  8%|▊         | 7/93 [00:02<00:29,  2.87it/s]
  9%|▊         | 8/93 [00:02<00:29,  2.89it/s]
 10%|▉         | 9/93 [00:02<00:29,  2.87it/s]
 11%|█         | 10/93 [00:03<00:29,  2.80it/s]
 12%|█▏        | 11/93 [00:03<00:29,  2.75it/s]
 13%|█▎        | 12/93 [00:03<00:28,  2.80it/s]
 14%|█▍        | 13/93 [00:04<00:29,  2.74it/s]
 15%|█▌        | 14/93 [00:04<00:30,  2.58it/s]
 16%|█▌        | 15/93 [00:05<00:30,  2.60it/s]
 17%|█▋        | 16/93 [00:05<00:28,  2.66it/s]
 18%|█▊        | 17/93 [00:06<00:31,  2.45it/s]
 19%|█▉        | 18/93 [00:06<00:29,  2.56it/s]
 20%|██        | 19/93 [00:06<00:29,  2.53it/s]
 22%|██▏       | 20/93 [00:07<00:28,  2.59it/s]
 23%|██▎       | 21/93 [00:07<00:26,  2.71it/s]
 24%|██▎       | 22/93 [00:07<00:28,  2.45it/s]
 25%|██▍       | 23/93 [00:08<00:28,  2.43it/s]
 26%|██▌       | 24/93 [00:08<00:27,  2.52it/s]
 27%|██▋       | 25/93 [00:09<00:27,  2.50it/s]
 28%|██▊       | 26/93 [00:09<00:27,  2.46it/s]
 29%|██▉       | 27/93 [00:09<00:25,  2.60it/s]
 30%|███       | 28/93 [00:10<00:23,  2.73it/s]
 31%|███       | 29/93 [00:10<00:23,  2.76it/s]
 32%|███▏      | 30/93 [00:11<00:23,  2.64it/s]
 33%|███▎      | 31/93 [00:11<00:28,  2.21it/s]
 34%|███▍      | 32/93 [00:11<00:25,  2.42it/s]
 35%|███▌      | 33/93 [00:12<00:24,  2.48it/s]
 37%|███▋      | 34/93 [00:12<00:22,  2.63it/s]
 38%|███▊      | 35/93 [00:13<00:21,  2.68it/s]
 39%|███▊      | 36/93 [00:13<00:19,  2.87it/s]
 40%|███▉      | 37/93 [00:13<00:18,  2.95it/s]
 41%|████      | 38/93 [00:13<00:19,  2.87it/s]
 42%|████▏     | 39/93 [00:14<00:17,  3.02it/s]
 43%|████▎     | 40/93 [00:14<00:17,  2.96it/s]
 44%|████▍     | 41/93 [00:14<00:17,  2.99it/s]
 45%|████▌     | 42/93 [00:15<00:18,  2.69it/s]
 46%|████▌     | 43/93 [00:15<00:19,  2.58it/s]
 47%|████▋     | 44/93 [00:16<00:19,  2.55it/s]
 48%|████▊     | 45/93 [00:16<00:17,  2.71it/s]
 49%|████▉     | 46/93 [00:16<00:16,  2.80it/s]
 51%|█████     | 47/93 [00:17<00:17,  2.63it/s]
 52%|█████▏    | 48/93 [00:17<00:16,  2.75it/s]
 53%|█████▎    | 49/93 [00:17<00:15,  2.91it/s]
 54%|█████▍    | 50/93 [00:18<00:15,  2.85it/s]
 55%|█████▍    | 51/93 [00:18<00:15,  2.79it/s]
 56%|█████▌    | 52/93 [00:19<00:15,  2.65it/s]
 57%|█████▋    | 53/93 [00:19<00:15,  2.60it/s]
 58%|█████▊    | 54/93 [00:19<00:14,  2.62it/s]
 59%|█████▉    | 55/93 [00:20<00:13,  2.73it/s]
 60%|██████    | 56/93 [00:20<00:13,  2.83it/s]
 61%|██████▏   | 57/93 [00:20<00:12,  2.92it/s]
 62%|██████▏   | 58/93 [00:21<00:11,  2.94it/s]
 63%|██████▎   | 59/93 [00:21<00:11,  3.00it/s]
 65%|██████▍   | 60/93 [00:21<00:10,  3.05it/s]
 66%|██████▌   | 61/93 [00:22<00:10,  3.09it/s]
 67%|██████▋   | 62/93 [00:22<00:10,  2.82it/s]
 68%|██████▊   | 63/93 [00:22<00:10,  2.87it/s]
 69%|██████▉   | 64/93 [00:23<00:12,  2.36it/s]
 70%|██████▉   | 65/93 [00:23<00:11,  2.51it/s]
 71%|███████   | 66/93 [00:24<00:11,  2.42it/s]
 72%|███████▏  | 67/93 [00:24<00:10,  2.47it/s]
 73%|███████▎  | 68/93 [00:25<00:10,  2.48it/s]
 74%|███████▍  | 69/93 [00:25<00:09,  2.47it/s]
 75%|███████▌  | 70/93 [00:25<00:09,  2.49it/s]
 76%|███████▋  | 71/93 [00:26<00:08,  2.67it/s]
 77%|███████▋  | 72/93 [00:26<00:08,  2.55it/s]
 78%|███████▊  | 73/93 [00:26<00:07,  2.74it/s]
 80%|███████▉  | 74/93 [00:27<00:07,  2.57it/s]
 81%|████████  | 75/93 [00:27<00:06,  2.79it/s]
 82%|████████▏ | 76/93 [00:28<00:06,  2.55it/s]
 83%|████████▎ | 77/93 [00:28<00:06,  2.66it/s]
 84%|████████▍ | 78/93 [00:28<00:05,  2.70it/s]
 85%|████████▍ | 79/93 [00:29<00:06,  2.27it/s]
 86%|████████▌ | 80/93 [00:29<00:05,  2.20it/s]
 87%|████████▋ | 81/93 [00:30<00:05,  2.32it/s]
 88%|████████▊ | 82/93 [00:30<00:04,  2.46it/s]
 89%|████████▉ | 83/93 [00:31<00:03,  2.52it/s]
 90%|█████████ | 84/93 [00:31<00:03,  2.34it/s]
 91%|█████████▏| 85/93 [00:31<00:03,  2.46it/s]
 92%|█████████▏| 86/93 [00:32<00:02,  2.60it/s]
 94%|█████████▎| 87/93 [00:32<00:02,  2.67it/s]
 95%|█████████▍| 88/93 [00:32<00:01,  2.74it/s]
 96%|█████████▌| 89/93 [00:33<00:01,  2.65it/s]
 97%|█████████▋| 90/93 [00:33<00:01,  2.62it/s]
 98%|█████████▊| 91/93 [00:34<00:00,  2.39it/s]
 99%|█████████▉| 92/93 [00:34<00:00,  2.37it/s]
100%|██████████| 93/93 [00:34<00:00,  2.50it/s]
100%|██████████| 93/93 [00:35<00:00,  2.64it/s]
***** predict_test_ko_KR metrics *****
  predict_ex_match_acc         =     0.6967
  predict_ex_match_acc_stderr  =     0.0084
  predict_intent_acc           =     0.8642
  predict_intent_acc_stderr    =     0.0063
  predict_loss                 =     0.1606
  predict_runtime              = 0:00:35.57
  predict_samples              =       2974
  predict_samples_per_second   =     83.608
  predict_slot_micro_f1        =     0.8051
  predict_slot_micro_f1_stderr =     0.0033
  predict_steps_per_second     =      2.614