FredNajjar commited on
Commit
7f7b012
1 Parent(s): 6a1d245

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -58
README.md CHANGED
@@ -1,64 +1,65 @@
1
- ---
2
- license: apache-2.0
3
- base_model: google/bigbird-roberta-base
4
  tags:
5
- - generated_from_trainer
 
 
 
6
  datasets:
7
  - squad_v2
8
- model-index:
9
- - name: bigbird-QA-squad_v2.2
10
- results: []
11
  ---
12
 
13
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
- should probably proofread and complete it, then remove this comment. -->
15
-
16
- # bigbird-QA-squad_v2.2
17
-
18
- This model is a fine-tuned version of [google/bigbird-roberta-base](https://huggingface.co/google/bigbird-roberta-base) on the squad_v2 dataset.
19
- It achieves the following results on the evaluation set:
20
- - Loss: 0.8585
21
-
22
- ## Model description
23
-
24
- More information needed
25
-
26
- ## Intended uses & limitations
27
-
28
- More information needed
29
-
30
- ## Training and evaluation data
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
- More information needed
33
-
34
- ## Training procedure
35
-
36
- ### Training hyperparameters
37
-
38
- The following hyperparameters were used during training:
39
- - learning_rate: 3e-05
40
- - train_batch_size: 16
41
- - eval_batch_size: 8
42
- - seed: 42
43
- - gradient_accumulation_steps: 8
44
- - total_train_batch_size: 128
45
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
- - lr_scheduler_type: linear
47
- - lr_scheduler_warmup_steps: 121
48
- - num_epochs: 3
49
-
50
- ### Training results
51
-
52
- | Training Loss | Epoch | Step | Validation Loss |
53
- |:-------------:|:-----:|:----:|:---------------:|
54
- | 1.0955 | 1.0 | 814 | 0.9719 |
55
- | 0.8505 | 2.0 | 1629 | 0.8657 |
56
- | 0.6993 | 3.0 | 2442 | 0.8585 |
57
-
58
-
59
- ### Framework versions
60
-
61
- - Transformers 4.34.0
62
- - Pytorch 2.0.1+cu118
63
- - Datasets 2.14.5
64
- - Tokenizers 0.14.1
 
1
+ $---
2
+ language: english
 
3
  tags:
4
+ - bigbird
5
+ - question-answering
6
+ - squad-v2.2
7
+ license: apache-2.0
8
  datasets:
9
  - squad_v2
10
+ metrics:
11
+ - f1
12
+ - exact_match
13
  ---
14
 
15
+ # FredNajjar/bigbird-QA-squad_v2.2
16
+
17
+ Fine-tuned [`google/bigbird-roberta-base`](https://huggingface.co/google/bigbird-roberta-base) model on the SQuAD 2.0 dataset for English extractive question answering.
18
+
19
+ ## Model Details
20
+ - **Language Model**: [google/bigbird-roberta-base](https://huggingface.co/google/bigbird-roberta-base)
21
+ - **Language**: English
22
+ - **Task**: Extractive QA
23
+ - **Training Data**: [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/)
24
+ - **Eval Data**: [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/)
25
+ - **Framework Versions**:
26
+ - Transformers: 4.34.0
27
+ - Pytorch: 2.0.1+cu118
28
+ - Datasets: 2.14.5
29
+ - Tokenizers: 0.14.1
30
+ - **Infrastructure**: 1x Tesla A100
31
+
32
+ ## Training Hyperparameters
33
+ - Learning Rate: 3e-05
34
+ - Train Batch Size: 16
35
+ - Eval Batch Size: 8
36
+ - Seed: 42
37
+ - Gradient Accumulation Steps: 8
38
+ - Total Train Batch Size: 128
39
+ - Optimizer: Adam (betas=(0.9,0.999), epsilon=1e-08)
40
+ - LR Scheduler: Linear with 121 warmup steps
41
+ - Number of Epochs: 3
42
+
43
+ ## Results on SQuAD 2.0
44
+ - **F1 Score**: 81.39%
45
+ - **Exact Match**: 77.82%
46
+
47
+ ## Usage
48
+ ```python
49
+ from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
50
+ model_name = "FredNajjar/bigbird-QA-squad_v2.2"
51
+ nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
52
+ QA_input = {
53
+ 'question': 'Your question here',
54
+ 'context': 'Your context here'
55
+ }
56
+ res = nlp(QA_input)
57
+ ```
58
+
59
+ ## Limitations and Bias
60
+ This model inherits limitations and potential biases from the base BigBird model and the SQuAD 2.0 training data.
61
+
62
+ ## Contact
63
+ For inquiries, please reach out via [LinkedIn](https://www.linkedin.com/in/frednajjar/).
64
 
65
+ ---