Mohamed Boghdady commited on
Commit
4cdab3b
1 Parent(s): 61df0cf

English to arabic model trained using Quran-Classical-Arabic-English-Parallel-texts

Browse files
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
  license: apache-2.0
3
- base_model: Helsinki-NLP/opus-mt-ar-en
4
  tags:
5
  - generated_from_trainer
6
  metrics:
@@ -13,14 +13,15 @@ model-index:
13
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
  should probably proofread and complete it, then remove this comment. -->
15
 
16
- [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/boghdady95/huggingface/runs/oy10h8oj)
 
17
  # working
18
 
19
- This model is a fine-tuned version of [Helsinki-NLP/opus-mt-ar-en](https://huggingface.co/Helsinki-NLP/opus-mt-ar-en) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.2987
22
- - Bleu: 31.8518
23
- - Gen Len: 34.0136
24
 
25
  ## Model description
26
 
@@ -52,16 +53,16 @@ The following hyperparameters were used during training:
52
 
53
  | Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
54
  |:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|
55
- | No log | 1.0 | 312 | 0.3560 | 25.127 | 33.2927 |
56
- | 0.6275 | 2.0 | 624 | 0.3309 | 27.7903 | 34.3769 |
57
- | 0.6275 | 3.0 | 936 | 0.3176 | 28.8587 | 33.1307 |
58
- | 0.2524 | 4.0 | 1248 | 0.3098 | 29.8775 | 33.5782 |
59
- | 0.219 | 5.0 | 1560 | 0.3057 | 30.629 | 33.6736 |
60
- | 0.219 | 6.0 | 1872 | 0.3026 | 31.2601 | 33.854 |
61
- | 0.2029 | 7.0 | 2184 | 0.3007 | 31.6296 | 33.9687 |
62
- | 0.2029 | 8.0 | 2496 | 0.2997 | 31.6706 | 33.6327 |
63
- | 0.1895 | 9.0 | 2808 | 0.2989 | 31.5995 | 33.9904 |
64
- | 0.182 | 10.0 | 3120 | 0.2987 | 31.8518 | 34.0136 |
65
 
66
 
67
  ### Framework versions
 
1
  ---
2
  license: apache-2.0
3
+ base_model: Helsinki-NLP/opus-mt-en-ar
4
  tags:
5
  - generated_from_trainer
6
  metrics:
 
13
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
  should probably proofread and complete it, then remove this comment. -->
15
 
16
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/boghdady95/huggingface/runs/wq61zns9)
17
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/boghdady95/huggingface/runs/wq61zns9)
18
  # working
19
 
20
+ This model is a fine-tuned version of [Helsinki-NLP/opus-mt-en-ar](https://huggingface.co/Helsinki-NLP/opus-mt-en-ar) on an unknown dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 0.1143
23
+ - Bleu: 21.836
24
+ - Gen Len: 67.4314
25
 
26
  ## Model description
27
 
 
53
 
54
  | Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
55
  |:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|
56
+ | No log | 1.0 | 312 | 0.1450 | 13.6092 | 66.5229 |
57
+ | 0.1446 | 2.0 | 624 | 0.1337 | 15.8043 | 67.4747 |
58
+ | 0.1446 | 3.0 | 936 | 0.1267 | 17.6106 | 67.2133 |
59
+ | 0.1222 | 4.0 | 1248 | 0.1217 | 18.8785 | 67.4451 |
60
+ | 0.1031 | 5.0 | 1560 | 0.1190 | 19.5595 | 67.1748 |
61
+ | 0.1031 | 6.0 | 1872 | 0.1170 | 20.6884 | 67.7779 |
62
+ | 0.0937 | 7.0 | 2184 | 0.1158 | 21.133 | 67.3496 |
63
+ | 0.0937 | 8.0 | 2496 | 0.1148 | 21.4563 | 67.4739 |
64
+ | 0.0859 | 9.0 | 2808 | 0.1144 | 21.8952 | 67.3031 |
65
+ | 0.0814 | 10.0 | 3120 | 0.1143 | 21.836 | 67.4314 |
66
 
67
 
68
  ### Framework versions
generation_config.json CHANGED
@@ -1,16 +1,16 @@
1
  {
2
  "bad_words_ids": [
3
  [
4
- 62833
5
  ]
6
  ],
7
  "bos_token_id": 0,
8
- "decoder_start_token_id": 62833,
9
  "eos_token_id": 0,
10
  "forced_eos_token_id": 0,
11
  "max_length": 512,
12
  "num_beams": 4,
13
- "pad_token_id": 62833,
14
  "renormalize_logits": true,
15
  "transformers_version": "4.42.3"
16
  }
 
1
  {
2
  "bad_words_ids": [
3
  [
4
+ 62801
5
  ]
6
  ],
7
  "bos_token_id": 0,
8
+ "decoder_start_token_id": 62801,
9
  "eos_token_id": 0,
10
  "forced_eos_token_id": 0,
11
  "max_length": 512,
12
  "num_beams": 4,
13
+ "pad_token_id": 62801,
14
  "renormalize_logits": true,
15
  "transformers_version": "4.42.3"
16
  }
runs/Jul19_15-13-23_8df4908137f9/events.out.tfevents.1721402005.8df4908137f9.35.1 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fc17085243a662bf817d7b60895d82b3e643bc4691b248a3edea859e4ca73dc2
3
- size 10332
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1264558a16e0636e1fbb89f498cc91b768b65c2da9191f05125f294f8b6d580
3
+ size 11056
wandb/debug-internal.log CHANGED
@@ -7578,3 +7578,414 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
7578
  2024-07-19 16:33:34,764 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7579
  2024-07-19 16:33:34,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7580
  2024-07-19 16:33:35,768 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7578
  2024-07-19 16:33:34,764 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7579
  2024-07-19 16:33:34,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7580
  2024-07-19 16:33:35,768 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7581
+ 2024-07-19 16:33:36,768 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7582
+ 2024-07-19 16:33:37,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7583
+ 2024-07-19 16:33:38,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7584
+ 2024-07-19 16:33:39,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7585
+ 2024-07-19 16:33:39,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7586
+ 2024-07-19 16:33:40,636 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7587
+ 2024-07-19 16:33:40,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7588
+ 2024-07-19 16:33:40,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7589
+ 2024-07-19 16:33:41,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7590
+ 2024-07-19 16:33:42,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7591
+ 2024-07-19 16:33:43,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7592
+ 2024-07-19 16:33:44,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7593
+ 2024-07-19 16:33:45,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7594
+ 2024-07-19 16:33:45,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7595
+ 2024-07-19 16:33:46,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7596
+ 2024-07-19 16:33:47,771 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7597
+ 2024-07-19 16:33:48,771 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7598
+ 2024-07-19 16:33:49,771 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7599
+ 2024-07-19 16:33:50,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7600
+ 2024-07-19 16:33:50,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7601
+ 2024-07-19 16:33:51,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7602
+ 2024-07-19 16:33:52,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7603
+ 2024-07-19 16:33:53,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7604
+ 2024-07-19 16:33:54,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7605
+ 2024-07-19 16:33:54,795 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7606
+ 2024-07-19 16:33:55,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7607
+ 2024-07-19 16:33:55,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7608
+ 2024-07-19 16:33:55,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7609
+ 2024-07-19 16:33:56,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7610
+ 2024-07-19 16:33:56,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7611
+ 2024-07-19 16:33:57,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7612
+ 2024-07-19 16:33:58,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7613
+ 2024-07-19 16:33:59,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7614
+ 2024-07-19 16:34:00,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7615
+ 2024-07-19 16:34:01,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7616
+ 2024-07-19 16:34:02,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7617
+ 2024-07-19 16:34:02,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7618
+ 2024-07-19 16:34:03,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7619
+ 2024-07-19 16:34:04,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7620
+ 2024-07-19 16:34:05,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7621
+ 2024-07-19 16:34:06,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7622
+ 2024-07-19 16:34:07,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7623
+ 2024-07-19 16:34:07,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7624
+ 2024-07-19 16:34:08,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7625
+ 2024-07-19 16:34:09,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7626
+ 2024-07-19 16:34:10,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7627
+ 2024-07-19 16:34:10,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7628
+ 2024-07-19 16:34:10,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7629
+ 2024-07-19 16:34:11,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7630
+ 2024-07-19 16:34:12,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7631
+ 2024-07-19 16:34:12,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7632
+ 2024-07-19 16:34:13,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7633
+ 2024-07-19 16:34:14,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7634
+ 2024-07-19 16:34:15,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7635
+ 2024-07-19 16:34:16,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7636
+ 2024-07-19 16:34:17,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7637
+ 2024-07-19 16:34:18,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7638
+ 2024-07-19 16:34:18,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7639
+ 2024-07-19 16:34:19,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7640
+ 2024-07-19 16:34:20,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7641
+ 2024-07-19 16:34:21,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7642
+ 2024-07-19 16:34:22,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7643
+ 2024-07-19 16:34:23,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7644
+ 2024-07-19 16:34:23,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7645
+ 2024-07-19 16:34:24,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7646
+ 2024-07-19 16:34:24,795 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7647
+ 2024-07-19 16:34:25,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7648
+ 2024-07-19 16:34:25,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7649
+ 2024-07-19 16:34:25,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7650
+ 2024-07-19 16:34:26,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7651
+ 2024-07-19 16:34:27,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7652
+ 2024-07-19 16:34:28,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7653
+ 2024-07-19 16:34:29,779 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7654
+ 2024-07-19 16:34:29,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7655
+ 2024-07-19 16:34:30,779 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7656
+ 2024-07-19 16:34:31,779 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7657
+ 2024-07-19 16:34:32,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7658
+ 2024-07-19 16:34:33,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7659
+ 2024-07-19 16:34:34,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7660
+ 2024-07-19 16:34:34,990 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7661
+ 2024-07-19 16:34:35,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7662
+ 2024-07-19 16:34:36,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7663
+ 2024-07-19 16:34:37,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7664
+ 2024-07-19 16:34:38,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7665
+ 2024-07-19 16:34:39,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7666
+ 2024-07-19 16:34:40,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7667
+ 2024-07-19 16:34:40,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7668
+ 2024-07-19 16:34:40,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7669
+ 2024-07-19 16:34:40,856 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7670
+ 2024-07-19 16:34:41,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7671
+ 2024-07-19 16:34:42,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7672
+ 2024-07-19 16:34:43,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7673
+ 2024-07-19 16:34:44,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7674
+ 2024-07-19 16:34:45,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7675
+ 2024-07-19 16:34:45,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7676
+ 2024-07-19 16:34:46,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7677
+ 2024-07-19 16:34:47,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7678
+ 2024-07-19 16:34:48,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7679
+ 2024-07-19 16:34:49,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7680
+ 2024-07-19 16:34:50,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7681
+ 2024-07-19 16:34:50,990 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7682
+ 2024-07-19 16:34:51,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7683
+ 2024-07-19 16:34:52,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7684
+ 2024-07-19 16:34:53,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7685
+ 2024-07-19 16:34:54,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7686
+ 2024-07-19 16:34:54,796 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7687
+ 2024-07-19 16:34:55,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7688
+ 2024-07-19 16:34:55,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7689
+ 2024-07-19 16:34:55,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7690
+ 2024-07-19 16:34:56,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7691
+ 2024-07-19 16:34:56,990 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7692
+ 2024-07-19 16:34:57,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7693
+ 2024-07-19 16:34:58,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7694
+ 2024-07-19 16:34:59,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7695
+ 2024-07-19 16:35:00,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7696
+ 2024-07-19 16:35:01,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7697
+ 2024-07-19 16:35:01,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7698
+ 2024-07-19 16:35:02,785 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7699
+ 2024-07-19 16:35:03,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7700
+ 2024-07-19 16:35:04,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7701
+ 2024-07-19 16:35:05,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7702
+ 2024-07-19 16:35:06,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7703
+ 2024-07-19 16:35:06,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7704
+ 2024-07-19 16:35:07,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7705
+ 2024-07-19 16:35:08,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7706
+ 2024-07-19 16:35:09,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7707
+ 2024-07-19 16:35:10,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7708
+ 2024-07-19 16:35:10,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7709
+ 2024-07-19 16:35:10,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7710
+ 2024-07-19 16:35:11,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7711
+ 2024-07-19 16:35:12,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7712
+ 2024-07-19 16:35:12,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7713
+ 2024-07-19 16:35:13,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7714
+ 2024-07-19 16:35:14,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7715
+ 2024-07-19 16:35:15,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7716
+ 2024-07-19 16:35:16,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7717
+ 2024-07-19 16:35:17,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7718
+ 2024-07-19 16:35:17,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7719
+ 2024-07-19 16:35:18,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7720
+ 2024-07-19 16:35:19,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7721
+ 2024-07-19 16:35:20,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7722
+ 2024-07-19 16:35:21,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7723
+ 2024-07-19 16:35:22,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7724
+ 2024-07-19 16:35:23,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7725
+ 2024-07-19 16:35:23,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7726
+ 2024-07-19 16:35:24,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7727
+ 2024-07-19 16:35:24,797 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7728
+ 2024-07-19 16:35:25,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7729
+ 2024-07-19 16:35:25,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7730
+ 2024-07-19 16:35:25,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7731
+ 2024-07-19 16:35:26,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7732
+ 2024-07-19 16:35:27,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7733
+ 2024-07-19 16:35:28,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7734
+ 2024-07-19 16:35:29,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7735
+ 2024-07-19 16:35:29,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7736
+ 2024-07-19 16:35:30,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7737
+ 2024-07-19 16:35:31,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7738
+ 2024-07-19 16:35:32,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7739
+ 2024-07-19 16:35:33,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7740
+ 2024-07-19 16:35:34,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7741
+ 2024-07-19 16:35:34,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7742
+ 2024-07-19 16:35:35,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7743
+ 2024-07-19 16:35:36,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7744
+ 2024-07-19 16:35:37,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7745
+ 2024-07-19 16:35:38,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7746
+ 2024-07-19 16:35:39,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7747
+ 2024-07-19 16:35:39,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7748
+ 2024-07-19 16:35:40,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7749
+ 2024-07-19 16:35:40,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7750
+ 2024-07-19 16:35:40,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7751
+ 2024-07-19 16:35:41,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7752
+ 2024-07-19 16:35:42,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7753
+ 2024-07-19 16:35:43,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7754
+ 2024-07-19 16:35:44,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7755
+ 2024-07-19 16:35:44,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7756
+ 2024-07-19 16:35:45,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7757
+ 2024-07-19 16:35:46,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7758
+ 2024-07-19 16:35:47,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7759
+ 2024-07-19 16:35:48,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7760
+ 2024-07-19 16:35:49,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7761
+ 2024-07-19 16:35:49,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7762
+ 2024-07-19 16:35:50,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7763
+ 2024-07-19 16:35:51,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7764
+ 2024-07-19 16:35:52,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7765
+ 2024-07-19 16:35:53,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7766
+ 2024-07-19 16:35:54,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7767
+ 2024-07-19 16:35:54,798 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7768
+ 2024-07-19 16:35:55,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7769
+ 2024-07-19 16:35:55,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7770
+ 2024-07-19 16:35:55,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7771
+ 2024-07-19 16:35:55,840 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7772
+ 2024-07-19 16:35:56,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7773
+ 2024-07-19 16:35:57,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7774
+ 2024-07-19 16:35:58,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7775
+ 2024-07-19 16:35:59,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7776
+ 2024-07-19 16:36:00,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7777
+ 2024-07-19 16:36:00,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7778
+ 2024-07-19 16:36:01,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7779
+ 2024-07-19 16:36:02,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7780
+ 2024-07-19 16:36:03,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7781
+ 2024-07-19 16:36:04,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7782
+ 2024-07-19 16:36:05,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7783
+ 2024-07-19 16:36:05,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7784
+ 2024-07-19 16:36:06,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7785
+ 2024-07-19 16:36:07,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7786
+ 2024-07-19 16:36:08,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7787
+ 2024-07-19 16:36:09,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7788
+ 2024-07-19 16:36:10,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7789
+ 2024-07-19 16:36:10,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7790
+ 2024-07-19 16:36:10,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7791
+ 2024-07-19 16:36:10,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7792
+ 2024-07-19 16:36:11,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7793
+ 2024-07-19 16:36:12,798 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7794
+ 2024-07-19 16:36:13,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7795
+ 2024-07-19 16:36:14,798 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7796
+ 2024-07-19 16:36:15,798 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7797
+ 2024-07-19 16:36:16,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7798
+ 2024-07-19 16:36:16,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7799
+ 2024-07-19 16:36:17,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7800
+ 2024-07-19 16:36:18,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7801
+ 2024-07-19 16:36:19,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7802
+ 2024-07-19 16:36:20,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7803
+ 2024-07-19 16:36:21,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7804
+ 2024-07-19 16:36:22,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7805
+ 2024-07-19 16:36:22,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7806
+ 2024-07-19 16:36:23,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7807
+ 2024-07-19 16:36:24,798 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7808
+ 2024-07-19 16:36:24,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7809
+ 2024-07-19 16:36:25,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7810
+ 2024-07-19 16:36:25,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7811
+ 2024-07-19 16:36:25,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7812
+ 2024-07-19 16:36:26,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7813
+ 2024-07-19 16:36:27,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7814
+ 2024-07-19 16:36:27,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7815
+ 2024-07-19 16:36:28,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7816
+ 2024-07-19 16:36:29,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7817
+ 2024-07-19 16:36:30,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7818
+ 2024-07-19 16:36:31,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7819
+ 2024-07-19 16:36:32,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7820
+ 2024-07-19 16:36:33,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7821
+ 2024-07-19 16:36:33,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7822
+ 2024-07-19 16:36:34,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7823
+ 2024-07-19 16:36:35,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7824
+ 2024-07-19 16:36:36,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7825
+ 2024-07-19 16:36:37,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7826
+ 2024-07-19 16:36:38,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7827
+ 2024-07-19 16:36:38,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7828
+ 2024-07-19 16:36:39,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7829
+ 2024-07-19 16:36:40,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7830
+ 2024-07-19 16:36:40,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7831
+ 2024-07-19 16:36:40,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7832
+ 2024-07-19 16:36:41,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7833
+ 2024-07-19 16:36:42,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7834
+ 2024-07-19 16:36:43,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7835
+ 2024-07-19 16:36:44,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7836
+ 2024-07-19 16:36:44,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7837
+ 2024-07-19 16:36:45,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7838
+ 2024-07-19 16:36:46,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7839
+ 2024-07-19 16:36:47,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7840
+ 2024-07-19 16:36:48,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7841
+ 2024-07-19 16:36:49,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7842
+ 2024-07-19 16:36:49,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7843
+ 2024-07-19 16:36:50,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7844
+ 2024-07-19 16:36:51,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7845
+ 2024-07-19 16:36:52,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7846
+ 2024-07-19 16:36:53,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7847
+ 2024-07-19 16:36:54,799 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7848
+ 2024-07-19 16:36:54,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7849
+ 2024-07-19 16:36:55,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7850
+ 2024-07-19 16:36:55,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7851
+ 2024-07-19 16:36:55,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7852
+ 2024-07-19 16:36:55,827 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7853
+ 2024-07-19 16:36:56,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7854
+ 2024-07-19 16:36:57,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7855
+ 2024-07-19 16:36:58,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7856
+ 2024-07-19 16:36:59,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7857
+ 2024-07-19 16:37:00,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7858
+ 2024-07-19 16:37:00,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7859
+ 2024-07-19 16:37:01,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7860
+ 2024-07-19 16:37:02,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7861
+ 2024-07-19 16:37:03,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7862
+ 2024-07-19 16:37:04,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7863
+ 2024-07-19 16:37:05,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7864
+ 2024-07-19 16:37:05,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7865
+ 2024-07-19 16:37:06,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7866
+ 2024-07-19 16:37:07,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7867
+ 2024-07-19 16:37:08,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7868
+ 2024-07-19 16:37:09,809 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7869
+ 2024-07-19 16:37:10,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7870
+ 2024-07-19 16:37:10,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7871
+ 2024-07-19 16:37:10,809 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7872
+ 2024-07-19 16:37:10,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7873
+ 2024-07-19 16:37:11,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7874
+ 2024-07-19 16:37:12,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7875
+ 2024-07-19 16:37:13,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7876
+ 2024-07-19 16:37:14,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7877
+ 2024-07-19 16:37:15,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7878
+ 2024-07-19 16:37:15,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7879
+ 2024-07-19 16:37:16,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7880
+ 2024-07-19 16:37:17,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7881
+ 2024-07-19 16:37:18,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7882
+ 2024-07-19 16:37:19,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7883
+ 2024-07-19 16:37:20,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7884
+ 2024-07-19 16:37:20,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7885
+ 2024-07-19 16:37:21,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7886
+ 2024-07-19 16:37:22,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7887
+ 2024-07-19 16:37:23,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7888
+ 2024-07-19 16:37:24,800 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7889
+ 2024-07-19 16:37:24,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7890
+ 2024-07-19 16:37:25,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7891
+ 2024-07-19 16:37:25,640 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7892
+ 2024-07-19 16:37:25,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7893
+ 2024-07-19 16:37:25,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7894
+ 2024-07-19 16:37:26,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7895
+ 2024-07-19 16:37:27,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7896
+ 2024-07-19 16:37:28,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7897
+ 2024-07-19 16:37:29,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7898
+ 2024-07-19 16:37:30,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7899
+ 2024-07-19 16:37:30,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7900
+ 2024-07-19 16:37:31,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7901
+ 2024-07-19 16:37:32,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7902
+ 2024-07-19 16:37:33,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7903
+ 2024-07-19 16:37:34,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7904
+ 2024-07-19 16:37:35,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7905
+ 2024-07-19 16:37:35,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7906
+ 2024-07-19 16:37:36,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7907
+ 2024-07-19 16:37:37,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7908
+ 2024-07-19 16:37:38,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7909
+ 2024-07-19 16:37:39,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7910
+ 2024-07-19 16:37:40,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7911
+ 2024-07-19 16:37:40,640 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7912
+ 2024-07-19 16:37:40,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7913
+ 2024-07-19 16:37:40,997 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7914
+ 2024-07-19 16:37:41,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7915
+ 2024-07-19 16:37:42,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7916
+ 2024-07-19 16:37:43,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7917
+ 2024-07-19 16:37:44,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7918
+ 2024-07-19 16:37:45,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7919
+ 2024-07-19 16:37:45,997 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7920
+ 2024-07-19 16:37:46,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7921
+ 2024-07-19 16:37:47,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7922
+ 2024-07-19 16:37:49,021 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7923
+ 2024-07-19 16:37:50,016 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7924
+ 2024-07-19 16:37:50,974 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7925
+ 2024-07-19 16:37:50,998 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7926
+ 2024-07-19 16:37:51,949 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7927
+ 2024-07-19 16:37:52,938 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7928
+ 2024-07-19 16:37:53,944 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7929
+ 2024-07-19 16:37:54,801 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7930
+ 2024-07-19 16:37:54,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7931
+ 2024-07-19 16:37:55,753 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7932
+ 2024-07-19 16:37:55,754 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7933
+ 2024-07-19 16:37:55,969 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7934
+ 2024-07-19 16:37:56,184 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7935
+ 2024-07-19 16:37:57,005 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7936
+ 2024-07-19 16:37:58,046 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7937
+ 2024-07-19 16:37:59,015 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7938
+ 2024-07-19 16:38:00,057 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7939
+ 2024-07-19 16:38:01,085 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7940
+ 2024-07-19 16:38:02,048 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7941
+ 2024-07-19 16:38:02,065 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7942
+ 2024-07-19 16:38:03,012 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7943
+ 2024-07-19 16:38:04,147 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7944
+ 2024-07-19 16:38:05,100 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7945
+ 2024-07-19 16:38:05,931 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7946
+ 2024-07-19 16:38:06,254 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: partial_history
7947
+ 2024-07-19 16:38:06,256 DEBUG SenderThread:148 [sender.py:send():379] send: history
7948
+ 2024-07-19 16:38:06,256 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7949
+ 2024-07-19 16:38:06,256 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7950
+ 2024-07-19 16:38:06,296 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: partial_history
7951
+ 2024-07-19 16:38:06,297 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: pause
7952
+ 2024-07-19 16:38:06,297 INFO HandlerThread:148 [handler.py:handle_request_pause():724] stopping system metrics thread
7953
+ 2024-07-19 16:38:06,297 INFO HandlerThread:148 [system_monitor.py:finish():203] Stopping system monitor
7954
+ 2024-07-19 16:38:06,298 INFO HandlerThread:148 [interfaces.py:finish():200] Joined cpu monitor
7955
+ 2024-07-19 16:38:06,299 DEBUG SenderThread:148 [sender.py:send():379] send: telemetry
7956
+ 2024-07-19 16:38:06,299 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7957
+ 2024-07-19 16:38:06,299 INFO HandlerThread:148 [interfaces.py:finish():200] Joined disk monitor
7958
+ 2024-07-19 16:38:06,300 DEBUG SystemMonitor:148 [system_monitor.py:_start():179] Finished system metrics aggregation loop
7959
+ 2024-07-19 16:38:06,300 DEBUG SystemMonitor:148 [system_monitor.py:_start():183] Publishing last batch of metrics
7960
+ 2024-07-19 16:38:06,301 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7961
+ 2024-07-19 16:38:06,301 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7962
+ 2024-07-19 16:38:06,302 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7963
+ 2024-07-19 16:38:06,302 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7964
+ 2024-07-19 16:38:06,303 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7965
+ 2024-07-19 16:38:06,303 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7966
+ 2024-07-19 16:38:06,303 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7967
+ 2024-07-19 16:38:06,304 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7968
+ 2024-07-19 16:38:06,304 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7969
+ 2024-07-19 16:38:06,304 DEBUG SenderThread:148 [sender.py:send():379] send: history
7970
+ 2024-07-19 16:38:06,305 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7971
+ 2024-07-19 16:38:06,305 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7972
+ 2024-07-19 16:38:06,312 INFO HandlerThread:148 [interfaces.py:finish():200] Joined gpu monitor
7973
+ 2024-07-19 16:38:06,312 INFO HandlerThread:148 [interfaces.py:finish():200] Joined memory monitor
7974
+ 2024-07-19 16:38:06,313 INFO HandlerThread:148 [interfaces.py:finish():200] Joined network monitor
7975
+ 2024-07-19 16:38:06,313 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: resume
7976
+ 2024-07-19 16:38:06,313 INFO HandlerThread:148 [handler.py:handle_request_resume():715] starting system metrics thread
7977
+ 2024-07-19 16:38:06,313 INFO HandlerThread:148 [system_monitor.py:start():194] Starting system monitor
7978
+ 2024-07-19 16:38:06,313 INFO SystemMonitor:148 [system_monitor.py:_start():158] Starting system asset monitoring threads
7979
+ 2024-07-19 16:38:06,314 INFO SystemMonitor:148 [interfaces.py:start():188] Started cpu monitoring
7980
+ 2024-07-19 16:38:06,315 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7981
+ 2024-07-19 16:38:06,315 INFO SystemMonitor:148 [interfaces.py:start():188] Started disk monitoring
7982
+ 2024-07-19 16:38:06,316 INFO SystemMonitor:148 [interfaces.py:start():188] Started gpu monitoring
7983
+ 2024-07-19 16:38:06,316 INFO SystemMonitor:148 [interfaces.py:start():188] Started memory monitoring
7984
+ 2024-07-19 16:38:06,317 INFO SystemMonitor:148 [interfaces.py:start():188] Started network monitoring
7985
+ 2024-07-19 16:38:06,626 INFO Thread-12 :148 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_150308-wq61zns9/files/wandb-summary.json
7986
+ 2024-07-19 16:38:07,117 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7987
+ 2024-07-19 16:38:07,320 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7988
+ 2024-07-19 16:38:07,626 INFO Thread-12 :148 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_150308-wq61zns9/files/config.yaml
7989
+ 2024-07-19 16:38:07,627 INFO Thread-12 :148 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_150308-wq61zns9/files/output.log
7990
+ 2024-07-19 16:38:08,117 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7991
+ 2024-07-19 16:38:09,117 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
wandb/debug.log CHANGED
@@ -49,3 +49,6 @@ config: {}
49
  2024-07-19 15:13:25,433 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb None None {'vocab_size': 62802, 'decoder_vocab_size': 62802, 'max_position_embeddings': 512, 'd_model': 512, 'encoder_ffn_dim': 2048, 'encoder_layers': 6, 'encoder_attention_heads': 8, 'decoder_ffn_dim': 2048, 'decoder_layers': 6, 'decoder_attention_heads': 8, 'dropout': 0.1, 'attention_dropout': 0.0, 'activation_dropout': 0.0, 'activation_function': 'swish', 'init_std': 0.02, 'encoder_layerdrop': 0.0, 'decoder_layerdrop': 0.0, 'use_cache': True, 'num_hidden_layers': 6, 'scale_embedding': True, 'share_encoder_decoder_embeddings': True, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': None, 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': True, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 512, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 4, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': [[62801]], 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': 0, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MarianMTModel'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1', 2: 'LABEL_2'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1, 'LABEL_2': 2}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 0, 'pad_token_id': 62801, 'eos_token_id': 0, 'sep_token_id': None, 'decoder_start_token_id': 62801, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'Helsinki-NLP/opus-mt-en-ar', 'transformers_version': '4.42.3', 'add_bias_logits': False, 'add_final_layer_norm': False, 'classif_dropout': 0.0, 'classifier_dropout': 0.0, 'extra_pos_embeddings': 62802, 'model_type': 'marian', 'normalize_before': False, 'normalize_embedding': False, 'static_position_embeddings': True, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 16, 'per_device_eval_batch_size': 16, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 1, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 2e-05, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jul19_15-13-23_8df4908137f9', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 500, 'save_total_limit': 3, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'adamw_torch', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False, 'eval_on_start': False, 'sortish_sampler': False, 'predict_with_generate': True, 'generation_max_length': None, 'generation_num_beams': None, 'generation_config': None}
50
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_config.py:__setitem__():151] config set model/num_parameters = 76817408 - <bound method Run._config_callback of <wandb.sdk.wandb_run.Run object at 0x7ac41ae98400>>
51
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb model/num_parameters 76817408 None
 
 
 
 
49
  2024-07-19 15:13:25,433 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb None None {'vocab_size': 62802, 'decoder_vocab_size': 62802, 'max_position_embeddings': 512, 'd_model': 512, 'encoder_ffn_dim': 2048, 'encoder_layers': 6, 'encoder_attention_heads': 8, 'decoder_ffn_dim': 2048, 'decoder_layers': 6, 'decoder_attention_heads': 8, 'dropout': 0.1, 'attention_dropout': 0.0, 'activation_dropout': 0.0, 'activation_function': 'swish', 'init_std': 0.02, 'encoder_layerdrop': 0.0, 'decoder_layerdrop': 0.0, 'use_cache': True, 'num_hidden_layers': 6, 'scale_embedding': True, 'share_encoder_decoder_embeddings': True, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': None, 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': True, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 512, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 4, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': [[62801]], 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': 0, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MarianMTModel'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1', 2: 'LABEL_2'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1, 'LABEL_2': 2}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 0, 'pad_token_id': 62801, 'eos_token_id': 0, 'sep_token_id': None, 'decoder_start_token_id': 62801, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'Helsinki-NLP/opus-mt-en-ar', 'transformers_version': '4.42.3', 'add_bias_logits': False, 'add_final_layer_norm': False, 'classif_dropout': 0.0, 'classifier_dropout': 0.0, 'extra_pos_embeddings': 62802, 'model_type': 'marian', 'normalize_before': False, 'normalize_embedding': False, 'static_position_embeddings': True, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 16, 'per_device_eval_batch_size': 16, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 1, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 2e-05, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jul19_15-13-23_8df4908137f9', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 500, 'save_total_limit': 3, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'adamw_torch', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False, 'eval_on_start': False, 'sortish_sampler': False, 'predict_with_generate': True, 'generation_max_length': None, 'generation_num_beams': None, 'generation_config': None}
50
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_config.py:__setitem__():151] config set model/num_parameters = 76817408 - <bound method Run._config_callback of <wandb.sdk.wandb_run.Run object at 0x7ac41ae98400>>
51
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb model/num_parameters 76817408 None
52
+ 2024-07-19 16:38:06,262 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
53
+ 2024-07-19 16:38:06,262 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
54
+ 2024-07-19 16:38:06,267 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
wandb/run-20240719_150308-wq61zns9/files/config.yaml CHANGED
@@ -46,6 +46,7 @@ _wandb:
46
  - 13
47
  - 19
48
  - 23
 
49
  - 66
50
  4: 3.10.13
51
  5: 0.17.4
 
46
  - 13
47
  - 19
48
  - 23
49
+ - 62
50
  - 66
51
  4: 3.10.13
52
  5: 0.17.4
wandb/run-20240719_150308-wq61zns9/files/output.log CHANGED
@@ -11,3 +11,5 @@ Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_word
11
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
12
  Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_words_ids': [[62801]], 'forced_eos_token_id': 0}
13
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
 
 
 
11
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
12
  Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_words_ids': [[62801]], 'forced_eos_token_id': 0}
13
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
14
+ Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_words_ids': [[62801]], 'forced_eos_token_id': 0}
15
+ Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
wandb/run-20240719_150308-wq61zns9/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"eval/loss": 0.11444152891635895, "eval/bleu": 21.8952, "eval/gen_len": 67.3031, "eval/runtime": 280.9076, "eval/samples_per_second": 4.439, "eval/steps_per_second": 0.278, "train/epoch": 9.615384615384615, "train/global_step": 3000, "_timestamp": 1721406708.9238005, "_runtime": 5320.384981393814, "_step": 14, "train/loss": 0.0814, "train/grad_norm": 0.2923108637332916, "train/learning_rate": 7.692307692307694e-07}
 
1
+ {"eval/loss": 0.11429202556610107, "eval/bleu": 21.836, "eval/gen_len": 67.4314, "eval/runtime": 286.1152, "eval/samples_per_second": 4.358, "eval/steps_per_second": 0.273, "train/epoch": 10.0, "train/global_step": 3120, "_timestamp": 1721407086.2585027, "_runtime": 5697.719683647156, "_step": 16, "train/loss": 0.0814, "train/grad_norm": 0.2923108637332916, "train/learning_rate": 7.692307692307694e-07, "train_runtime": 5080.8423, "train_samples_per_second": 9.817, "train_steps_per_second": 0.614, "total_flos": 4011811107766272.0, "train_loss": 0.10427434016496707}
wandb/run-20240719_150308-wq61zns9/logs/debug-internal.log CHANGED
@@ -7578,3 +7578,414 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
7578
  2024-07-19 16:33:34,764 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7579
  2024-07-19 16:33:34,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7580
  2024-07-19 16:33:35,768 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7578
  2024-07-19 16:33:34,764 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7579
  2024-07-19 16:33:34,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7580
  2024-07-19 16:33:35,768 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7581
+ 2024-07-19 16:33:36,768 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7582
+ 2024-07-19 16:33:37,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7583
+ 2024-07-19 16:33:38,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7584
+ 2024-07-19 16:33:39,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7585
+ 2024-07-19 16:33:39,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7586
+ 2024-07-19 16:33:40,636 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7587
+ 2024-07-19 16:33:40,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7588
+ 2024-07-19 16:33:40,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7589
+ 2024-07-19 16:33:41,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7590
+ 2024-07-19 16:33:42,769 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7591
+ 2024-07-19 16:33:43,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7592
+ 2024-07-19 16:33:44,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7593
+ 2024-07-19 16:33:45,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7594
+ 2024-07-19 16:33:45,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7595
+ 2024-07-19 16:33:46,770 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7596
+ 2024-07-19 16:33:47,771 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7597
+ 2024-07-19 16:33:48,771 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7598
+ 2024-07-19 16:33:49,771 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7599
+ 2024-07-19 16:33:50,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7600
+ 2024-07-19 16:33:50,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7601
+ 2024-07-19 16:33:51,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7602
+ 2024-07-19 16:33:52,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7603
+ 2024-07-19 16:33:53,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7604
+ 2024-07-19 16:33:54,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7605
+ 2024-07-19 16:33:54,795 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7606
+ 2024-07-19 16:33:55,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7607
+ 2024-07-19 16:33:55,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7608
+ 2024-07-19 16:33:55,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7609
+ 2024-07-19 16:33:56,772 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7610
+ 2024-07-19 16:33:56,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7611
+ 2024-07-19 16:33:57,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7612
+ 2024-07-19 16:33:58,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7613
+ 2024-07-19 16:33:59,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7614
+ 2024-07-19 16:34:00,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7615
+ 2024-07-19 16:34:01,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7616
+ 2024-07-19 16:34:02,773 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7617
+ 2024-07-19 16:34:02,987 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7618
+ 2024-07-19 16:34:03,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7619
+ 2024-07-19 16:34:04,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7620
+ 2024-07-19 16:34:05,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7621
+ 2024-07-19 16:34:06,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7622
+ 2024-07-19 16:34:07,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7623
+ 2024-07-19 16:34:07,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7624
+ 2024-07-19 16:34:08,774 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7625
+ 2024-07-19 16:34:09,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7626
+ 2024-07-19 16:34:10,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7627
+ 2024-07-19 16:34:10,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7628
+ 2024-07-19 16:34:10,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7629
+ 2024-07-19 16:34:11,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7630
+ 2024-07-19 16:34:12,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7631
+ 2024-07-19 16:34:12,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7632
+ 2024-07-19 16:34:13,775 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7633
+ 2024-07-19 16:34:14,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7634
+ 2024-07-19 16:34:15,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7635
+ 2024-07-19 16:34:16,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7636
+ 2024-07-19 16:34:17,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7637
+ 2024-07-19 16:34:18,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7638
+ 2024-07-19 16:34:18,988 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7639
+ 2024-07-19 16:34:19,776 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7640
+ 2024-07-19 16:34:20,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7641
+ 2024-07-19 16:34:21,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7642
+ 2024-07-19 16:34:22,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7643
+ 2024-07-19 16:34:23,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7644
+ 2024-07-19 16:34:23,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7645
+ 2024-07-19 16:34:24,777 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7646
+ 2024-07-19 16:34:24,795 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7647
+ 2024-07-19 16:34:25,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7648
+ 2024-07-19 16:34:25,637 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7649
+ 2024-07-19 16:34:25,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7650
+ 2024-07-19 16:34:26,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7651
+ 2024-07-19 16:34:27,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7652
+ 2024-07-19 16:34:28,778 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7653
+ 2024-07-19 16:34:29,779 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7654
+ 2024-07-19 16:34:29,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7655
+ 2024-07-19 16:34:30,779 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7656
+ 2024-07-19 16:34:31,779 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7657
+ 2024-07-19 16:34:32,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7658
+ 2024-07-19 16:34:33,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7659
+ 2024-07-19 16:34:34,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7660
+ 2024-07-19 16:34:34,990 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7661
+ 2024-07-19 16:34:35,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7662
+ 2024-07-19 16:34:36,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7663
+ 2024-07-19 16:34:37,780 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7664
+ 2024-07-19 16:34:38,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7665
+ 2024-07-19 16:34:39,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7666
+ 2024-07-19 16:34:40,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7667
+ 2024-07-19 16:34:40,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7668
+ 2024-07-19 16:34:40,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7669
+ 2024-07-19 16:34:40,856 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7670
+ 2024-07-19 16:34:41,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7671
+ 2024-07-19 16:34:42,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7672
+ 2024-07-19 16:34:43,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7673
+ 2024-07-19 16:34:44,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7674
+ 2024-07-19 16:34:45,781 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7675
+ 2024-07-19 16:34:45,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7676
+ 2024-07-19 16:34:46,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7677
+ 2024-07-19 16:34:47,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7678
+ 2024-07-19 16:34:48,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7679
+ 2024-07-19 16:34:49,782 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7680
+ 2024-07-19 16:34:50,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7681
+ 2024-07-19 16:34:50,990 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7682
+ 2024-07-19 16:34:51,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7683
+ 2024-07-19 16:34:52,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7684
+ 2024-07-19 16:34:53,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7685
+ 2024-07-19 16:34:54,783 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7686
+ 2024-07-19 16:34:54,796 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7687
+ 2024-07-19 16:34:55,637 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7688
+ 2024-07-19 16:34:55,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7689
+ 2024-07-19 16:34:55,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7690
+ 2024-07-19 16:34:56,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7691
+ 2024-07-19 16:34:56,990 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7692
+ 2024-07-19 16:34:57,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7693
+ 2024-07-19 16:34:58,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7694
+ 2024-07-19 16:34:59,784 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7695
+ 2024-07-19 16:35:00,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7696
+ 2024-07-19 16:35:01,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7697
+ 2024-07-19 16:35:01,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7698
+ 2024-07-19 16:35:02,785 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7699
+ 2024-07-19 16:35:03,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7700
+ 2024-07-19 16:35:04,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7701
+ 2024-07-19 16:35:05,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7702
+ 2024-07-19 16:35:06,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7703
+ 2024-07-19 16:35:06,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7704
+ 2024-07-19 16:35:07,786 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7705
+ 2024-07-19 16:35:08,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7706
+ 2024-07-19 16:35:09,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7707
+ 2024-07-19 16:35:10,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7708
+ 2024-07-19 16:35:10,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7709
+ 2024-07-19 16:35:10,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7710
+ 2024-07-19 16:35:11,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7711
+ 2024-07-19 16:35:12,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7712
+ 2024-07-19 16:35:12,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7713
+ 2024-07-19 16:35:13,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7714
+ 2024-07-19 16:35:14,787 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7715
+ 2024-07-19 16:35:15,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7716
+ 2024-07-19 16:35:16,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7717
+ 2024-07-19 16:35:17,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7718
+ 2024-07-19 16:35:17,991 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7719
+ 2024-07-19 16:35:18,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7720
+ 2024-07-19 16:35:19,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7721
+ 2024-07-19 16:35:20,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7722
+ 2024-07-19 16:35:21,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7723
+ 2024-07-19 16:35:22,788 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7724
+ 2024-07-19 16:35:23,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7725
+ 2024-07-19 16:35:23,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7726
+ 2024-07-19 16:35:24,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7727
+ 2024-07-19 16:35:24,797 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7728
+ 2024-07-19 16:35:25,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7729
+ 2024-07-19 16:35:25,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7730
+ 2024-07-19 16:35:25,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7731
+ 2024-07-19 16:35:26,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7732
+ 2024-07-19 16:35:27,789 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7733
+ 2024-07-19 16:35:28,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7734
+ 2024-07-19 16:35:29,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7735
+ 2024-07-19 16:35:29,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7736
+ 2024-07-19 16:35:30,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7737
+ 2024-07-19 16:35:31,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7738
+ 2024-07-19 16:35:32,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7739
+ 2024-07-19 16:35:33,790 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7740
+ 2024-07-19 16:35:34,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7741
+ 2024-07-19 16:35:34,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7742
+ 2024-07-19 16:35:35,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7743
+ 2024-07-19 16:35:36,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7744
+ 2024-07-19 16:35:37,791 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7745
+ 2024-07-19 16:35:38,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7746
+ 2024-07-19 16:35:39,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7747
+ 2024-07-19 16:35:39,992 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7748
+ 2024-07-19 16:35:40,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7749
+ 2024-07-19 16:35:40,638 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7750
+ 2024-07-19 16:35:40,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7751
+ 2024-07-19 16:35:41,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7752
+ 2024-07-19 16:35:42,792 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7753
+ 2024-07-19 16:35:43,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7754
+ 2024-07-19 16:35:44,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7755
+ 2024-07-19 16:35:44,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7756
+ 2024-07-19 16:35:45,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7757
+ 2024-07-19 16:35:46,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7758
+ 2024-07-19 16:35:47,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7759
+ 2024-07-19 16:35:48,793 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7760
+ 2024-07-19 16:35:49,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7761
+ 2024-07-19 16:35:49,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7762
+ 2024-07-19 16:35:50,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7763
+ 2024-07-19 16:35:51,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7764
+ 2024-07-19 16:35:52,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7765
+ 2024-07-19 16:35:53,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7766
+ 2024-07-19 16:35:54,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7767
+ 2024-07-19 16:35:54,798 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7768
+ 2024-07-19 16:35:55,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7769
+ 2024-07-19 16:35:55,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7770
+ 2024-07-19 16:35:55,794 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7771
+ 2024-07-19 16:35:55,840 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7772
+ 2024-07-19 16:35:56,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7773
+ 2024-07-19 16:35:57,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7774
+ 2024-07-19 16:35:58,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7775
+ 2024-07-19 16:35:59,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7776
+ 2024-07-19 16:36:00,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7777
+ 2024-07-19 16:36:00,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7778
+ 2024-07-19 16:36:01,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7779
+ 2024-07-19 16:36:02,795 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7780
+ 2024-07-19 16:36:03,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7781
+ 2024-07-19 16:36:04,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7782
+ 2024-07-19 16:36:05,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7783
+ 2024-07-19 16:36:05,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7784
+ 2024-07-19 16:36:06,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7785
+ 2024-07-19 16:36:07,796 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7786
+ 2024-07-19 16:36:08,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7787
+ 2024-07-19 16:36:09,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7788
+ 2024-07-19 16:36:10,638 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7789
+ 2024-07-19 16:36:10,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7790
+ 2024-07-19 16:36:10,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7791
+ 2024-07-19 16:36:10,993 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7792
+ 2024-07-19 16:36:11,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7793
+ 2024-07-19 16:36:12,798 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7794
+ 2024-07-19 16:36:13,797 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7795
+ 2024-07-19 16:36:14,798 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7796
+ 2024-07-19 16:36:15,798 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7797
+ 2024-07-19 16:36:16,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7798
+ 2024-07-19 16:36:16,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7799
+ 2024-07-19 16:36:17,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7800
+ 2024-07-19 16:36:18,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7801
+ 2024-07-19 16:36:19,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7802
+ 2024-07-19 16:36:20,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7803
+ 2024-07-19 16:36:21,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7804
+ 2024-07-19 16:36:22,799 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7805
+ 2024-07-19 16:36:22,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7806
+ 2024-07-19 16:36:23,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7807
+ 2024-07-19 16:36:24,798 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7808
+ 2024-07-19 16:36:24,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7809
+ 2024-07-19 16:36:25,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7810
+ 2024-07-19 16:36:25,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7811
+ 2024-07-19 16:36:25,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7812
+ 2024-07-19 16:36:26,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7813
+ 2024-07-19 16:36:27,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7814
+ 2024-07-19 16:36:27,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7815
+ 2024-07-19 16:36:28,800 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7816
+ 2024-07-19 16:36:29,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7817
+ 2024-07-19 16:36:30,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7818
+ 2024-07-19 16:36:31,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7819
+ 2024-07-19 16:36:32,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7820
+ 2024-07-19 16:36:33,801 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7821
+ 2024-07-19 16:36:33,994 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7822
+ 2024-07-19 16:36:34,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7823
+ 2024-07-19 16:36:35,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7824
+ 2024-07-19 16:36:36,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7825
+ 2024-07-19 16:36:37,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7826
+ 2024-07-19 16:36:38,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7827
+ 2024-07-19 16:36:38,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7828
+ 2024-07-19 16:36:39,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7829
+ 2024-07-19 16:36:40,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7830
+ 2024-07-19 16:36:40,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7831
+ 2024-07-19 16:36:40,802 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7832
+ 2024-07-19 16:36:41,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7833
+ 2024-07-19 16:36:42,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7834
+ 2024-07-19 16:36:43,803 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7835
+ 2024-07-19 16:36:44,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7836
+ 2024-07-19 16:36:44,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7837
+ 2024-07-19 16:36:45,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7838
+ 2024-07-19 16:36:46,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7839
+ 2024-07-19 16:36:47,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7840
+ 2024-07-19 16:36:48,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7841
+ 2024-07-19 16:36:49,804 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7842
+ 2024-07-19 16:36:49,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7843
+ 2024-07-19 16:36:50,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7844
+ 2024-07-19 16:36:51,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7845
+ 2024-07-19 16:36:52,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7846
+ 2024-07-19 16:36:53,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7847
+ 2024-07-19 16:36:54,799 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7848
+ 2024-07-19 16:36:54,805 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7849
+ 2024-07-19 16:36:55,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7850
+ 2024-07-19 16:36:55,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7851
+ 2024-07-19 16:36:55,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7852
+ 2024-07-19 16:36:55,827 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7853
+ 2024-07-19 16:36:56,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7854
+ 2024-07-19 16:36:57,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7855
+ 2024-07-19 16:36:58,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7856
+ 2024-07-19 16:36:59,806 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7857
+ 2024-07-19 16:37:00,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7858
+ 2024-07-19 16:37:00,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7859
+ 2024-07-19 16:37:01,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7860
+ 2024-07-19 16:37:02,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7861
+ 2024-07-19 16:37:03,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7862
+ 2024-07-19 16:37:04,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7863
+ 2024-07-19 16:37:05,807 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7864
+ 2024-07-19 16:37:05,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7865
+ 2024-07-19 16:37:06,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7866
+ 2024-07-19 16:37:07,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7867
+ 2024-07-19 16:37:08,808 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7868
+ 2024-07-19 16:37:09,809 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7869
+ 2024-07-19 16:37:10,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7870
+ 2024-07-19 16:37:10,639 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7871
+ 2024-07-19 16:37:10,809 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7872
+ 2024-07-19 16:37:10,995 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7873
+ 2024-07-19 16:37:11,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7874
+ 2024-07-19 16:37:12,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7875
+ 2024-07-19 16:37:13,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7876
+ 2024-07-19 16:37:14,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7877
+ 2024-07-19 16:37:15,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7878
+ 2024-07-19 16:37:15,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7879
+ 2024-07-19 16:37:16,810 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7880
+ 2024-07-19 16:37:17,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7881
+ 2024-07-19 16:37:18,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7882
+ 2024-07-19 16:37:19,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7883
+ 2024-07-19 16:37:20,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7884
+ 2024-07-19 16:37:20,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7885
+ 2024-07-19 16:37:21,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7886
+ 2024-07-19 16:37:22,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7887
+ 2024-07-19 16:37:23,811 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7888
+ 2024-07-19 16:37:24,800 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7889
+ 2024-07-19 16:37:24,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7890
+ 2024-07-19 16:37:25,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7891
+ 2024-07-19 16:37:25,640 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7892
+ 2024-07-19 16:37:25,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7893
+ 2024-07-19 16:37:25,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7894
+ 2024-07-19 16:37:26,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7895
+ 2024-07-19 16:37:27,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7896
+ 2024-07-19 16:37:28,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7897
+ 2024-07-19 16:37:29,816 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7898
+ 2024-07-19 16:37:30,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7899
+ 2024-07-19 16:37:30,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7900
+ 2024-07-19 16:37:31,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7901
+ 2024-07-19 16:37:32,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7902
+ 2024-07-19 16:37:33,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7903
+ 2024-07-19 16:37:34,817 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7904
+ 2024-07-19 16:37:35,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7905
+ 2024-07-19 16:37:35,996 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7906
+ 2024-07-19 16:37:36,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7907
+ 2024-07-19 16:37:37,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7908
+ 2024-07-19 16:37:38,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7909
+ 2024-07-19 16:37:39,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7910
+ 2024-07-19 16:37:40,639 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7911
+ 2024-07-19 16:37:40,640 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7912
+ 2024-07-19 16:37:40,818 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7913
+ 2024-07-19 16:37:40,997 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7914
+ 2024-07-19 16:37:41,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7915
+ 2024-07-19 16:37:42,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7916
+ 2024-07-19 16:37:43,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7917
+ 2024-07-19 16:37:44,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7918
+ 2024-07-19 16:37:45,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7919
+ 2024-07-19 16:37:45,997 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7920
+ 2024-07-19 16:37:46,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7921
+ 2024-07-19 16:37:47,819 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7922
+ 2024-07-19 16:37:49,021 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7923
+ 2024-07-19 16:37:50,016 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7924
+ 2024-07-19 16:37:50,974 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7925
+ 2024-07-19 16:37:50,998 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7926
+ 2024-07-19 16:37:51,949 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7927
+ 2024-07-19 16:37:52,938 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7928
+ 2024-07-19 16:37:53,944 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7929
+ 2024-07-19 16:37:54,801 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7930
+ 2024-07-19 16:37:54,989 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7931
+ 2024-07-19 16:37:55,753 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: stop_status
7932
+ 2024-07-19 16:37:55,754 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: stop_status
7933
+ 2024-07-19 16:37:55,969 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7934
+ 2024-07-19 16:37:56,184 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7935
+ 2024-07-19 16:37:57,005 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7936
+ 2024-07-19 16:37:58,046 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7937
+ 2024-07-19 16:37:59,015 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7938
+ 2024-07-19 16:38:00,057 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7939
+ 2024-07-19 16:38:01,085 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7940
+ 2024-07-19 16:38:02,048 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7941
+ 2024-07-19 16:38:02,065 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7942
+ 2024-07-19 16:38:03,012 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7943
+ 2024-07-19 16:38:04,147 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7944
+ 2024-07-19 16:38:05,100 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7945
+ 2024-07-19 16:38:05,931 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7946
+ 2024-07-19 16:38:06,254 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: partial_history
7947
+ 2024-07-19 16:38:06,256 DEBUG SenderThread:148 [sender.py:send():379] send: history
7948
+ 2024-07-19 16:38:06,256 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7949
+ 2024-07-19 16:38:06,256 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7950
+ 2024-07-19 16:38:06,296 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: partial_history
7951
+ 2024-07-19 16:38:06,297 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: pause
7952
+ 2024-07-19 16:38:06,297 INFO HandlerThread:148 [handler.py:handle_request_pause():724] stopping system metrics thread
7953
+ 2024-07-19 16:38:06,297 INFO HandlerThread:148 [system_monitor.py:finish():203] Stopping system monitor
7954
+ 2024-07-19 16:38:06,298 INFO HandlerThread:148 [interfaces.py:finish():200] Joined cpu monitor
7955
+ 2024-07-19 16:38:06,299 DEBUG SenderThread:148 [sender.py:send():379] send: telemetry
7956
+ 2024-07-19 16:38:06,299 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7957
+ 2024-07-19 16:38:06,299 INFO HandlerThread:148 [interfaces.py:finish():200] Joined disk monitor
7958
+ 2024-07-19 16:38:06,300 DEBUG SystemMonitor:148 [system_monitor.py:_start():179] Finished system metrics aggregation loop
7959
+ 2024-07-19 16:38:06,300 DEBUG SystemMonitor:148 [system_monitor.py:_start():183] Publishing last batch of metrics
7960
+ 2024-07-19 16:38:06,301 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7961
+ 2024-07-19 16:38:06,301 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7962
+ 2024-07-19 16:38:06,302 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7963
+ 2024-07-19 16:38:06,302 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7964
+ 2024-07-19 16:38:06,303 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7965
+ 2024-07-19 16:38:06,303 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7966
+ 2024-07-19 16:38:06,303 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7967
+ 2024-07-19 16:38:06,304 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7968
+ 2024-07-19 16:38:06,304 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7969
+ 2024-07-19 16:38:06,304 DEBUG SenderThread:148 [sender.py:send():379] send: history
7970
+ 2024-07-19 16:38:06,305 DEBUG SenderThread:148 [sender.py:send_request():406] send_request: summary_record
7971
+ 2024-07-19 16:38:06,305 INFO SenderThread:148 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end
7972
+ 2024-07-19 16:38:06,312 INFO HandlerThread:148 [interfaces.py:finish():200] Joined gpu monitor
7973
+ 2024-07-19 16:38:06,312 INFO HandlerThread:148 [interfaces.py:finish():200] Joined memory monitor
7974
+ 2024-07-19 16:38:06,313 INFO HandlerThread:148 [interfaces.py:finish():200] Joined network monitor
7975
+ 2024-07-19 16:38:06,313 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: resume
7976
+ 2024-07-19 16:38:06,313 INFO HandlerThread:148 [handler.py:handle_request_resume():715] starting system metrics thread
7977
+ 2024-07-19 16:38:06,313 INFO HandlerThread:148 [system_monitor.py:start():194] Starting system monitor
7978
+ 2024-07-19 16:38:06,313 INFO SystemMonitor:148 [system_monitor.py:_start():158] Starting system asset monitoring threads
7979
+ 2024-07-19 16:38:06,314 INFO SystemMonitor:148 [interfaces.py:start():188] Started cpu monitoring
7980
+ 2024-07-19 16:38:06,315 DEBUG SenderThread:148 [sender.py:send():379] send: stats
7981
+ 2024-07-19 16:38:06,315 INFO SystemMonitor:148 [interfaces.py:start():188] Started disk monitoring
7982
+ 2024-07-19 16:38:06,316 INFO SystemMonitor:148 [interfaces.py:start():188] Started gpu monitoring
7983
+ 2024-07-19 16:38:06,316 INFO SystemMonitor:148 [interfaces.py:start():188] Started memory monitoring
7984
+ 2024-07-19 16:38:06,317 INFO SystemMonitor:148 [interfaces.py:start():188] Started network monitoring
7985
+ 2024-07-19 16:38:06,626 INFO Thread-12 :148 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_150308-wq61zns9/files/wandb-summary.json
7986
+ 2024-07-19 16:38:07,117 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7987
+ 2024-07-19 16:38:07,320 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: status_report
7988
+ 2024-07-19 16:38:07,626 INFO Thread-12 :148 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_150308-wq61zns9/files/config.yaml
7989
+ 2024-07-19 16:38:07,627 INFO Thread-12 :148 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_150308-wq61zns9/files/output.log
7990
+ 2024-07-19 16:38:08,117 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
7991
+ 2024-07-19 16:38:09,117 DEBUG HandlerThread:148 [handler.py:handle_request():158] handle_request: internal_messages
wandb/run-20240719_150308-wq61zns9/logs/debug.log CHANGED
@@ -49,3 +49,6 @@ config: {}
49
  2024-07-19 15:13:25,433 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb None None {'vocab_size': 62802, 'decoder_vocab_size': 62802, 'max_position_embeddings': 512, 'd_model': 512, 'encoder_ffn_dim': 2048, 'encoder_layers': 6, 'encoder_attention_heads': 8, 'decoder_ffn_dim': 2048, 'decoder_layers': 6, 'decoder_attention_heads': 8, 'dropout': 0.1, 'attention_dropout': 0.0, 'activation_dropout': 0.0, 'activation_function': 'swish', 'init_std': 0.02, 'encoder_layerdrop': 0.0, 'decoder_layerdrop': 0.0, 'use_cache': True, 'num_hidden_layers': 6, 'scale_embedding': True, 'share_encoder_decoder_embeddings': True, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': None, 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': True, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 512, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 4, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': [[62801]], 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': 0, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MarianMTModel'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1', 2: 'LABEL_2'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1, 'LABEL_2': 2}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 0, 'pad_token_id': 62801, 'eos_token_id': 0, 'sep_token_id': None, 'decoder_start_token_id': 62801, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'Helsinki-NLP/opus-mt-en-ar', 'transformers_version': '4.42.3', 'add_bias_logits': False, 'add_final_layer_norm': False, 'classif_dropout': 0.0, 'classifier_dropout': 0.0, 'extra_pos_embeddings': 62802, 'model_type': 'marian', 'normalize_before': False, 'normalize_embedding': False, 'static_position_embeddings': True, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 16, 'per_device_eval_batch_size': 16, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 1, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 2e-05, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jul19_15-13-23_8df4908137f9', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 500, 'save_total_limit': 3, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'adamw_torch', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False, 'eval_on_start': False, 'sortish_sampler': False, 'predict_with_generate': True, 'generation_max_length': None, 'generation_num_beams': None, 'generation_config': None}
50
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_config.py:__setitem__():151] config set model/num_parameters = 76817408 - <bound method Run._config_callback of <wandb.sdk.wandb_run.Run object at 0x7ac41ae98400>>
51
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb model/num_parameters 76817408 None
 
 
 
 
49
  2024-07-19 15:13:25,433 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb None None {'vocab_size': 62802, 'decoder_vocab_size': 62802, 'max_position_embeddings': 512, 'd_model': 512, 'encoder_ffn_dim': 2048, 'encoder_layers': 6, 'encoder_attention_heads': 8, 'decoder_ffn_dim': 2048, 'decoder_layers': 6, 'decoder_attention_heads': 8, 'dropout': 0.1, 'attention_dropout': 0.0, 'activation_dropout': 0.0, 'activation_function': 'swish', 'init_std': 0.02, 'encoder_layerdrop': 0.0, 'decoder_layerdrop': 0.0, 'use_cache': True, 'num_hidden_layers': 6, 'scale_embedding': True, 'share_encoder_decoder_embeddings': True, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': None, 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': True, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 512, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 4, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': [[62801]], 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': 0, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MarianMTModel'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1', 2: 'LABEL_2'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1, 'LABEL_2': 2}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 0, 'pad_token_id': 62801, 'eos_token_id': 0, 'sep_token_id': None, 'decoder_start_token_id': 62801, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'Helsinki-NLP/opus-mt-en-ar', 'transformers_version': '4.42.3', 'add_bias_logits': False, 'add_final_layer_norm': False, 'classif_dropout': 0.0, 'classifier_dropout': 0.0, 'extra_pos_embeddings': 62802, 'model_type': 'marian', 'normalize_before': False, 'normalize_embedding': False, 'static_position_embeddings': True, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 16, 'per_device_eval_batch_size': 16, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 1, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 2e-05, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jul19_15-13-23_8df4908137f9', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 500, 'save_total_limit': 3, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'adamw_torch', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False, 'eval_on_start': False, 'sortish_sampler': False, 'predict_with_generate': True, 'generation_max_length': None, 'generation_num_beams': None, 'generation_config': None}
50
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_config.py:__setitem__():151] config set model/num_parameters = 76817408 - <bound method Run._config_callback of <wandb.sdk.wandb_run.Run object at 0x7ac41ae98400>>
51
  2024-07-19 15:13:25,439 INFO MainThread:35 [wandb_run.py:_config_callback():1382] config_cb model/num_parameters 76817408 None
52
+ 2024-07-19 16:38:06,262 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
53
+ 2024-07-19 16:38:06,262 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
54
+ 2024-07-19 16:38:06,267 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
wandb/run-20240719_150308-wq61zns9/run-wq61zns9.wandb CHANGED
Binary files a/wandb/run-20240719_150308-wq61zns9/run-wq61zns9.wandb and b/wandb/run-20240719_150308-wq61zns9/run-wq61zns9.wandb differ