Mohamed Boghdady commited on
Commit
dcf45cb
1 Parent(s): df91f07

End of training

Browse files
wandb/debug-internal.log CHANGED
@@ -5133,3 +5133,33 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
5133
  2024-07-19 10:06:43,661 INFO Thread-12 :143 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_090532-oy10h8oj/files/output.log
5134
  2024-07-19 10:06:44,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5135
  2024-07-19 10:06:45,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5133
  2024-07-19 10:06:43,661 INFO Thread-12 :143 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_090532-oy10h8oj/files/output.log
5134
  2024-07-19 10:06:44,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5135
  2024-07-19 10:06:45,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5136
+ 2024-07-19 10:06:45,884 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: status_report
5137
+ 2024-07-19 10:06:46,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5138
+ 2024-07-19 10:06:47,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5139
+ 2024-07-19 10:06:48,287 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5140
+ 2024-07-19 10:06:48,324 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: pause
5141
+ 2024-07-19 10:06:48,325 INFO HandlerThread:143 [handler.py:handle_request_pause():724] stopping system metrics thread
5142
+ 2024-07-19 10:06:48,325 INFO HandlerThread:143 [system_monitor.py:finish():203] Stopping system monitor
5143
+ 2024-07-19 10:06:48,325 INFO HandlerThread:143 [interfaces.py:finish():200] Joined cpu monitor
5144
+ 2024-07-19 10:06:48,325 DEBUG SystemMonitor:143 [system_monitor.py:_start():172] Starting system metrics aggregation loop
5145
+ 2024-07-19 10:06:48,326 DEBUG SystemMonitor:143 [system_monitor.py:_start():179] Finished system metrics aggregation loop
5146
+ 2024-07-19 10:06:48,326 DEBUG SystemMonitor:143 [system_monitor.py:_start():183] Publishing last batch of metrics
5147
+ 2024-07-19 10:06:48,326 INFO HandlerThread:143 [interfaces.py:finish():200] Joined disk monitor
5148
+ 2024-07-19 10:06:48,334 INFO HandlerThread:143 [interfaces.py:finish():200] Joined gpu monitor
5149
+ 2024-07-19 10:06:48,334 INFO HandlerThread:143 [interfaces.py:finish():200] Joined memory monitor
5150
+ 2024-07-19 10:06:48,334 INFO HandlerThread:143 [interfaces.py:finish():200] Joined network monitor
5151
+ 2024-07-19 10:06:48,335 DEBUG SenderThread:143 [sender.py:send():379] send: stats
5152
+ 2024-07-19 10:06:48,365 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: resume
5153
+ 2024-07-19 10:06:48,365 INFO HandlerThread:143 [handler.py:handle_request_resume():715] starting system metrics thread
5154
+ 2024-07-19 10:06:48,365 INFO HandlerThread:143 [system_monitor.py:start():194] Starting system monitor
5155
+ 2024-07-19 10:06:48,366 INFO SystemMonitor:143 [system_monitor.py:_start():158] Starting system asset monitoring threads
5156
+ 2024-07-19 10:06:48,366 INFO SystemMonitor:143 [interfaces.py:start():188] Started cpu monitoring
5157
+ 2024-07-19 10:06:48,367 INFO SystemMonitor:143 [interfaces.py:start():188] Started disk monitoring
5158
+ 2024-07-19 10:06:48,368 INFO SystemMonitor:143 [interfaces.py:start():188] Started gpu monitoring
5159
+ 2024-07-19 10:06:48,369 INFO SystemMonitor:143 [interfaces.py:start():188] Started memory monitoring
5160
+ 2024-07-19 10:06:48,370 INFO SystemMonitor:143 [interfaces.py:start():188] Started network monitoring
5161
+ 2024-07-19 10:06:49,287 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5162
+ 2024-07-19 10:06:49,525 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: stop_status
5163
+ 2024-07-19 10:06:49,526 DEBUG SenderThread:143 [sender.py:send_request():406] send_request: stop_status
5164
+ 2024-07-19 10:06:49,663 INFO Thread-12 :143 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_090532-oy10h8oj/files/output.log
5165
+ 2024-07-19 10:06:50,287 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
wandb/debug.log CHANGED
@@ -74,3 +74,6 @@ config: {}
74
  2024-07-19 10:06:42,046 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
75
  2024-07-19 10:06:42,047 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
76
  2024-07-19 10:06:42,064 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
 
 
 
 
74
  2024-07-19 10:06:42,046 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
75
  2024-07-19 10:06:42,047 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
76
  2024-07-19 10:06:42,064 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
77
+ 2024-07-19 10:06:48,324 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
78
+ 2024-07-19 10:06:48,324 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
79
+ 2024-07-19 10:06:48,330 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
wandb/run-20240719_090532-oy10h8oj/files/output.log CHANGED
@@ -13,3 +13,5 @@ Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_word
13
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
14
  Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_words_ids': [[62833]], 'forced_eos_token_id': 0}
15
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
 
 
 
13
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
14
  Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_words_ids': [[62833]], 'forced_eos_token_id': 0}
15
  Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
16
+ Non-default generation parameters: {'max_length': 512, 'num_beams': 4, 'bad_words_ids': [[62833]], 'forced_eos_token_id': 0}
17
+ Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
wandb/run-20240719_090532-oy10h8oj/logs/debug-internal.log CHANGED
@@ -5133,3 +5133,33 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
5133
  2024-07-19 10:06:43,661 INFO Thread-12 :143 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_090532-oy10h8oj/files/output.log
5134
  2024-07-19 10:06:44,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5135
  2024-07-19 10:06:45,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5133
  2024-07-19 10:06:43,661 INFO Thread-12 :143 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_090532-oy10h8oj/files/output.log
5134
  2024-07-19 10:06:44,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5135
  2024-07-19 10:06:45,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5136
+ 2024-07-19 10:06:45,884 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: status_report
5137
+ 2024-07-19 10:06:46,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5138
+ 2024-07-19 10:06:47,286 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5139
+ 2024-07-19 10:06:48,287 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5140
+ 2024-07-19 10:06:48,324 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: pause
5141
+ 2024-07-19 10:06:48,325 INFO HandlerThread:143 [handler.py:handle_request_pause():724] stopping system metrics thread
5142
+ 2024-07-19 10:06:48,325 INFO HandlerThread:143 [system_monitor.py:finish():203] Stopping system monitor
5143
+ 2024-07-19 10:06:48,325 INFO HandlerThread:143 [interfaces.py:finish():200] Joined cpu monitor
5144
+ 2024-07-19 10:06:48,325 DEBUG SystemMonitor:143 [system_monitor.py:_start():172] Starting system metrics aggregation loop
5145
+ 2024-07-19 10:06:48,326 DEBUG SystemMonitor:143 [system_monitor.py:_start():179] Finished system metrics aggregation loop
5146
+ 2024-07-19 10:06:48,326 DEBUG SystemMonitor:143 [system_monitor.py:_start():183] Publishing last batch of metrics
5147
+ 2024-07-19 10:06:48,326 INFO HandlerThread:143 [interfaces.py:finish():200] Joined disk monitor
5148
+ 2024-07-19 10:06:48,334 INFO HandlerThread:143 [interfaces.py:finish():200] Joined gpu monitor
5149
+ 2024-07-19 10:06:48,334 INFO HandlerThread:143 [interfaces.py:finish():200] Joined memory monitor
5150
+ 2024-07-19 10:06:48,334 INFO HandlerThread:143 [interfaces.py:finish():200] Joined network monitor
5151
+ 2024-07-19 10:06:48,335 DEBUG SenderThread:143 [sender.py:send():379] send: stats
5152
+ 2024-07-19 10:06:48,365 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: resume
5153
+ 2024-07-19 10:06:48,365 INFO HandlerThread:143 [handler.py:handle_request_resume():715] starting system metrics thread
5154
+ 2024-07-19 10:06:48,365 INFO HandlerThread:143 [system_monitor.py:start():194] Starting system monitor
5155
+ 2024-07-19 10:06:48,366 INFO SystemMonitor:143 [system_monitor.py:_start():158] Starting system asset monitoring threads
5156
+ 2024-07-19 10:06:48,366 INFO SystemMonitor:143 [interfaces.py:start():188] Started cpu monitoring
5157
+ 2024-07-19 10:06:48,367 INFO SystemMonitor:143 [interfaces.py:start():188] Started disk monitoring
5158
+ 2024-07-19 10:06:48,368 INFO SystemMonitor:143 [interfaces.py:start():188] Started gpu monitoring
5159
+ 2024-07-19 10:06:48,369 INFO SystemMonitor:143 [interfaces.py:start():188] Started memory monitoring
5160
+ 2024-07-19 10:06:48,370 INFO SystemMonitor:143 [interfaces.py:start():188] Started network monitoring
5161
+ 2024-07-19 10:06:49,287 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
5162
+ 2024-07-19 10:06:49,525 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: stop_status
5163
+ 2024-07-19 10:06:49,526 DEBUG SenderThread:143 [sender.py:send_request():406] send_request: stop_status
5164
+ 2024-07-19 10:06:49,663 INFO Thread-12 :143 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240719_090532-oy10h8oj/files/output.log
5165
+ 2024-07-19 10:06:50,287 DEBUG HandlerThread:143 [handler.py:handle_request():158] handle_request: internal_messages
wandb/run-20240719_090532-oy10h8oj/logs/debug.log CHANGED
@@ -74,3 +74,6 @@ config: {}
74
  2024-07-19 10:06:42,046 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
75
  2024-07-19 10:06:42,047 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
76
  2024-07-19 10:06:42,064 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
 
 
 
 
74
  2024-07-19 10:06:42,046 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
75
  2024-07-19 10:06:42,047 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
76
  2024-07-19 10:06:42,064 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
77
+ 2024-07-19 10:06:48,324 INFO MainThread:35 [jupyter.py:save_ipynb():372] not saving jupyter notebook
78
+ 2024-07-19 10:06:48,324 INFO MainThread:35 [wandb_init.py:_pause_backend():440] pausing backend
79
+ 2024-07-19 10:06:48,330 INFO MainThread:35 [wandb_init.py:_resume_backend():445] resuming backend
wandb/run-20240719_090532-oy10h8oj/run-oy10h8oj.wandb CHANGED
Binary files a/wandb/run-20240719_090532-oy10h8oj/run-oy10h8oj.wandb and b/wandb/run-20240719_090532-oy10h8oj/run-oy10h8oj.wandb differ