Saving weights and logs of epoch 1

flax_model.msgpack

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f83fca6aac012f0d8d1d7998627499ab247ce39eb6e24ea266ae1c63b2f00522
+oid sha256:52cf7647d248e462c32ef2fdc2363f645c2265810beee6b6f4d639838911e230
 size 2353635949

wandb/run-20220405_210343-1w1tow7f/files/media/table/predictions/epoch_2_333_1efa89735ecf2ed7ea1c.table.json

@@ -0,0 +1 @@
+{"columns": ["label_str", "pred_str"], "data": [["never drink any hard liquors such as whisky brandy gin or cocktails with oysters or clams as it is liable to upset you for the rest of the evening", "and the people of the city"], ["give me my horse and arms and wait for me here i will go in quest of this knight and dead or alive i will make him keep his word plighted to so great beauty", "and the first thing he did was to take him to the doctor"], ["the lofty icy shasta towering high above all seems but an hour's walk from you though the distance in an air line is about sixty miles", "and the first thing he did was to take him to the doctor"], ["shortly after passing one of these chapels we came suddenly upon a village which started up out of the mist and i was alarmed lest i should be made an object of curiosity or dislike", "and the people of the city"], ["he wrote the question and gave it to me i read it i could not understand either the subject or the meaning of the words but it did not matter i had to give an answer", "and the first man to see him"], ["whatever her sweet eyes are turned upon spirits of love do issue thence in flame which through their eyes who then may look on them pierce to the heart's deep chamber every one", "and the first thing he did was to take him to the doctor"], ["probably a pessimistic suspicion with regard to the entire situation of man will find expression perhaps a condemnation of man together with his situation", "and the whole city was a mass of them"], ["my guides however were well known and the natural politeness of the people prevented them from putting me to any inconvenience but they could not help eyeing me nor i them", "and the first thing he did was to take him to the doctor"], ["so lonely and so solemn with the sad grey clouds above and no sound save a lost lamb bleating upon the mountain side as though its little heart were breaking", "and the first thing he did was to take him to the doctor"], ["a man's estimates of value betray something of the structure of his soul and wherein it sees its conditions of life its intrinsic needs", "and the first thing he did was to take him to the doctor"], ["he lives thy loss he dies from every limb mangled by thee lightnings of godhead shine from which thy darkness hath not where to hide", "and the first thing he did was to take him to the doctor"], ["there was no one in the whole world who had the smallest idea save those who were themselves on the other side of it if indeed there was any one at all could i hope to cross it", "and the first thing he did was to take him to the doctor"], ["as he went about his eyes were ever cast downwards and he walked with a quick shuffling gait and he suspected others feeling that he himself was suspected", "and the first thing he did was to take him to the doctor"], ["the regent's lady ordered one of her servants to give the eighty crowns that had been assessed as her share at once for the captains had already paid down their sixty", "and the first thing he did was to take him to the doctor"], ["little did i expect however the spectacle which awaited us when we reached the peninsula of sneffels where agglomerations of nature's ruins form a kind of terrible chaos", "and the man who had been a friend of his"], ["we saw the united states flag flying from the ramparts and thought that yank would probably be asleep or catching lice or maybe engaged in a game of seven up", "and the first thing he did was to take him to the doctor"], ["beat the yolks of the other two eggs add them with a little flour and salt to those pounded mix all well together and roll into balls", "and the first thing he did was to take him to the doctor"], ["its yellow bristles rather a mane than a head of hair covered and concealed a lofty brow evidently made to contain thought", "and the man who had been a friend of his"], ["then i had much pleasure in reading it but was indeed surprised at the many little points of similarity between the two books in spite of their entire independence to one another", "and the first thing he did was to take him to the doctor"], ["any one with the slightest pretensions to refined cookery must in this particular implicitly follow the example of our friends across the channel", "and the whole city was a mass of them"], ["when a married woman has followers and the husband don't go the wrong side of the post too or it ain't proved again him that he do they'll never let her have nothing to do with the children", "and the man who had been a friend of his"], ["thereafter this sonnet bred in me desire to write down in verse four other things touching my condition the which things it seemed to me that i had not yet made manifest", "and the man who had been a friend of his"], ["and so strong was my phantasy that i wept again in very truth and said with my true voice o excellent soul how blessed is he that now looketh upon thee", "and the people of the city were not so frightened"], ["her meeting with letty was indescribably tender and the days that followed were pretty equally divided between her and her brother in nursing the one and loving the other", "and the first thing he did was to take him to the doctor"], ["at last he sent word to say that he himself would be in england before the end of march and would see that the majesty of the law should be vindicated in his favour", "and the first thing he did was to take him to the doctor"], ["which thing being thus there came a day when certain ladies to whom it was well known they having been with me at divers times in my trouble were met together for the pleasure of gentle company", "and the man who had been a friend of his"], ["one of newsome's chief pleasures in life had been the educating of his woodland bride and it was the ambition of both that the little minnetaki and her brother be reared in the ways of white children", "and the man who had been a friend of his"], ["but trevelyan was of a different opinion and he was disgusted and revolted most unreasonably by the appearance of his minister's domestic arrangements", "and the first thing he did was to take him to the doctor"], ["right before me i saw the long dry grass all bending toward a common center and i knew that it was an old well and that my comrade had fallen in it", "and the man who had been a friend of his"], ["there were a good many sheep which i knew as two or three black ewes and a black lamb or two and several others which had some distinguishing mark whereby i could tell them", "and the first thing he did was to take him to the doctor"], ["mode put the whole of the ingredients into a bottle and let it remain for a fortnight in a warm place occasionally shaking up the contents", "and the man who had been a friend of his"], ["it is an idea that is worth while but unfortunately the proprietors depend too much on the decorative feature and too little on the food and how they serve it", "and the man who had been a friend of his"], ["this book in its original form was received with favour and settled the claim of rossetti to rank as a poetic translator or indeed as a poet in his own right", "and the whole city was in a state of surprise"], ["this juice which is called citric acid may be preserved in bottles for a considerable time by covering it with a thin stratum of oil", "and the first thing he did was to take him to the doctor"], ["if you know how to order and do not care to count the cost when you order probably the best dinner at these restaurants can be had at either blanco's or the poodle dog", "and the people of the city"], ["everywhere that slave morality gains the ascendancy language shows a tendency to approximate the significations of the words good and stupid", "and the first man to see him"], ["nor had this good woman's officious labours taken the least from the awkwardness of the silence which as soon as the bustle she had made was over returned in its full force", "and the whole city was in a state of surprise"], ["put the udder into a stewpan with sufficient water to cover it let it stew gently till quite done when take it out to cool", "and the first thing he did was to take him to the doctor"], ["salad two cups of apples cut into small pieces one cup celery cut into small pieces one cup english walnuts", "and the man who had been a friend of his"], ["but this power of discernment was denied them and only in after years with the loved ones of their own firesides close about them was the whole picture revealed", "and the man who had been a friend of the family"], ["your apartment is ready you may send your clothes you shall have a servant a gondola at your orders my own table and ten sequins a month", "and the first man to see him"], ["missus horton rose from her seat moved the decanters and fruit round the table stirred the fire and came back to her seat again before another word was uttered", "and the whole city was in a state of surprise"], ["he was mounted upon a powerful horse and had on a coat of mail with four of the pistols they call petronels in that country at his waist", "and the first thing he did was to take him to the doctor"], ["cabbage salad chop or shave fine half a medium size head of cabbage that has been left in cold water until crisp then drain", "and the man who had been a friend of his"], ["a thousand thousand voices are heard but so finely blended they seem a part of the night itself and make a deeper silence", "and the first thing he did was to take him to the doctor"], ["cauliflower mayonnaise take cold boiled cauliflower break into branches adding salt pepper and vinegar to season", "and the first man to see him"], ["i saw but few inhabitants during my excursion but i met a crowd on the beach drying salting and loading codfish the principal article of exportation", "and the man who had been a friend of his"], ["missus bozzle who well understood that business was business and that wives were not business felt no anger at this and handed her husband his best coat", "and the man who had been a friend of his"], ["i don't want you to i love to see you so young and happy only you are not the old david and i've got to get acquainted with the new one", "and the first thing he did was to take him to the doctor"], ["this second part is divided into two for in the one i speak of the eyes which are the beginning of love in the second i speak of the mouth which is the end of love", "and the first thing he did was to take him to the doctor"]]}

wandb/run-20220405_210343-1w1tow7f/files/output.log

@@ -970,3 +970,165 @@ Model weights saved in /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/flax_m
 tokenizer config file saved in ./tokenizer_config.json
 Special tokens file saved in ./special_tokens_map.json
 04/05/2022 21:29:22 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['~/cache/huggingface/datasets/downloads/tmpqvtsgw_8']. This may take a bit of time if the files are large.
+Adding files tracked by Git LFS: ['~/cache/huggingface/datasets/downloads/tmpqvtsgw_8']. This may take a bit of time if the files are large.
+Epoch... (1/10 | Eval Loss: 4.616994857788086 | Eval wer: 1.0104942772898766 |):  10%| | 1/10 [25:28<3:49:12, 1528.08s/i
+
+
+
+
+
+
+
+
+Training...:   5%|███▎                                                                  | 8/167 [00:15<04:55,  1.86s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Training...:  20%|██████████████                                                       | 34/167 [00:50<02:17,  1.04s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Training...:  35%|████████████████████████▍                                            | 59/167 [01:26<02:57,  1.64s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+Training...:  50%|██████████████████████████████████▎                                  | 83/167 [01:50<01:26,  1.03s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Training...:  65%|███████████████████████████████████████████▉                        | 108/167 [02:30<01:10,  1.19s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Training...:  81%|██████████████████████████████████████████████████████▉             | 135/167 [03:06<00:58,  1.84s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+Training...:  94%|███████████████████████████████████████████████████████████████▉    | 157/167 [03:33<00:09,  1.04it/s]
+
+
+
+
+Training...: 100%|████████████████████████████████████████████████████████████████████| 167/167 [03:42<00:00,  1.13s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Evaluating ...:  99%|████████████████████████████████████████████████████████████████▌| 132/133 [01:18<00:00,  3.07it/s]
+
+
+tcmalloc: large alloc 1730322432 bytes == 0x5f44ee000 @  0x7fe042c4c680 0x7fe042c6cbdd 0x7fdedde251ff 0x7fdedde3442c 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde2f164 0x7fdedde2f91e 0x503fb6 0x56b1da 0x56939a 0x5f6a13 0x56c28c 0x56939a 0x5f6a13 0x56c28c 0x5f6836 0x56b0ae 0x56939a 0x50aaa0 0x56c28c 0x56939a 0x5f6a13 0x56b0ae 0x56939a 0x68d047
+tcmalloc: large alloc 3493740544 bytes == 0x6927a0000 @  0x7fe042c4c680 0x7fe042c6cbdd 0x7fdedde251ff 0x7fdedde3442c 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde3541d 0x7fdedde2f164 0x7fdedde2f91e 0x503fb6 0x56b1da 0x56939a 0x5f6a13 0x56c28c 0x56939a 0x5f6a13 0x56c28c 0x5f6836 0x56b0ae 0x56939a 0x50aaa0 0x56c28c 0x56939a 0x5f6a13 0x56b0ae 0x56939a 0x68d047
+tcmalloc: large alloc 2353643520 bytes == 0x59e8c2000 @  0x7fe042c4c680 0x7fe042c6d824 0x5f8a01 0x7fdedde2f209 0x7fdedde2f91e 0x503fb6 0x56b1da 0x56939a 0x5f6a13 0x56c28c 0x56939a 0x5f6a13 0x56c28c 0x5f6836 0x56b0ae 0x56939a 0x50aaa0 0x56c28c 0x56939a 0x5f6a13 0x56b0ae 0x56939a 0x68d047 0x67e351 0x67e3cf 0x67e471 0x67e817 0x6b6fe2 0x6b736d 0x7fe042a5e0b3 0x5fa5ce
+Model weights saved in /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/flax_model.msgpack
+tokenizer config file saved in ./tokenizer_config.json

wandb/run-20220405_210343-1w1tow7f/logs/debug-internal.log

@@ -503,3 +503,282 @@
 2022-04-05 21:29:23,730 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
 2022-04-05 21:29:24,903 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
 2022-04-05 21:29:24,903 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:29:39,949 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:29:39,949 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:29:41,738 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:43,739 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:46,740 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:48,382 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:29:48,742 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:50,743 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:52,743 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:54,744 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:55,265 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:29:55,293 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:29:56,745 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:29:58,184 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:29:58,673 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:29:58,746 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:00,747 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:02,748 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:04,749 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:06,750 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:09,751 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:10,514 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:30:10,536 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:30:11,752 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:13,753 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:15,754 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:17,755 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:18,459 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:30:19,756 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:21,758 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:23,759 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:25,760 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:25,782 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:30:25,782 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:30:27,761 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:29,762 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:30,705 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:30:30,713 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:30:30,716 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:30:30,719 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:30:30,762 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:30:31,104 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:30:31,763 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:32,763 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:33,764 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:35,765 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:37,766 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:39,767 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:40,993 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:30:40,993 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:30:41,768 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:43,768 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:46,770 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:48,583 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:30:48,771 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:50,772 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:52,773 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:54,774 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:56,179 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:30:56,180 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:30:56,775 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:30:58,776 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:00,777 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:02,778 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:04,779 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:06,780 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:07,134 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:31:07,142 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:31:07,145 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:31:07,149 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:31:07,550 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:31:07,780 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:31:08,781 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:09,781 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:10,782 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:11,245 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:31:11,245 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:31:13,783 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:15,784 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:17,785 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:18,684 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:31:19,786 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:21,787 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:23,788 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:25,789 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:26,347 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:31:26,347 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:31:27,790 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:29,791 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:31,792 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:33,793 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:33,838 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:31:33,845 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:31:33,848 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:31:33,852 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:31:34,314 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:31:34,794 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:31:35,794 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:37,795 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:39,796 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:41,504 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:31:41,504 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:31:41,797 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:43,798 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:46,799 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:48,784 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:31:48,800 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:50,801 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:52,802 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:54,803 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:56,683 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:31:56,684 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:31:56,804 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:31:58,806 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:00,807 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:02,808 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:04,809 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:06,809 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:08,810 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:10,811 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:11,822 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:32:11,822 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:32:12,479 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:32:12,486 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:32:12,489 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:32:12,493 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:32:12,812 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:32:12,813 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:12,918 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:32:13,813 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:15,814 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:17,815 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:18,872 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:32:19,816 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:21,817 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:23,818 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:25,819 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:27,017 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:32:27,018 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:32:27,820 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:29,821 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:31,822 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:33,823 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:35,824 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:37,825 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:39,826 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:41,827 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:42,211 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:32:42,211 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:32:43,828 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:45,031 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:32:45,038 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:32:45,041 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:32:45,045 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:32:45,504 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:32:45,829 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:32:46,830 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:48,831 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:48,956 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:32:50,832 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:52,833 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:54,834 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:56,835 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:32:57,858 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:32:57,858 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:32:58,836 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:00,837 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:02,838 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:04,839 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:06,840 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:08,841 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:10,842 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:13,158 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:33:13,158 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:33:13,844 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:15,437 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:33:15,444 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:33:15,447 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:33:15,450 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:33:15,794 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:33:15,845 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:33:15,845 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:16,845 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:17,846 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:19,032 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:33:19,847 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:21,848 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:23,849 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:25,850 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:27,851 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:28,320 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:33:28,320 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:33:29,852 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:31,853 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:33,854 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:36,855 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:38,856 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:40,857 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:42,858 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:43,505 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:33:43,506 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:33:44,859 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:46,860 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:48,861 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:49,110 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:33:50,862 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:52,863 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:54,864 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:56,865 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:33:58,610 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:33:58,610 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:33:58,866 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:00,866 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:02,867 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:04,868 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:06,869 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:08,870 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:11,872 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:13,714 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:34:13,714 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:34:13,873 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:15,874 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:17,875 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:19,188 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:34:19,876 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:21,877 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:23,878 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:25,879 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:27,880 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:28,849 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:34:28,850 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:34:29,881 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:31,882 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:33,883 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:35,884 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:37,885 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:39,886 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:41,887 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:42,160 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:34:42,168 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:34:42,168 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:34:42,171 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:34:42,175 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:34:42,227 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: log_artifact
+2022-04-05 21:34:42,227 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: log_artifact
+2022-04-05 21:34:42,291 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: partial_history
+2022-04-05 21:34:42,670 INFO    Thread-16 :1450636 [upload_job.py:push():95] Uploaded file /home/sanchitgandhi/.cache/wandb/artifacts/obj/md5/c6/ff7944c7032c2c6cf9ed25f7fca81d
+2022-04-05 21:34:42,888 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:34:42,888 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/media/table/predictions/epoch_2_333_1efa89735ecf2ed7ea1c.table.json
+2022-04-05 21:34:42,888 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/media/table/predictions
+2022-04-05 21:34:43,118 INFO    SenderThread:1450636 [sender.py:send_request_log_artifact():976] logged artifact run-1w1tow7f-predictionsepoch_2 - {'id': 'QXJ0aWZhY3Q6MTEwMzA4MTMx', 'digest': 'ae38b1b050a4ef542810677a57472fec', 'state': 'PENDING', 'aliases': [], 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjE4MDAwMjQ1', 'latestArtifact': None}, 'version': 'latest'}
+2022-04-05 21:34:43,118 DEBUG   SenderThread:1450636 [sender.py:send():235] send: files
+2022-04-05 21:34:43,118 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file media/table/predictions/epoch_2_333_1efa89735ecf2ed7ea1c.table.json with policy now
+2022-04-05 21:34:43,119 DEBUG   SenderThread:1450636 [sender.py:send():235] send: history
+2022-04-05 21:34:43,119 DEBUG   SenderThread:1450636 [sender.py:send():235] send: summary
+2022-04-05 21:34:43,123 INFO    SenderThread:1450636 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
+2022-04-05 21:34:43,601 INFO    Thread-17 :1450636 [upload_job.py:push():137] Uploaded file /tmp/tmpyk_s48iiwandb/xbl0orvn-media/table/predictions/epoch_2_333_1efa89735ecf2ed7ea1c.table.json
+2022-04-05 21:34:43,889 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/wandb-summary.json
+2022-04-05 21:34:43,889 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:44,093 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:34:44,094 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:34:48,891 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:49,267 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:34:51,893 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:53,894 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:34:59,158 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:34:59,158 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:35:11,901 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:35:13,903 INFO    Thread-7  :1450636 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-large-cnn/wandb/run-20220405_210343-1w1tow7f/files/output.log
+2022-04-05 21:35:14,204 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:35:14,204 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:35:19,344 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:35:29,252 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:35:29,252 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:35:44,295 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:35:44,295 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status
+2022-04-05 21:35:49,422 DEBUG   SenderThread:1450636 [sender.py:send():235] send: stats
+2022-04-05 21:35:59,345 DEBUG   HandlerThread:1450636 [handler.py:handle_request():141] handle_request: stop_status
+2022-04-05 21:35:59,346 DEBUG   SenderThread:1450636 [sender.py:send_request():249] send_request: stop_status