Update README.md

README.md

@@ -3,6 +3,8 @@ license: apache-2.0
 datasets:
 - bigcode/the-stack
 - HuggingFaceFW/fineweb
+base_model:
+- upiter/TinyCodeLM-400M
 ---
 
 
@@ -24,6 +26,9 @@ Despite being trained on only 72 billion tokens of text, the models outperform m
 
 **Instruction Tuning Data** TinyCodeLMs are instruction tuned on paired instruction and Python edit sequence data. These edit sequences are generated with the LintSeq algorithm over a source dataset of paired instruction and Python programs drawn from the Magicoder and StarCoder2 OSS-Instruct datasets (Wei et al., 2024).
 
+# Training Details
+TinyCodeLM models were pretrained from scratch on a single H100 node (four GPUs) for two epochs. Pretraining took about two days and six days, respectively. Instruction tuning was conducted on a single H100 GPU using DeepSpeed and took no more than several hours.
+
 # Benchmarks 
 
 **Pretrained (Temperature 0)**
@@ -54,3 +59,6 @@ Despite being trained on only 72 billion tokens of text, the models outperform m
       primaryClass={cs.LG}
 }
 ```
+
+# Safety
+This work explores data-driven mechanisms for improving the quality of language model-generated code. Our synthetic data generation method relies on open-source data and our experiments leverage open-source software and resources. It is important to acknowledge that all language models for code synthesis have the potential to be misused – whether intentionally or unintentionally – for generation of code with vulnerabilities and/or malicious behaviors. Any and all model generated code has thepotential to be harmful and must not be executed without precautions.