Add Astraios

README.md

@@ -17,7 +17,7 @@ pinned: false
 
 # BigCode
 
-BigCode is an open scientific collaboration working on responsible training of large language models for coding applications. You can find more information on the main [website](https://www.bigcode-project.org/) or follow Big Code on [Twitter](https://twitter.com/BigCodeProject). In this organization you can find the artefacts of this collaboration: **StarCoder**, a state-of-the-art language model for code, **OctoPack**, artifacts for instruction tuning large code models, **The Stack**, the largest available pretraining dataset with perimssive code, and **SantaCoder**, a 1.1B parameter model for code.
+BigCode is an open scientific collaboration working on responsible training of large language models for coding applications. You can find more information on the main [website](https://www.bigcode-project.org/) or follow Big Code on [Twitter](https://twitter.com/BigCodeProject). In this organization you can find the artefacts of this collaboration: **StarCoder 2**, a state-of-the-art language model for code, and the previous **StarCoder** family of models, **The Stack**, the largest available pretraining dataset with perimssive code, **Astraios**, scaling instruction-tuned language models for code via diverse fine-tuning methods, **OctoPack**, artifacts for instruction tuning large code models, and **SantaCoder**, a 1.1B parameter model for code.
 
 ---
 <details>
@@ -83,6 +83,17 @@ BigCode is an open scientific collaboration working on responsible training of l
   - [StarCoder Membership Test](https://stack.dataportraits.org/): Blazing fast test if code was present in pretraining dataset.
 </details>
 ---
+<details>
+  <summary>
+    <b><font size="+1">📑The Stack</font></b>
+  </summary>
+  The Stack v1 is a 6.4TB dataset of source code in 358 programming languages from permissive licenses.
+  
+  - [The Stack](https://huggingface.co/datasets/bigcode/the-stack): Exact deduplicated version of The Stack.
+  - [The Stack dedup](https://huggingface.co/datasets/bigcode/the-stack-dedup): Near deduplicated version of The Stack (recommended for training).
+  - [Am I in the Stack](https://huggingface.co/spaces/bigcode/in-the-stack): Check if your data is in The Stack and request opt-out.
+</details>
+---
 <details>
   <summary>
     <b><font size="+1">🐙OctoPack</font></b>
@@ -103,13 +114,17 @@ BigCode is an open scientific collaboration working on responsible training of l
 ---
 <details>
   <summary>
-    <b><font size="+1">📑The Stack</font></b>
+    <b><font size="+1">✨Astraios</font></b>
   </summary>
-  The Stack v1 is a 6.4TB dataset of source code in 358 programming languages from permissive licenses.
   
-  - [The Stack](https://huggingface.co/datasets/bigcode/the-stack): Exact deduplicated version of The Stack.
-  - [The Stack dedup](https://huggingface.co/datasets/bigcode/the-stack-dedup): Near deduplicated version of The Stack (recommended for training).
-  - [Am I in the Stack](https://huggingface.co/spaces/bigcode/in-the-stack): Check if your data is in The Stack and request opt-out.
+  Astraios is a model suite of scaling 28 instruction-tuned language models for code.
+  
+  - [Paper](https://arxiv.org/abs/2401.00788): Research paper with details about all components of Astraios.
+  - [GitHub](https://github.com/bigcode-project/astraios): All code used for the creation of Astraios.
+  - [Astraios-1B](https://huggingface.co/collections/bigcode/astraios-1b-6576ff1b8e449026ae327c1c): Collection of StarCoderBase-1B models instruction tuned on CommitPackFT + OASST with 7 method.
+  - [Astraios-3B](https://huggingface.co/collections/bigcode/astraios-3b-6577127317ee44ff547252d3): Collection of StarCoderBase-3B models instruction tuned on CommitPackFT + OASST with 7 method.
+  - [Astraios-7B](https://huggingface.co/collections/bigcode/astraios-7b-65788b509c5c26f96c08d576): Collection of StarCoderBase-7B models instruction tuned on CommitPackFT + OASST with 7 method.
+  - [Astraios-15B](https://huggingface.co/collections/bigcode/astraios-15b-65788b7476b6de79781054cc): Collection of StarCoderBase-15B models instruction tuned on CommitPackFT + OASST with 7 method.
 </details>
 ---
 <details>