jeffra commited on
Commit
0a2b9c6
1 Parent(s): 3cc5dd0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -38
README.md CHANGED
@@ -11,13 +11,17 @@ tags:
11
  Arctic is a Dense-MoE Hybrid transformer architecture pre-trained from scratch by the Snowflake AI
12
  Research Team. We are releasing model checkpoints for both the base and instruct-tuned versions of
13
  Arctic under an Apache-2.0 license. This means you can use them freely in your own research,
14
- prototypes, and products. Please see our blog [Snowflake Arctic: Efficient Intelligence, Truly Open]()
 
15
  for more information on Arctic and links to other relevant resources such as our series of cookbooks
16
  covering topics around training your own custom MoE models, how to produce high-quality training data,
17
  and much more.
18
 
19
- * [Arctic-Base](link-here)
20
- * [Acrtic-Instruct](link-to-instruct)
 
 
 
21
 
22
  **Model developers** Snowflake
23
 
@@ -43,49 +47,19 @@ model implementation. Until this support is released please follow these instruc
43
  required dependencies for using Arctic:
44
 
45
  ```python
46
- pip install git+https://github.com/Snowflake-Labs/transformers.git
47
  ```
48
 
49
  Arctic leverages several features from [DeepSpeed](https://github.com/microsoft/DeepSpeed), you will need to
50
  install the latest version of DeepSpeed to get all of these required features:
51
 
52
  ```python
53
- pip install "deepspeed>=0.15.0"
54
  ```
55
 
56
  ### Inference
57
 
58
- To get the best performance with Arctic we highly recommend using TRT-LLM or vLLM for inference. However you
59
- can also use `transformers` to load
60
- the model for text generation. Due to the model size we recommend using a single 8xH100 instance from your
61
- favorite cloud provider such as: AWS [p5.48xlarge](https://aws.amazon.com/ec2/instance-types/p5/),
62
- Azure [ND96isr_H100_v5](https://learn.microsoft.com/en-us/azure/virtual-machines/nd-h100-v5-series), etc.
63
-
64
- In addition, if you would like to access Acrtic via API we have colloborated with several inference API
65
- providers to host Acrtic such as AWS, Microsoft Azure, NVIDIA Foundry, Lamini, Perplexity, Replicate and Together.
66
-
67
- ```python
68
- import torch
69
- from transformers import AutoTokenizer, AutoModelForCausalLM
70
-
71
- tokenizer = AutoTokenizer.from_pretrained("snowflake/arctic")
72
- model = AutoModelForCausalLM.from_pretrained("snowflake/arctic", device_map="auto", torch_dtype=torch.bfloat16)
73
-
74
- input_text = "Hello my name is "
75
- input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
76
-
77
- outputs = model.generate(**input_ids, max_new_tokens=20)
78
- print(tokenizer.decode(outputs[0]))
79
- ```
80
-
81
- ### Fine-Tuning
82
-
83
- TODO: add link and extra details about fine-tuning scripts
84
-
85
- ## Metrics
86
-
87
- TODO: add summary of metrics here, we don't necessarily need to compare to others but we can if we want
88
-
89
- ## Training Data
90
 
91
- TODO: add short description and links to training data related cookbook(s)
 
 
11
  Arctic is a Dense-MoE Hybrid transformer architecture pre-trained from scratch by the Snowflake AI
12
  Research Team. We are releasing model checkpoints for both the base and instruct-tuned versions of
13
  Arctic under an Apache-2.0 license. This means you can use them freely in your own research,
14
+ prototypes, and products. Please see our blog
15
+ [Snowflake Arctic: The Best LLM for Enterprise AI — Efficiently Intelligent, Truly Open](https://www.snowflake.com/blog/arctic-open-and-efficient-foundation-language-models-snowflake)
16
  for more information on Arctic and links to other relevant resources such as our series of cookbooks
17
  covering topics around training your own custom MoE models, how to produce high-quality training data,
18
  and much more.
19
 
20
+ * [Arctic-Base](https://huggingface.co/Snowflake/snowflake-arctic-base/)
21
+ * [Acrtic-Instruct](https://huggingface.co/Snowflake/snowflake-arctic-instruct/)
22
+
23
+ For the latest details about Snowflake Arctic including tutorials, etc. please refer to our github repo:
24
+ * https://github.com/Snowflake-Labs/snowflake-arctic
25
 
26
  **Model developers** Snowflake
27
 
 
47
  required dependencies for using Arctic:
48
 
49
  ```python
50
+ pip install git+https://github.com/Snowflake-Labs/transformers.git@arctic
51
  ```
52
 
53
  Arctic leverages several features from [DeepSpeed](https://github.com/microsoft/DeepSpeed), you will need to
54
  install the latest version of DeepSpeed to get all of these required features:
55
 
56
  ```python
57
+ pip install "deepspeed>=0.14.2"
58
  ```
59
 
60
  ### Inference
61
 
62
+ The Arctic github page has several resources around running inference:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
 
64
+ * Example with pure-HF: https://github.com/Snowflake-Labs/snowflake-arctic/blob/main/inference
65
+ * Tutorial using vLLM: https://github.com/Snowflake-Labs/snowflake-arctic/tree/main/inference/vllm