Delete .ipynb_checkpoints
Browse files- .ipynb_checkpoints/README-checkpoint.md +0 -77
- .ipynb_checkpoints/config-checkpoint.json +0 -28
- .ipynb_checkpoints/config-checkpoint.py +0 -28
- .ipynb_checkpoints/generation_config-checkpoint.json +0 -8
- .ipynb_checkpoints/special_tokens_map-checkpoint.json +0 -11
- .ipynb_checkpoints/tokenizer_config-checkpoint.json +0 -48
.ipynb_checkpoints/README-checkpoint.md
DELETED
@@ -1,77 +0,0 @@
|
|
1 |
-
---
|
2 |
-
language:
|
3 |
-
- en
|
4 |
-
pipeline_tag: text-generation
|
5 |
-
tags:
|
6 |
-
- shining-valiant
|
7 |
-
- valiant
|
8 |
-
- valiant-labs
|
9 |
-
- llama
|
10 |
-
- llama-2
|
11 |
-
- llama-2-chat
|
12 |
-
- 70b
|
13 |
-
model_type: llama
|
14 |
-
license: llama2
|
15 |
-
---
|
16 |
-
|
17 |
-
|
18 |
-
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64f267a8a4f79a118e0fcc89/5rUJPhu_6LyDvSQogSVhk.jpeg)
|
19 |
-
|
20 |
-
|
21 |
-
Shining Valiant is a chat model built on the Llama 2 architecture, finetuned on our data for insight, creativity, passion, and friendliness.
|
22 |
-
- Uses the llama-2-70b-chat model, with safetensors
|
23 |
-
- Finetuned on multiple runs across private and public data
|
24 |
-
- Data focused on knowledge, enthusiasm, and structured reasoning
|
25 |
-
|
26 |
-
## Version
|
27 |
-
|
28 |
-
The current version is **1.3!**
|
29 |
-
|
30 |
-
We're thrilled to bring you our newest release!
|
31 |
-
|
32 |
-
Previous versions remain available in the repository. New models will be released for everyone once our team's training and validation process is complete.
|
33 |
-
|
34 |
-
## Evaluation
|
35 |
-
|
36 |
-
| Model | Avg | ARC | HS | MMLU | TQA |
|
37 |
-
|-----------------------|--------|-------|-------|--------|-------|
|
38 |
-
| **Shining Valiant 1.2** | 74.17 | 72.95 | 87.88 | 70.97 | 64.88 |
|
39 |
-
| Llama 2 | 67.35 | 67.32 | 87.33 | 69.83 | 44.92 |
|
40 |
-
| Llama 2 Chat | 66.80 | 64.59 | 85.88 | 63.91 | 52.80 |
|
41 |
-
|
42 |
-
**Shining Valiant 1.3** is awaiting full results from the Open LLM Leaderboard.
|
43 |
-
|
44 |
-
SV 1.3 outperformed SV 1.2 on our internal testing.
|
45 |
-
|
46 |
-
## Prompting Guide
|
47 |
-
Shining Valiant uses the same prompt format as Llama 2 Chat - feel free to use your existing prompts and scripts!
|
48 |
-
A few examples of different formats:
|
49 |
-
|
50 |
-
1. [INST] Good morning! Can you let me know how to parse a text file and turn the semicolons into commas? [/INST]
|
51 |
-
|
52 |
-
2. [INST] (You are an intelligent, helpful AI assistant.) Hello, can you write me a thank you letter? [/INST]
|
53 |
-
|
54 |
-
3. [INST] << SYS >>You are an intelligent, helpful AI assistant.<< /SYS >>Deep dive about a country with interesting history: [/INST]
|
55 |
-
|
56 |
-
## The Model
|
57 |
-
Shining Valiant is built on top of Sunset Boulevard, which uses Llama 2's 70b parameter architecture and features upgraded general capability.
|
58 |
-
|
59 |
-
From there, we've created Shining Valiant through multiple finetuning runs on different compositions of our private dataset.
|
60 |
-
|
61 |
-
Our private data focuses primarily on applying Shining Valiant's personality: she's friendly, enthusiastic, insightful, knowledgeable, and loves to learn!
|
62 |
-
|
63 |
-
We are actively working on expanding and improving the Shining Valiant dataset for use in future releases of this model and others.
|
64 |
-
|
65 |
-
|
66 |
-
|
67 |
-
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/63444f2687964b331809eb55/VCJ8Fmefd8cdVhXSSxJiD.jpeg)
|
68 |
-
|
69 |
-
|
70 |
-
Shining Valiant is created by [Valiant Labs.](http://valiantlabs.ca/)
|
71 |
-
|
72 |
-
[Follow us on X for updates on our models!](https://twitter.com/valiant_labs)
|
73 |
-
|
74 |
-
We care about open source.
|
75 |
-
For everyone to use.
|
76 |
-
|
77 |
-
We encourage others to finetune further from our models.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/config-checkpoint.json
DELETED
@@ -1,28 +0,0 @@
|
|
1 |
-
{
|
2 |
-
"_name_or_path": "./SunsetBoulevard/",
|
3 |
-
"architectures": [
|
4 |
-
"LlamaForCausalLM"
|
5 |
-
],
|
6 |
-
"attention_bias": false,
|
7 |
-
"bos_token_id": 1,
|
8 |
-
"eos_token_id": 2,
|
9 |
-
"hidden_act": "silu",
|
10 |
-
"hidden_size": 8192,
|
11 |
-
"initializer_range": 0.02,
|
12 |
-
"intermediate_size": 28672,
|
13 |
-
"max_position_embeddings": 4096,
|
14 |
-
"model_type": "llama",
|
15 |
-
"num_attention_heads": 64,
|
16 |
-
"num_hidden_layers": 80,
|
17 |
-
"num_key_value_heads": 8,
|
18 |
-
"pad_token_id": 0,
|
19 |
-
"pretraining_tp": 1,
|
20 |
-
"rms_norm_eps": 1e-05,
|
21 |
-
"rope_scaling": null,
|
22 |
-
"rope_theta": 10000.0,
|
23 |
-
"tie_word_embeddings": false,
|
24 |
-
"torch_dtype": "float32",
|
25 |
-
"transformers_version": "4.35.0",
|
26 |
-
"use_cache": false,
|
27 |
-
"vocab_size": 32000
|
28 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/config-checkpoint.py
DELETED
@@ -1,28 +0,0 @@
|
|
1 |
-
{
|
2 |
-
"_name_or_path": "ValiantLabs/ShiningValiant",
|
3 |
-
"architectures": [
|
4 |
-
"LlamaForCausalLM"
|
5 |
-
],
|
6 |
-
"attention_bias": false,
|
7 |
-
"bos_token_id": 1,
|
8 |
-
"eos_token_id": 2,
|
9 |
-
"hidden_act": "silu",
|
10 |
-
"hidden_size": 8192,
|
11 |
-
"initializer_range": 0.02,
|
12 |
-
"intermediate_size": 28672,
|
13 |
-
"max_position_embeddings": 4096,
|
14 |
-
"model_type": "llama",
|
15 |
-
"num_attention_heads": 64,
|
16 |
-
"num_hidden_layers": 80,
|
17 |
-
"num_key_value_heads": 8,
|
18 |
-
"pad_token_id": 0,
|
19 |
-
"pretraining_tp": 1,
|
20 |
-
"rms_norm_eps": 1e-05,
|
21 |
-
"rope_scaling": null,
|
22 |
-
"rope_theta": 10000.0,
|
23 |
-
"tie_word_embeddings": false,
|
24 |
-
"torch_dtype": "float32",
|
25 |
-
"transformers_version": "4.35.0",
|
26 |
-
"use_cache": false,
|
27 |
-
"vocab_size": 32000
|
28 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/generation_config-checkpoint.json
DELETED
@@ -1,8 +0,0 @@
|
|
1 |
-
{
|
2 |
-
"_from_model_config": true,
|
3 |
-
"bos_token_id": 1,
|
4 |
-
"eos_token_id": 2,
|
5 |
-
"pad_token_id": 0,
|
6 |
-
"transformers_version": "4.35.0",
|
7 |
-
"use_cache": false
|
8 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/special_tokens_map-checkpoint.json
DELETED
@@ -1,11 +0,0 @@
|
|
1 |
-
{
|
2 |
-
"additional_special_tokens": [
|
3 |
-
"<unk>",
|
4 |
-
"<s>",
|
5 |
-
"</s>"
|
6 |
-
],
|
7 |
-
"bos_token": "<s>",
|
8 |
-
"eos_token": "</s>",
|
9 |
-
"pad_token": "</s>",
|
10 |
-
"unk_token": "<unk>"
|
11 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/tokenizer_config-checkpoint.json
DELETED
@@ -1,48 +0,0 @@
|
|
1 |
-
{
|
2 |
-
"added_tokens_decoder": {
|
3 |
-
"0": {
|
4 |
-
"content": "<unk>",
|
5 |
-
"lstrip": false,
|
6 |
-
"normalized": false,
|
7 |
-
"rstrip": false,
|
8 |
-
"single_word": false,
|
9 |
-
"special": true
|
10 |
-
},
|
11 |
-
"1": {
|
12 |
-
"content": "<s>",
|
13 |
-
"lstrip": false,
|
14 |
-
"normalized": false,
|
15 |
-
"rstrip": false,
|
16 |
-
"single_word": false,
|
17 |
-
"special": true
|
18 |
-
},
|
19 |
-
"2": {
|
20 |
-
"content": "</s>",
|
21 |
-
"lstrip": false,
|
22 |
-
"normalized": false,
|
23 |
-
"rstrip": false,
|
24 |
-
"single_word": false,
|
25 |
-
"special": true
|
26 |
-
}
|
27 |
-
},
|
28 |
-
"additional_special_tokens": [
|
29 |
-
"<unk>",
|
30 |
-
"<s>",
|
31 |
-
"</s>"
|
32 |
-
],
|
33 |
-
"bos_token": "<s>",
|
34 |
-
"clean_up_tokenization_spaces": false,
|
35 |
-
"eos_token": "</s>",
|
36 |
-
"legacy": false,
|
37 |
-
"max_length": 2048,
|
38 |
-
"model_max_length": 1000000000000000019884624838656,
|
39 |
-
"pad_token": "</s>",
|
40 |
-
"padding_side": "right",
|
41 |
-
"sp_model_kwargs": {},
|
42 |
-
"stride": 0,
|
43 |
-
"tokenizer_class": "LlamaTokenizer",
|
44 |
-
"truncation_side": "right",
|
45 |
-
"truncation_strategy": "longest_first",
|
46 |
-
"unk_token": "<unk>",
|
47 |
-
"use_default_system_prompt": true
|
48 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|