mylesgoose commited on
Commit
9c0d0a8
1 Parent(s): a3f0065

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -4,11 +4,13 @@ license_name: other
4
  license_link: https://ai.meta.com/llama/license
5
  ---
6
  Repairing the chat template for the model.
7
- There is a slight problem with your chat template. If you train a model with that current chat template the model starts to output as the first token <|eot_id|> and naturally the script will then halt generation. the model learns to see this:
 
 
8
 
9
  <|begin_of_text|><|start_header_id|>user<|end_header_id|>
10
 
11
- <|image|>If I had to write a haiku for this one, it would be: <|eot_id|><|start_header_id|>assistant<|end_header_id|>
12
 
13
  Here is a haiku for the image:
14
 
 
4
  license_link: https://ai.meta.com/llama/license
5
  ---
6
  Repairing the chat template for the model.
7
+ There is a slight problem with the eorigianl llam 3.2 chat template. If you train a model with that current chat template and if the training script builds the prompts
8
+ frmm a json file with the chat temaplte the model starts to output as its first token <|eot_id|> and naturally the script will then halt generation.
9
+ the model learns to see this:
10
 
11
  <|begin_of_text|><|start_header_id|>user<|end_header_id|>
12
 
13
+ <|image|>Wite Haiku: <|eot_id|><|start_header_id|>assistant<|end_header_id|>
14
 
15
  Here is a haiku for the image:
16