Upload togethercomputer/GPT-JT-6B-v0 ctranslate fp16 weights

README.md

@@ -22,18 +22,18 @@ widget:
   example_title: "Question Answering"
 ---
 # # Fast-Inference with Ctranslate2
-Speedup inference by 2x-8x using int8 inference in C++
+Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on CPU or GPU.
 
 quantized version of [togethercomputer/GPT-JT-6B-v0](https://huggingface.co/togethercomputer/GPT-JT-6B-v0)
 ```bash
-pip install hf-hub-ctranslate2>=2.0.6 ctranslate2>=3.13.0
+pip install hf-hub-ctranslate2>=2.0.6 
 ```
 Converted on 2023-05-19 using
 ```
 ct2-transformers-converter --model togethercomputer/GPT-JT-6B-v0 --output_dir /home/michael/tmp-ct2fast-GPT-JT-6B-v0 --force --copy_files merges.txt tokenizer.json README.md tokenizer_config.json vocab.json special_tokens_map.json added_tokens.json .gitattributes --quantization float16
 ```
 
-Checkpoint compatible to [ctranslate2](https://github.com/OpenNMT/CTranslate2) and [hf-hub-ctranslate2](https://github.com/michaelfeil/hf-hub-ctranslate2)
+Checkpoint compatible to [ctranslate2>=3.13.0](https://github.com/OpenNMT/CTranslate2) and [hf-hub-ctranslate2>=2.0.6](https://github.com/michaelfeil/hf-hub-ctranslate2)
 - `compute_type=int8_float16` for `device="cuda"` 
 - `compute_type=int8`  for `device="cpu"`
 
@@ -51,7 +51,7 @@ model = GeneratorCT2fromHfHub(
         tokenizer=AutoTokenizer.from_pretrained("togethercomputer/GPT-JT-6B-v0")
 )
 outputs = model.generate(
-    text=["How do you call a fast Flan-ingo?", "User: How are you doing?"],
+    text=["How do you call a fast Flan-ingo?", "User: How are you doing? Bot:"],
 )
 print(outputs)
 ```