Upload folder using huggingface_hub

README.md

@@ -1,99 +1,65 @@
 ---
 license: mit
-language:
-  - en
+license_link: https://choosealicense.com/licenses/mit/
 ---
-
 # dolly-v2-3b-int8-ov
-
- * Model creator: [Databricks](https://huggingface.co/databricks)
+* Model creator: [Databricks](https://huggingface.co/databricks)
  * Original model: [dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b)
 
 ## Description
-
-This is [dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to int8 by [NNCF](https://github.com/openvinotoolkit/nncf).
+This is [dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to INT8 by [NNCF](https://github.com/openvinotoolkit/nncf).
 
 ## Quantization Parameters
 
 Weight compression was performed using `nncf.compress_weights` with the following parameters:
 
-* mode: **INT8_ASYM**
-* sensitivity_metric: **weight_quantization_error**
+* mode: **int8_asym**
+* ratio: **1**
 
 For more information on quantization, check the [OpenVINO model optimization guide](https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/weight-compression.html).
 
+
 ## Compatibility
 
 The provided OpenVINO™ IR model is compatible with:
 
-* OpenVINO version 2024.2.0 and higher
-* Optimum Intel 1.17.0 and higher
+* OpenVINO version 2024.4.0 and higher
+* Optimum Intel 1.20.0 and higher
 
-## Running Model Inference with [Optimum Intel](https://huggingface.co/docs/optimum/intel/index)
+## Running Model Inference
 
 1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
 
-    ```
-    pip install optimum[openvino]
-    ```
-
-2. Run model inference:
-
-    ```
-    from transformers import AutoTokenizer
-    from optimum.intel.openvino import OVModelForCausalLM
-    
-    model_id = "OpenVINO/dolly-v2-3b-int8-ov"
-    tokenizer = AutoTokenizer.from_pretrained(model_id)
-    model = OVModelForCausalLM.from_pretrained(model_id)
-    
-    inputs = tokenizer("What is OpenVINO?", return_tensors="pt")
-    
-    outputs = model.generate(**inputs, max_length=200)
-    text = tokenizer.batch_decode(outputs)[0]
-    print(text)
-    ```
-
-For more examples and possible optimizations, refer to the [OpenVINO Large Language Model Inference Guide](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide.html).
-
-## Running Model Inference with [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai)
-
-1. Install packages required for using OpenVINO GenAI.
-```
-pip install openvino-genai huggingface_hub
 ```
-
-2. Download model from HuggingFace Hub
-   
+pip install optimum[openvino]
 ```
-import huggingface_hub as hf_hub
-
-model_id = "OpenVINO/dolly-v2-3b-int8-ov"
-model_path = "dolly-v2-3b-int8-ov"
 
-hf_hub.snapshot_download(model_id, local_dir=model_path)
+2. Run model inference:
 
 ```
+from transformers import AutoTokenizer
+from optimum.intel.openvino import OVModelForCausalLM
 
-3. Run model inference:
+model_id = "OpenVINO/dolly-v2-3b-int8-ov"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = OVModelForCausalLM.from_pretrained(model_id)
 
-```
-import openvino_genai as ov_genai
+inputs = tokenizer("What is OpenVINO?", return_tensors="pt")
 
-device = "CPU"
-pipe = ov_genai.LLMPipeline(model_path, device)
-print(pipe.generate("What is OpenVINO?", max_length=200))
+outputs = model.generate(**inputs, max_length=200)
+text = tokenizer.batch_decode(outputs)[0]
+print(text)
 ```
 
-More GenAI usage examples can be found in OpenVINO GenAI library [docs](https://github.com/openvinotoolkit/openvino.genai/blob/master/src/README.md) and [samples](https://github.com/openvinotoolkit/openvino.genai?tab=readme-ov-file#openvino-genai-samples)
+For more examples and possible optimizations, refer to the [OpenVINO Large Language Model Inference Guide](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide.html).
 
 ## Limitations
 
-Check the original model card for [limitations](https://huggingface.co/databricks/dolly-v2-3b#known-limitations).
+Check the original model card for [original model card](https://huggingface.co/databricks/dolly-v2-3b) for limitations.
 
 ## Legal information
 
-The original model is distributed under [MIT](https://choosealicense.com/licenses/mit/) license. More details can be found in [original model card](https://huggingface.co/databricks/dolly-v2-3b).
+The original model is distributed under [mit](https://choosealicense.com/licenses/mit/) license. More details can be found in [original model card](https://huggingface.co/databricks/dolly-v2-3b).
 
 ## Disclaimer
 

config.json

@@ -25,13 +25,15 @@
   "model_type": "gpt_neox",
   "num_attention_heads": 32,
   "num_hidden_layers": 32,
+  "partial_rotary_factor": 0.25,
   "rope_scaling": null,
+  "rope_theta": 10000,
   "rotary_emb_base": 10000,
   "rotary_pct": 0.25,
   "tie_word_embeddings": false,
-  "torch_dtype": "float32",
-  "transformers_version": "4.40.1",
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.45.2",
   "use_cache": true,
   "use_parallel_residual": true,
   "vocab_size": 50280
-}
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "transformers_version": "4.45.2"
+}

openvino_detokenizer.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e3d0218341805b3876fc9c8e95c98d75cc9ddd0fa34fac6df212e790b6f91a08
-size 558494
+oid sha256:f1e43770f23d5b9dbfc8bf99bbea4fe501870adf36235dff20156f6c0a129a47
+size 514078

openvino_detokenizer.xml

@@ -1,16 +1,16 @@
 <?xml version="1.0"?>
 <net name="detokenizer" version="11">
 	<layers>
-		<layer id="0" name="Parameter_191" type="Parameter" version="opset1">
+		<layer id="0" name="Parameter_62647" type="Parameter" version="opset1">
 			<data shape="?,?" element_type="i64" />
 			<output>
-				<port id="0" precision="I64" names="Parameter_191">
+				<port id="0" precision="I64" names="Parameter_62647">
 					<dim>-1</dim>
 					<dim>-1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="1" name="Convert_207" type="Convert" version="opset1">
+		<layer id="1" name="Convert_62658" type="Convert" version="opset1">
 			<data destination_type="i32" />
 			<input>
 				<port id="0" precision="I64">
@@ -25,19 +25,19 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="2" name="Constant_89" type="Const" version="opset1">
-			<data element_type="u8" shape="558445" offset="0" size="558445" />
+		<layer id="2" name="Constant_62617" type="Const" version="opset1">
+			<data element_type="u8" shape="514078" offset="0" size="514078" />
 			<output>
 				<port id="0" precision="U8">
-					<dim>558445</dim>
+					<dim>514078</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="3" name="StringTensorUnpack_90" type="StringTensorUnpack" version="extension">
+		<layer id="3" name="StringTensorUnpack_62618" type="StringTensorUnpack" version="extension">
 			<data mode="begins_ends" />
 			<input>
 				<port id="0" precision="U8">
-					<dim>558445</dim>
+					<dim>514078</dim>
 				</port>
 			</input>
 			<output>
@@ -52,7 +52,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="4" name="VocabDecoder_192" type="VocabDecoder" version="extension">
+		<layer id="4" name="VocabDecoder_62648" type="VocabDecoder" version="extension">
 			<data skip_tokens="0, 1, 50277, 50278, 50279" />
 			<input>
 				<port id="0" precision="I32">
@@ -87,7 +87,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="5" name="CharsToBytes_193" type="CharsToBytes" version="extension">
+		<layer id="5" name="FuzeRagged_62649" type="FuzeRagged" version="extension">
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -101,70 +101,17 @@
 				<port id="3" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="4" precision="U8">
-					<dim>-1</dim>
-				</port>
 			</input>
 			<output>
-				<port id="5" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="6" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="7" precision="U8">
-					<dim>-1</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="6" name="Constant_195" type="Const" version="opset1">
-			<data element_type="u8" shape="47" offset="558445" size="47" />
-			<output>
-				<port id="0" precision="U8">
-					<dim>47</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="7" name="Constant_197" type="Const" version="opset1">
-			<data element_type="u8" shape="2" offset="558492" size="2" />
-			<output>
-				<port id="0" precision="U8">
-					<dim>2</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="8" name="RegexNormalization_198" type="RegexNormalization" version="extension">
-			<data global_replace="true" />
-			<input>
-				<port id="0" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="1" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="2" precision="U8">
+				<port id="4" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="3" precision="U8">
-					<dim>47</dim>
-				</port>
-				<port id="4" precision="U8">
-					<dim>2</dim>
-				</port>
-			</input>
-			<output>
 				<port id="5" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="6" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="7" precision="U8">
-					<dim>-1</dim>
-				</port>
 			</output>
 		</layer>
-		<layer id="9" name="StringTensorPack_199" type="StringTensorPack" version="extension">
+		<layer id="6" name="StringTensorPack_62650" type="StringTensorPack" version="extension">
 			<data mode="begins_ends" />
 			<input>
 				<port id="0" precision="I32">
@@ -183,7 +130,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="10" name="Result_200" type="Result" version="opset1">
+		<layer id="7" name="Result_62651" type="Result" version="opset1">
 			<input>
 				<port id="0" precision="STRING">
 					<dim>-1</dim>
@@ -198,22 +145,19 @@
 		<edge from-layer="3" from-port="1" to-layer="4" to-port="1" />
 		<edge from-layer="3" from-port="2" to-layer="4" to-port="2" />
 		<edge from-layer="3" from-port="3" to-layer="4" to-port="3" />
-		<edge from-layer="4" from-port="8" to-layer="5" to-port="4" />
-		<edge from-layer="4" from-port="7" to-layer="5" to-port="3" />
-		<edge from-layer="4" from-port="6" to-layer="5" to-port="2" />
-		<edge from-layer="4" from-port="5" to-layer="5" to-port="1" />
 		<edge from-layer="4" from-port="4" to-layer="5" to-port="0" />
-		<edge from-layer="5" from-port="5" to-layer="8" to-port="0" />
-		<edge from-layer="5" from-port="6" to-layer="8" to-port="1" />
-		<edge from-layer="5" from-port="7" to-layer="8" to-port="2" />
-		<edge from-layer="6" from-port="0" to-layer="8" to-port="3" />
-		<edge from-layer="7" from-port="0" to-layer="8" to-port="4" />
-		<edge from-layer="8" from-port="5" to-layer="9" to-port="0" />
-		<edge from-layer="8" from-port="6" to-layer="9" to-port="1" />
-		<edge from-layer="8" from-port="7" to-layer="9" to-port="2" />
-		<edge from-layer="9" from-port="3" to-layer="10" to-port="0" />
+		<edge from-layer="4" from-port="5" to-layer="5" to-port="1" />
+		<edge from-layer="4" from-port="6" to-layer="5" to-port="2" />
+		<edge from-layer="4" from-port="7" to-layer="5" to-port="3" />
+		<edge from-layer="4" from-port="8" to-layer="6" to-port="2" />
+		<edge from-layer="5" from-port="4" to-layer="6" to-port="0" />
+		<edge from-layer="5" from-port="5" to-layer="6" to-port="1" />
+		<edge from-layer="6" from-port="3" to-layer="7" to-port="0" />
 	</edges>
 	<rt_info>
+		<bos_token_id value="0" />
 		<eos_token_id value="0" />
+		<original_tokenizer_class value="&lt;class 'transformers.models.gpt_neox.tokenization_gpt_neox_fast.GPTNeoXTokenizerFast'>" />
+		<pad_token_id value="0" />
 	</rt_info>
 </net>

openvino_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:de5171039a87f9b7585e4f2e5fe335e80663abb5ff88770efe4612b912d59ee5
-size 2783023114
+oid sha256:2e6603c8164e579a9cd59c84e14449d68fb63e697917c93d864f5fd7427ad25f
+size 2779990920

openvino_tokenizer.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf4bb4307e428d32680d2973f34247c51210fad1f4c6408b2087dfdfb053e210
-size 1166376
+oid sha256:c378d88077ae7c7e13ef61745e1ceef76412338e9a7398445c09632413e52abe
+size 1227935

openvino_tokenizer.xml

@@ -1,21 +1,21 @@
 <?xml version="1.0"?>
 <net name="tokenizer" version="11">
 	<layers>
-		<layer id="0" name="Parameter_1" type="Parameter" version="opset1">
+		<layer id="0" name="Parameter_62535" type="Parameter" version="opset1">
 			<data shape="?" element_type="string" />
 			<output>
-				<port id="0" precision="STRING" names="Parameter_1">
+				<port id="0" precision="STRING" names="Parameter_62535">
 					<dim>-1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="1" name="Constant_8" type="Const" version="opset1">
+		<layer id="1" name="Constant_62541" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="0" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="2" name="StringTensorUnpack_2" type="StringTensorUnpack" version="extension">
+		<layer id="2" name="StringTensorUnpack_62536" type="StringTensorUnpack" version="extension">
 			<data mode="begins_ends" />
 			<input>
 				<port id="0" precision="STRING">
@@ -34,32 +34,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="3" name="NormalizeUnicode_3" type="NormalizeUnicode" version="extension">
-			<data normalization_form="NFC" />
-			<input>
-				<port id="0" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="1" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="2" precision="U8">
-					<dim>-1</dim>
-				</port>
-			</input>
-			<output>
-				<port id="3" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="4" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="5" precision="U8">
-					<dim>-1</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="4" name="ShapeOf_4" type="ShapeOf" version="opset3">
+		<layer id="3" name="ShapeOf_62537" type="ShapeOf" version="opset3">
 			<data output_type="i64" />
 			<input>
 				<port id="0" precision="I32">
@@ -72,19 +47,19 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="5" name="Constant_5" type="Const" version="opset1">
+		<layer id="4" name="Constant_62538" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="0" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="6" name="Constant_6" type="Const" version="opset1">
+		<layer id="5" name="Constant_62539" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="0" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="7" name="Gather_7" type="Gather" version="opset8">
+		<layer id="6" name="Gather_62540" type="Gather" version="opset8">
 			<data batch_dims="0" />
 			<input>
 				<port id="0" precision="I64">
@@ -97,13 +72,13 @@
 				<port id="3" precision="I64" />
 			</output>
 		</layer>
-		<layer id="8" name="Constant_9" type="Const" version="opset1">
+		<layer id="7" name="Constant_62542" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="8" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="9" name="Range_10" type="Range" version="opset4">
+		<layer id="8" name="Range_62543" type="Range" version="opset4">
 			<data output_type="i32" />
 			<input>
 				<port id="0" precision="I64" />
@@ -116,19 +91,19 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="10" name="Constant_12" type="Const" version="opset1">
+		<layer id="9" name="Constant_62544" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="8" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="11" name="Constant_13" type="Const" version="opset1">
+		<layer id="10" name="Constant_62545" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="8" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="12" name="Add_14" type="Add" version="opset1">
+		<layer id="11" name="Add_62546" type="Add" version="opset1">
 			<data auto_broadcast="numpy" />
 			<input>
 				<port id="0" precision="I64" />
@@ -138,13 +113,13 @@
 				<port id="2" precision="I64" />
 			</output>
 		</layer>
-		<layer id="13" name="Constant_15" type="Const" version="opset1">
+		<layer id="12" name="Constant_62547" type="Const" version="opset1">
 			<data element_type="i64" shape="" offset="8" size="8" />
 			<output>
 				<port id="0" precision="I64" />
 			</output>
 		</layer>
-		<layer id="14" name="Range_16" type="Range" version="opset4">
+		<layer id="13" name="Range_62548" type="Range" version="opset4">
 			<data output_type="i32" />
 			<input>
 				<port id="0" precision="I64" />
@@ -157,16 +132,15 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="15" name="Constant_79" type="Const" version="opset1">
-			<data element_type="u8" shape="673" offset="16" size="673" />
+		<layer id="14" name="Constant_62610" type="Const" version="opset1">
+			<data element_type="u8" shape="763" offset="16" size="763" />
 			<output>
 				<port id="0" precision="U8">
-					<dim>673</dim>
+					<dim>763</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="16" name="RegexSplit_80" type="RegexSplit" version="extension">
-			<data behaviour="isolate" invert="false" max_splits="-1" />
+		<layer id="15" name="SpecialTokensSplit_62611" type="SpecialTokensSplit" version="extension">
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -184,7 +158,7 @@
 					<dim>-1</dim>
 				</port>
 				<port id="5" precision="U8">
-					<dim>673</dim>
+					<dim>763</dim>
 				</port>
 			</input>
 			<output>
@@ -203,45 +177,13 @@
 				<port id="10" precision="U8">
 					<dim>-1</dim>
 				</port>
-			</output>
-		</layer>
-		<layer id="17" name="Constant_85" type="Const" version="opset1">
-			<data element_type="u8" shape="64" offset="689" size="64" />
-			<output>
-				<port id="0" precision="U8">
-					<dim>64</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="18" name="Constant_82" type="Const" version="opset1">
-			<data element_type="u8" shape="447" offset="753" size="447" />
-			<output>
-				<port id="0" precision="U8">
-					<dim>447</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="19" name="StringTensorUnpack_83" type="StringTensorUnpack" version="extension">
-			<data mode="begins_ends" />
-			<input>
-				<port id="0" precision="U8">
-					<dim>447</dim>
-				</port>
-			</input>
-			<output>
-				<port id="1" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="2" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="3" precision="U8">
+				<port id="11" precision="BOOL">
 					<dim>-1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="20" name="RegexSplit_86" type="RegexSplit" version="extension">
-			<data behaviour="isolate" invert="false" max_splits="-1" />
+		<layer id="16" name="NormalizeUnicode_62612" type="NormalizeUnicode" version="extension">
+			<data normalization_form="NFC" />
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -249,47 +191,38 @@
 				<port id="1" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="2" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="3" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="4" precision="U8">
-					<dim>-1</dim>
-				</port>
-				<port id="5" precision="U8">
-					<dim>64</dim>
-				</port>
-				<port id="6" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="7" precision="I32">
+				<port id="2" precision="U8">
 					<dim>-1</dim>
 				</port>
-				<port id="8" precision="U8">
+				<port id="3" precision="BOOL">
 					<dim>-1</dim>
 				</port>
 			</input>
 			<output>
-				<port id="9" precision="I32">
+				<port id="4" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="10" precision="I32">
+				<port id="5" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="11" precision="I32">
+				<port id="6" precision="U8">
 					<dim>-1</dim>
 				</port>
-				<port id="12" precision="I32">
+				<port id="7" precision="BOOL">
 					<dim>-1</dim>
 				</port>
-				<port id="13" precision="U8">
-					<dim>-1</dim>
+			</output>
+		</layer>
+		<layer id="17" name="Constant_62614" type="Const" version="opset1">
+			<data element_type="u8" shape="64" offset="779" size="64" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>64</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="21" name="BytesToChars_87" type="BytesToChars" version="extension">
+		<layer id="18" name="RegexSplit_62615" type="RegexSplit" version="extension">
+			<data behaviour="isolate" invert="false" max_splits="-1" />
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -306,38 +239,47 @@
 				<port id="4" precision="U8">
 					<dim>-1</dim>
 				</port>
+				<port id="5" precision="BOOL">
+					<dim>-1</dim>
+				</port>
+				<port id="6" precision="U8">
+					<dim>64</dim>
+				</port>
 			</input>
 			<output>
-				<port id="5" precision="I32">
+				<port id="7" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="6" precision="I32">
+				<port id="8" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="7" precision="I32">
+				<port id="9" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="8" precision="I32">
+				<port id="10" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="9" precision="U8">
+				<port id="11" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="12" precision="BOOL">
 					<dim>-1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="22" name="Constant_89" type="Const" version="opset1">
-			<data element_type="u8" shape="558445" offset="1200" size="558445" />
+		<layer id="19" name="Constant_62617" type="Const" version="opset1">
+			<data element_type="u8" shape="514078" offset="843" size="514078" />
 			<output>
 				<port id="0" precision="U8">
-					<dim>558445</dim>
+					<dim>514078</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="23" name="StringTensorUnpack_90" type="StringTensorUnpack" version="extension">
+		<layer id="20" name="StringTensorUnpack_62618" type="StringTensorUnpack" version="extension">
 			<data mode="begins_ends" />
 			<input>
 				<port id="0" precision="U8">
-					<dim>558445</dim>
+					<dim>514078</dim>
 				</port>
 			</input>
 			<output>
@@ -352,19 +294,19 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="24" name="Constant_170" type="Const" version="opset1">
-			<data element_type="u8" shape="606619" offset="559645" size="606619" />
+		<layer id="21" name="Constant_62623" type="Const" version="opset1">
+			<data element_type="u8" shape="362936" offset="514921" size="362936" />
 			<output>
 				<port id="0" precision="U8">
-					<dim>606619</dim>
+					<dim>362936</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="25" name="StringTensorUnpack_171" type="StringTensorUnpack" version="extension">
+		<layer id="22" name="StringTensorUnpack_62624" type="StringTensorUnpack" version="extension">
 			<data mode="begins_ends" />
 			<input>
 				<port id="0" precision="U8">
-					<dim>606619</dim>
+					<dim>362936</dim>
 				</port>
 			</input>
 			<output>
@@ -379,25 +321,19 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="26" name="Constant_98" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="0" size="8" />
-			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="27" name="Constant_92" type="Const" version="opset1">
-			<data element_type="u8" shape="447" offset="753" size="447" />
+		<layer id="23" name="Constant_62626" type="Const" version="opset1">
+			<data element_type="u8" shape="349500" offset="877857" size="349500" />
 			<output>
 				<port id="0" precision="U8">
-					<dim>447</dim>
+					<dim>349500</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="28" name="StringTensorUnpack_93" type="StringTensorUnpack" version="extension">
+		<layer id="24" name="StringTensorUnpack_62627" type="StringTensorUnpack" version="extension">
 			<data mode="begins_ends" />
 			<input>
 				<port id="0" precision="U8">
-					<dim>447</dim>
+					<dim>349500</dim>
 				</port>
 			</input>
 			<output>
@@ -412,150 +348,43 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="29" name="ShapeOf_94" type="ShapeOf" version="opset3">
-			<data output_type="i64" />
-			<input>
-				<port id="0" precision="I32">
-					<dim>-1</dim>
-				</port>
-			</input>
-			<output>
-				<port id="1" precision="I64">
-					<dim>1</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="30" name="Constant_95" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="0" size="8" />
-			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="31" name="Constant_96" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="0" size="8" />
+		<layer id="25" name="Constant_62620" type="Const" version="opset1">
+			<data element_type="u8" shape="462" offset="1227357" size="462" />
 			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="32" name="Gather_97" type="Gather" version="opset8">
-			<data batch_dims="0" />
-			<input>
-				<port id="0" precision="I64">
-					<dim>1</dim>
+				<port id="0" precision="U8">
+					<dim>462</dim>
 				</port>
-				<port id="1" precision="I64" />
-				<port id="2" precision="I64" />
-			</input>
-			<output>
-				<port id="3" precision="I64" />
 			</output>
 		</layer>
-		<layer id="33" name="Constant_99" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="8" size="8" />
-			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="34" name="Range_100" type="Range" version="opset4">
-			<data output_type="i32" />
+		<layer id="26" name="StringTensorUnpack_62621" type="StringTensorUnpack" version="extension">
+			<data mode="begins_ends" />
 			<input>
-				<port id="0" precision="I64" />
-				<port id="1" precision="I64" />
-				<port id="2" precision="I64" />
-			</input>
-			<output>
-				<port id="3" precision="I32">
-					<dim>-1</dim>
+				<port id="0" precision="U8">
+					<dim>462</dim>
 				</port>
-			</output>
-		</layer>
-		<layer id="35" name="Constant_102" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="8" size="8" />
-			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="36" name="Constant_103" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="8" size="8" />
-			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="37" name="Add_104" type="Add" version="opset1">
-			<data auto_broadcast="numpy" />
-			<input>
-				<port id="0" precision="I64" />
-				<port id="1" precision="I64" />
 			</input>
 			<output>
-				<port id="2" precision="I64" />
-			</output>
-		</layer>
-		<layer id="38" name="Constant_105" type="Const" version="opset1">
-			<data element_type="i64" shape="" offset="8" size="8" />
-			<output>
-				<port id="0" precision="I64" />
-			</output>
-		</layer>
-		<layer id="39" name="Range_106" type="Range" version="opset4">
-			<data output_type="i32" />
-			<input>
-				<port id="0" precision="I64" />
-				<port id="1" precision="I64" />
-				<port id="2" precision="I64" />
-			</input>
-			<output>
-				<port id="3" precision="I32">
-					<dim>-1</dim>
-				</port>
-			</output>
-		</layer>
-		<layer id="40" name="BytesToChars_168" type="BytesToChars" version="extension">
-			<input>
-				<port id="0" precision="I32">
-					<dim>-1</dim>
-				</port>
 				<port id="1" precision="I32">
 					<dim>-1</dim>
 				</port>
 				<port id="2" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="3" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="4" precision="U8">
-					<dim>-1</dim>
-				</port>
-			</input>
-			<output>
-				<port id="5" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="6" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="7" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="8" precision="I32">
-					<dim>-1</dim>
-				</port>
-				<port id="9" precision="U8">
+				<port id="3" precision="U8">
 					<dim>-1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="41" name="Constant_172" type="Const" version="opset1">
-			<data element_type="i32" shape="26" offset="1166264" size="104" />
+		<layer id="27" name="Constant_62628" type="Const" version="opset1">
+			<data element_type="i32" shape="27" offset="1227819" size="108" />
 			<output>
 				<port id="0" precision="I32">
-					<dim>26</dim>
+					<dim>27</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="42" name="BPETokenizer_173" type="BPETokenizer" version="extension">
-			<data unk_token="" fuse_unk="false" suffix_indicator="" end_suffix="" byte_fallback="false" />
+		<layer id="28" name="BPETokenizer_62629" type="BPETokenizer" version="extension">
+			<data unk_token="" fuse_unk="false" suffix_indicator="" end_suffix="" byte_fallback="false" cache_capacity="20000" />
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -600,22 +429,31 @@
 					<dim>-1</dim>
 				</port>
 				<port id="14" precision="I32">
-					<dim>26</dim>
+					<dim>-1</dim>
 				</port>
-			</input>
-			<output>
 				<port id="15" precision="I32">
 					<dim>-1</dim>
 				</port>
-				<port id="16" precision="I32">
+				<port id="16" precision="U8">
 					<dim>-1</dim>
 				</port>
 				<port id="17" precision="I32">
+					<dim>27</dim>
+				</port>
+			</input>
+			<output>
+				<port id="18" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="19" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="20" precision="I32">
 					<dim>-1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="43" name="Subtract_174" type="Subtract" version="opset1">
+		<layer id="29" name="Subtract_62630" type="Subtract" version="opset1">
 			<data auto_broadcast="numpy" />
 			<input>
 				<port id="0" precision="I32">
@@ -631,13 +469,13 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="44" name="Constant_175" type="Const" version="opset1">
-			<data element_type="i32" shape="" offset="1166368" size="4" />
+		<layer id="30" name="Constant_62631" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="1227927" size="4" />
 			<output>
 				<port id="0" precision="I32" />
 			</output>
 		</layer>
-		<layer id="45" name="Minimum_176" type="Minimum" version="opset1">
+		<layer id="31" name="Minimum_62632" type="Minimum" version="opset1">
 			<data auto_broadcast="numpy" />
 			<input>
 				<port id="0" precision="I32">
@@ -651,7 +489,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="46" name="Add_177" type="Add" version="opset1">
+		<layer id="32" name="Subtract_62633" type="Subtract" version="opset1">
 			<data auto_broadcast="numpy" />
 			<input>
 				<port id="0" precision="I32">
@@ -667,15 +505,15 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="47" name="Constant_178" type="Const" version="opset1">
-			<data element_type="i32" shape="1" offset="1166372" size="4" />
+		<layer id="33" name="Constant_62634" type="Const" version="opset1">
+			<data element_type="i32" shape="1" offset="1227931" size="4" />
 			<output>
 				<port id="0" precision="I32">
 					<dim>1</dim>
 				</port>
 			</output>
 		</layer>
-		<layer id="48" name="CombineSegments_179" type="CombineSegments" version="extension">
+		<layer id="34" name="CombineSegments_62635" type="CombineSegments" version="extension">
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -711,7 +549,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="49" name="Subtract_180" type="Subtract" version="opset1">
+		<layer id="35" name="Subtract_62636" type="Subtract" version="opset1">
 			<data auto_broadcast="numpy" />
 			<input>
 				<port id="0" precision="I32">
@@ -727,13 +565,13 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="50" name="Constant_181" type="Const" version="opset1">
-			<data element_type="i32" shape="" offset="1166372" size="4" />
+		<layer id="36" name="Constant_62637" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="1227931" size="4" />
 			<output>
 				<port id="0" precision="I32" />
 			</output>
 		</layer>
-		<layer id="51" name="ReduceMax_182" type="ReduceMax" version="opset1">
+		<layer id="37" name="ReduceMax_62638" type="ReduceMax" version="opset1">
 			<data keep_dims="false" />
 			<input>
 				<port id="0" precision="I32">
@@ -745,14 +583,14 @@
 				<port id="2" precision="I32" />
 			</output>
 		</layer>
-		<layer id="52" name="Constant_183" type="Const" version="opset1">
-			<data element_type="i32" shape="" offset="1166372" size="4" />
+		<layer id="38" name="Constant_62639" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="1227931" size="4" />
 			<output>
 				<port id="0" precision="I32" />
 			</output>
 		</layer>
-		<layer id="53" name="RaggedToDense_184" type="RaggedToDense" version="extension">
-			<data pad_right="true" />
+		<layer id="39" name="RaggedToDense_62640" type="RaggedToDense" version="extension">
+			<data pad_right="false" />
 			<input>
 				<port id="0" precision="I32">
 					<dim>-1</dim>
@@ -777,7 +615,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="54" name="Convert_185" type="Convert" version="opset1">
+		<layer id="40" name="Convert_62641" type="Convert" version="opset1">
 			<data destination_type="i32" />
 			<input>
 				<port id="0" precision="BOOL">
@@ -792,7 +630,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="55" name="Convert_185" type="Convert" version="opset1">
+		<layer id="41" name="Convert_62641" type="Convert" version="opset1">
 			<data destination_type="i64" />
 			<input>
 				<port id="0" precision="I32">
@@ -807,7 +645,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="57" name="RaggedToDense_184.0" type="Convert" version="opset1">
+		<layer id="43" name="RaggedToDense_62640.0" type="Convert" version="opset1">
 			<data destination_type="i64" />
 			<input>
 				<port id="0" precision="I32">
@@ -822,7 +660,7 @@
 				</port>
 			</output>
 		</layer>
-		<layer id="58" name="Result_188" type="Result" version="opset1">
+		<layer id="44" name="Result_62644" type="Result" version="opset1">
 			<input>
 				<port id="0" precision="I64">
 					<dim>-1</dim>
@@ -830,7 +668,7 @@
 				</port>
 			</input>
 		</layer>
-		<layer id="56" name="Result_190" type="Result" version="opset1">
+		<layer id="42" name="Result_62646" type="Result" version="opset1">
 			<input>
 				<port id="0" precision="I64">
 					<dim>-1</dim>
@@ -841,103 +679,86 @@
 	</layers>
 	<edges>
 		<edge from-layer="0" from-port="0" to-layer="2" to-port="0" />
-		<edge from-layer="1" from-port="0" to-layer="9" to-port="0" />
+		<edge from-layer="1" from-port="0" to-layer="8" to-port="0" />
 		<edge from-layer="2" from-port="1" to-layer="3" to-port="0" />
-		<edge from-layer="2" from-port="2" to-layer="3" to-port="1" />
-		<edge from-layer="2" from-port="3" to-layer="3" to-port="2" />
-		<edge from-layer="3" from-port="5" to-layer="16" to-port="4" />
-		<edge from-layer="3" from-port="4" to-layer="16" to-port="3" />
-		<edge from-layer="3" from-port="3" to-layer="16" to-port="2" />
-		<edge from-layer="3" from-port="3" to-layer="4" to-port="0" />
-		<edge from-layer="4" from-port="1" to-layer="7" to-port="0" />
-		<edge from-layer="5" from-port="0" to-layer="7" to-port="1" />
-		<edge from-layer="6" from-port="0" to-layer="7" to-port="2" />
-		<edge from-layer="7" from-port="3" to-layer="9" to-port="1" />
-		<edge from-layer="7" from-port="3" to-layer="12" to-port="0" />
-		<edge from-layer="8" from-port="0" to-layer="9" to-port="2" />
-		<edge from-layer="9" from-port="3" to-layer="16" to-port="0" />
-		<edge from-layer="10" from-port="0" to-layer="14" to-port="0" />
-		<edge from-layer="11" from-port="0" to-layer="12" to-port="1" />
-		<edge from-layer="12" from-port="2" to-layer="14" to-port="1" />
-		<edge from-layer="13" from-port="0" to-layer="14" to-port="2" />
-		<edge from-layer="14" from-port="3" to-layer="16" to-port="1" />
-		<edge from-layer="15" from-port="0" to-layer="16" to-port="5" />
-		<edge from-layer="16" from-port="6" to-layer="20" to-port="0" />
-		<edge from-layer="16" from-port="7" to-layer="20" to-port="1" />
-		<edge from-layer="16" from-port="8" to-layer="20" to-port="2" />
-		<edge from-layer="16" from-port="9" to-layer="20" to-port="3" />
-		<edge from-layer="16" from-port="10" to-layer="20" to-port="4" />
-		<edge from-layer="17" from-port="0" to-layer="20" to-port="5" />
-		<edge from-layer="18" from-port="0" to-layer="19" to-port="0" />
-		<edge from-layer="19" from-port="1" to-layer="20" to-port="6" />
-		<edge from-layer="19" from-port="2" to-layer="20" to-port="7" />
-		<edge from-layer="19" from-port="3" to-layer="20" to-port="8" />
-		<edge from-layer="20" from-port="9" to-layer="21" to-port="0" />
-		<edge from-layer="20" from-port="13" to-layer="21" to-port="4" />
-		<edge from-layer="20" from-port="12" to-layer="21" to-port="3" />
-		<edge from-layer="20" from-port="10" to-layer="21" to-port="1" />
-		<edge from-layer="20" from-port="11" to-layer="21" to-port="2" />
-		<edge from-layer="21" from-port="9" to-layer="42" to-port="4" />
-		<edge from-layer="21" from-port="8" to-layer="42" to-port="3" />
-		<edge from-layer="21" from-port="7" to-layer="42" to-port="2" />
-		<edge from-layer="21" from-port="6" to-layer="42" to-port="1" />
-		<edge from-layer="21" from-port="5" to-layer="42" to-port="0" />
-		<edge from-layer="22" from-port="0" to-layer="23" to-port="0" />
-		<edge from-layer="23" from-port="3" to-layer="42" to-port="7" />
-		<edge from-layer="23" from-port="2" to-layer="42" to-port="6" />
-		<edge from-layer="23" from-port="1" to-layer="42" to-port="5" />
-		<edge from-layer="24" from-port="0" to-layer="25" to-port="0" />
-		<edge from-layer="25" from-port="3" to-layer="42" to-port="10" />
-		<edge from-layer="25" from-port="2" to-layer="42" to-port="9" />
-		<edge from-layer="25" from-port="1" to-layer="42" to-port="8" />
-		<edge from-layer="26" from-port="0" to-layer="34" to-port="0" />
-		<edge from-layer="27" from-port="0" to-layer="28" to-port="0" />
-		<edge from-layer="28" from-port="2" to-layer="40" to-port="3" />
-		<edge from-layer="28" from-port="1" to-layer="29" to-port="0" />
-		<edge from-layer="28" from-port="3" to-layer="40" to-port="4" />
-		<edge from-layer="28" from-port="1" to-layer="40" to-port="2" />
-		<edge from-layer="29" from-port="1" to-layer="32" to-port="0" />
-		<edge from-layer="30" from-port="0" to-layer="32" to-port="1" />
-		<edge from-layer="31" from-port="0" to-layer="32" to-port="2" />
-		<edge from-layer="32" from-port="3" to-layer="34" to-port="1" />
-		<edge from-layer="32" from-port="3" to-layer="37" to-port="0" />
-		<edge from-layer="33" from-port="0" to-layer="34" to-port="2" />
-		<edge from-layer="34" from-port="3" to-layer="40" to-port="0" />
-		<edge from-layer="35" from-port="0" to-layer="39" to-port="0" />
+		<edge from-layer="2" from-port="3" to-layer="15" to-port="4" />
+		<edge from-layer="2" from-port="2" to-layer="15" to-port="3" />
+		<edge from-layer="2" from-port="1" to-layer="15" to-port="2" />
+		<edge from-layer="3" from-port="1" to-layer="6" to-port="0" />
+		<edge from-layer="4" from-port="0" to-layer="6" to-port="1" />
+		<edge from-layer="5" from-port="0" to-layer="6" to-port="2" />
+		<edge from-layer="6" from-port="3" to-layer="11" to-port="0" />
+		<edge from-layer="6" from-port="3" to-layer="8" to-port="1" />
+		<edge from-layer="7" from-port="0" to-layer="8" to-port="2" />
+		<edge from-layer="8" from-port="3" to-layer="15" to-port="0" />
+		<edge from-layer="9" from-port="0" to-layer="13" to-port="0" />
+		<edge from-layer="10" from-port="0" to-layer="11" to-port="1" />
+		<edge from-layer="11" from-port="2" to-layer="13" to-port="1" />
+		<edge from-layer="12" from-port="0" to-layer="13" to-port="2" />
+		<edge from-layer="13" from-port="3" to-layer="15" to-port="1" />
+		<edge from-layer="14" from-port="0" to-layer="15" to-port="5" />
+		<edge from-layer="15" from-port="9" to-layer="16" to-port="1" />
+		<edge from-layer="15" from-port="7" to-layer="18" to-port="1" />
+		<edge from-layer="15" from-port="6" to-layer="18" to-port="0" />
+		<edge from-layer="15" from-port="11" to-layer="16" to-port="3" />
+		<edge from-layer="15" from-port="10" to-layer="16" to-port="2" />
+		<edge from-layer="15" from-port="8" to-layer="16" to-port="0" />
+		<edge from-layer="16" from-port="4" to-layer="18" to-port="2" />
+		<edge from-layer="16" from-port="5" to-layer="18" to-port="3" />
+		<edge from-layer="16" from-port="6" to-layer="18" to-port="4" />
+		<edge from-layer="16" from-port="7" to-layer="18" to-port="5" />
+		<edge from-layer="17" from-port="0" to-layer="18" to-port="6" />
+		<edge from-layer="18" from-port="11" to-layer="28" to-port="4" />
+		<edge from-layer="18" from-port="10" to-layer="28" to-port="3" />
+		<edge from-layer="18" from-port="9" to-layer="28" to-port="2" />
+		<edge from-layer="18" from-port="8" to-layer="28" to-port="1" />
+		<edge from-layer="18" from-port="7" to-layer="28" to-port="0" />
+		<edge from-layer="19" from-port="0" to-layer="20" to-port="0" />
+		<edge from-layer="20" from-port="1" to-layer="28" to-port="5" />
+		<edge from-layer="20" from-port="2" to-layer="28" to-port="6" />
+		<edge from-layer="20" from-port="3" to-layer="28" to-port="7" />
+		<edge from-layer="21" from-port="0" to-layer="22" to-port="0" />
+		<edge from-layer="22" from-port="1" to-layer="28" to-port="8" />
+		<edge from-layer="22" from-port="2" to-layer="28" to-port="9" />
+		<edge from-layer="22" from-port="3" to-layer="28" to-port="10" />
+		<edge from-layer="23" from-port="0" to-layer="24" to-port="0" />
+		<edge from-layer="24" from-port="1" to-layer="28" to-port="11" />
+		<edge from-layer="24" from-port="2" to-layer="28" to-port="12" />
+		<edge from-layer="24" from-port="3" to-layer="28" to-port="13" />
+		<edge from-layer="25" from-port="0" to-layer="26" to-port="0" />
+		<edge from-layer="26" from-port="1" to-layer="28" to-port="14" />
+		<edge from-layer="26" from-port="2" to-layer="28" to-port="15" />
+		<edge from-layer="26" from-port="3" to-layer="28" to-port="16" />
+		<edge from-layer="27" from-port="0" to-layer="28" to-port="17" />
+		<edge from-layer="28" from-port="19" to-layer="29" to-port="0" />
+		<edge from-layer="28" from-port="18" to-layer="29" to-port="1" />
+		<edge from-layer="28" from-port="19" to-layer="32" to-port="0" />
+		<edge from-layer="28" from-port="20" to-layer="34" to-port="2" />
+		<edge from-layer="28" from-port="19" to-layer="34" to-port="1" />
+		<edge from-layer="29" from-port="2" to-layer="31" to-port="0" />
+		<edge from-layer="30" from-port="0" to-layer="31" to-port="1" />
+		<edge from-layer="31" from-port="2" to-layer="32" to-port="1" />
+		<edge from-layer="32" from-port="2" to-layer="34" to-port="0" />
+		<edge from-layer="33" from-port="0" to-layer="34" to-port="3" />
+		<edge from-layer="34" from-port="5" to-layer="39" to-port="1" />
+		<edge from-layer="34" from-port="6" to-layer="39" to-port="2" />
+		<edge from-layer="34" from-port="4" to-layer="39" to-port="0" />
+		<edge from-layer="34" from-port="4" to-layer="35" to-port="1" />
+		<edge from-layer="34" from-port="5" to-layer="35" to-port="0" />
+		<edge from-layer="35" from-port="2" to-layer="37" to-port="0" />
 		<edge from-layer="36" from-port="0" to-layer="37" to-port="1" />
-		<edge from-layer="37" from-port="2" to-layer="39" to-port="1" />
-		<edge from-layer="38" from-port="0" to-layer="39" to-port="2" />
-		<edge from-layer="39" from-port="3" to-layer="40" to-port="1" />
-		<edge from-layer="40" from-port="7" to-layer="42" to-port="11" />
-		<edge from-layer="40" from-port="8" to-layer="42" to-port="12" />
-		<edge from-layer="40" from-port="9" to-layer="42" to-port="13" />
-		<edge from-layer="41" from-port="0" to-layer="42" to-port="14" />
-		<edge from-layer="42" from-port="17" to-layer="48" to-port="2" />
-		<edge from-layer="42" from-port="15" to-layer="48" to-port="0" />
-		<edge from-layer="42" from-port="15" to-layer="46" to-port="0" />
-		<edge from-layer="42" from-port="15" to-layer="43" to-port="1" />
-		<edge from-layer="42" from-port="16" to-layer="43" to-port="0" />
-		<edge from-layer="43" from-port="2" to-layer="45" to-port="0" />
-		<edge from-layer="44" from-port="0" to-layer="45" to-port="1" />
-		<edge from-layer="45" from-port="2" to-layer="46" to-port="1" />
-		<edge from-layer="46" from-port="2" to-layer="48" to-port="1" />
-		<edge from-layer="47" from-port="0" to-layer="48" to-port="3" />
-		<edge from-layer="48" from-port="5" to-layer="49" to-port="0" />
-		<edge from-layer="48" from-port="4" to-layer="49" to-port="1" />
-		<edge from-layer="48" from-port="4" to-layer="53" to-port="0" />
-		<edge from-layer="48" from-port="5" to-layer="53" to-port="1" />
-		<edge from-layer="48" from-port="6" to-layer="53" to-port="2" />
-		<edge from-layer="49" from-port="2" to-layer="51" to-port="0" />
-		<edge from-layer="50" from-port="0" to-layer="51" to-port="1" />
-		<edge from-layer="51" from-port="2" to-layer="53" to-port="3" />
-		<edge from-layer="52" from-port="0" to-layer="53" to-port="4" />
-		<edge from-layer="53" from-port="6" to-layer="54" to-port="0" />
-		<edge from-layer="53" from-port="5" to-layer="57" to-port="0" />
-		<edge from-layer="54" from-port="1" to-layer="55" to-port="0" />
-		<edge from-layer="55" from-port="1" to-layer="56" to-port="0" />
-		<edge from-layer="57" from-port="1" to-layer="58" to-port="0" />
+		<edge from-layer="37" from-port="2" to-layer="39" to-port="3" />
+		<edge from-layer="38" from-port="0" to-layer="39" to-port="4" />
+		<edge from-layer="39" from-port="6" to-layer="40" to-port="0" />
+		<edge from-layer="39" from-port="5" to-layer="43" to-port="0" />
+		<edge from-layer="40" from-port="1" to-layer="41" to-port="0" />
+		<edge from-layer="41" from-port="1" to-layer="42" to-port="0" />
+		<edge from-layer="43" from-port="1" to-layer="44" to-port="0" />
 	</edges>
 	<rt_info>
+		<bos_token_id value="0" />
 		<eos_token_id value="0" />
+		<original_tokenizer_class value="&lt;class 'transformers.models.gpt_neox.tokenization_gpt_neox_fast.GPTNeoXTokenizerFast'>" />
+		<pad_token_id value="0" />
 	</rt_info>
 </net>

tokenizer_config.json

@@ -234,7 +234,7 @@
     "### Response:"
   ],
   "bos_token": "<|endoftext|>",
-  "clean_up_tokenization_spaces": true,
+  "clean_up_tokenization_spaces": false,
   "eos_token": "<|endoftext|>",
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "<|endoftext|>",