README

languages

Merge branch 'main' of hf.co:tangledgroup/tangled-llama-v-128k-base-v0.1

benchmark

model and eval

config.json

Merge branch 'main' of hf.co:tangledgroup/tangled-llama-v-128k-base-v0.1

pretrain/final

config.json

pretrain model, extend from 5 to 8 epochs

pretrain mode

pretrain mode

pretrain mode

general pretrain data generation

general pretrain data generation

pretrain dataset

pretrain dataset

pretrain dataset

pretrain dataset

pretrain dataset

pretrain dataset

pretrain dataset

new tokenizer 38400

fixed smaller pretrain dataset

smaller pretrain dataset

pretrain model

pretrain model

pretrain model

pretrain model

pretrain model

new tokenizer 38400

trained new 128k tokenizer

git config

initial commit