Train a Llama model from scratch
•
39
Noa Roggendorff
nroggendorff
AI & ML interests
Large language models, Illusion Diffusion, Stable Diffusion, Stable Video Diffusion
Articles
Organizations
nroggendorff's activity
replied to
their
post
1 day ago
How do you know ?
I suppose I don't
replied to
their
post
1 day ago
pretty sure
replied to
their
post
9 days ago
I wasn't hacked, I got flagged for cryptomining for some reason.
replied to
their
post
about 1 month ago
for the millionth time
posted
an
update
about 2 months ago
Post
4068
Datasets are down, I offer a solution
git lfs installgit clone https://huggingface.co/datasets/{dataset/id}from datasets import load_dataset
dataset = load_dataset("id")
posted
an
update
2 months ago
Post
545
What could have been
"{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '<|user|>\n' + message['content'] + '<|end|>\n' }}{% elif message['role'] == 'assistant' %}{{ '<|bot|>\n' + message['content'] }}{% if loop.last %}{{ '<|end|>' + eos_token }}{% else %}{{ '<|end|>\n' }}{% endif %}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
"{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '<|user|>\n' + message['content'] + '<|end|>\n' }}{% elif message['role'] == 'assistant' %}{{ '<|bot|>\n' + message['content'] }}{% if loop.last %}{{ '<|end|>' + eos_token }}{% else %}{{ '<|end|>\n' }}{% endif %}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
- 1 reply
we know, stop bragging
not that im jealous or anything
jk congrats
replied to
their
post
2 months ago
Im pretty sure you can add additional tokens and special tokens, so I suppose
but that doesnt really have anything to do with this article, thats just a transformers feature
replied to
their
post
2 months ago
Im pretty sure you can add additional tokens and special tokens, so I suppose
replied to
their
post
2 months ago
posted
an
update
2 months ago
Post
2045
Updated https://huggingface.co/blog/nroggendorff/train-with-llama-architecture so you can "train" your own tokenizer from your dataset.
-
- 6 replies
replied to
their
post
2 months ago
posted
an
update
2 months ago
Post
579
I updated
nroggendorff/llava to be even worse. The point of this space is to point out how bad the base model for Llava is, and how without an image it struggles quite a bit. In the new update, from my testing, even if you ask the model about the image, it won't be able to tell you. I experimented with quite a few things, including an image where all the values are 0 (shoutout np.zeros), an image with the most generic portrait photo I could think of (black hair, brown eyes, plain white shirt, etc..) (generated with
nroggendorff/epicrealismxl), an image that has text that reads "The image you are looking for is unavailable", and much.. much more.. *sigh*. The image I'm ending up with seems to work best. I don't think it would work with other multi-modal LLMs, but I'm not sure. If you find a problem, feel free to message me on the [huggingface discord](http://hf.co/join/discord), or open a pull request.
-
-
- 2 replies
posted
an
update
3 months ago
Post
729
I changed my mind about kaggle nb versions. They're goated
replied to
their
post
3 months ago
laughs in overpowered
replied to
their
post
3 months ago
indeed
replied to
their
post
3 months ago
I mean like replacing the binary data with decimals, so I could store a message
for example,
The letter "J" would be 74
replied to
their
post
3 months ago
You don't either?
@nroggendorff wth, i do on my end D:
not on your profile
replied to
their
post
3 months ago
hmm
replied to
their
post
3 months ago
we seek that fucking delete form of any subscribing to your platorm ; it is not ficking possible make it so ; because CEO I'll fuck you up personaly big time if you can't fucking get real
What is wrong with you
replied to
their
post
3 months ago
where is the delete account button ? we d\wish not to read hiilbillies
what??
replied to
their
post
3 months ago
You don't either?
replied to
their
post
3 months ago
ratio
posted
an
update
3 months ago
Post
2477
I'm about to start storing files in my TFLOPS count..
-
- 38 replies
replied to
their
post
3 months ago
Thanks
posted
an
update
3 months ago
Post
3317
I wish I had a cheaper and more appreciated by the masses hobby.
- 7 replies
replied to
their
post
4 months ago
It's not very good, but it's funny.