How to increase or decrease the context length?

by CouchCommander - opened May 3

Discussion

CouchCommander

May 3

I am trying to run my model with dynamic context length but it is not specified how to configure it.

Renu11

Google org Sep 17

•

edited Sep 17

Hi @CouchCommander , You can adjust the context length of the inputs using max_length or max_new_tokens in outputs = model.generate(**input_ids, max_length=100)or while instantiating the model.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment