performance
#8
by
rautsanket4086
- opened
Why this model response takes too much time
i am using Params: {'model': 'models/mistral-7b-instruct-v0.1.Q8_0.gguf', 'model_type': 'mistral', 'model_file': None, 'config': None}
why this takes 10 to 12 mins for execution
How can i decrease time for execution
rautsanket4086
changed discussion status to
closed
rautsanket4086
changed discussion status to
open
Anyone know this
Use other model, like de Q4_K_M. This would improve a lot the speed