`Running Evaluation Queue` seems to have gotten stuck on a model.

#268
by xzuyn - opened

dhmeltzer/llama-7b-SFT-qlora-eli5-wiki_DPO_ds_RM_top_2_1024_r_64_alpha_16 has been sitting alone in the Running Evaluation Queue for hours, while the Pending Evaluation Queue has crept up slowly all the way up to 109 models right now.

Yeah, something's wrong here. I guess the machine responsible for evaluating the models has an error?

Open LLM Leaderboard org

Hi ! Evaluations cannot be "stuck" on a model, as it is simply launching them and then the front end is responsible for gathering the results. However, your model did stay in the RUNNING state because we stopped running evaluation to be able to add new features. Since the backend was not running, the status of the model you mentioned was not changed, even though it did get evaluated and is even present in the leadeboard.

clefourrier changed discussion status to closed

Sign up or log in to comment