Spaces:

upstage

/

open-ko-llm-leaderboard

like 474

Running on CPU Upgrade

Hello. The Mistral model is also supported.

However, during the recent server changing process, several Mistral evaluations were not conducted, but i will fixed it to ensure all are re-evaluated until 11/5

Thank you for reporting the error.

Now, it fixed. Thanks

Hello..!

Could you find 'kyujinpy/Korean-OpenOrca-v3' model?
It finished the evaluation, but there is not in model list.

Thank you.

Hello kyujinpy.
Your model (kyujinpy/Korean-OpenOrca-v3) dosen't finished evaluation. [Link]

Can you check if there are any issues when running the code below?

from transformers import AutoConfig, AutoModel, AutoTokenizer
config = AutoConfig.from_pretrained("your model name", revision=revision)
model = AutoModel.from_pretrained("your model name", revision=revision)
tokenizer = AutoTokenizer.from_pretrained("your model name", revision=revision)

The actual error occurred when loading above code.
I've seen in the model commit log that there's a record of the relevant part being modified. If the issue has been resolved, please resubmit it. Then, it seems it will be evaluated. (If you're unable to resubmit or if it still fails, please leave a message.)

Thank you.

안녕하세요.
모델 자체는 평가가 되지 않았습니다. [Link]

모델의 평가가 되려면 위의 코드가 무사히 불러와지고, lm-eval-harness 까지 문제가 없어야합니다. 평가의 경우는 위의 코드를 불러오는 과정에서 실패했습니다.
모델 커밋 로그를 보니 파일명에 대한 변경이 있었던 것 같은데 실패원인도 해당 부분에 대한 오류로 보였습니다. 만약, 해당 오류가 수정된거라면 다시 제출을 해주시면 평가가 될걸로 보입니다. (만약, 그럼에도 제출이 다시 안되거나 오류가 생긴다면 다시 글을 남겨주시면 도움 드리도록 하겠습니다)
감사합니다.

Thank you for your comment!

But, when I try to re-submit, 'This model has been already submitted.' message maintains.
How can I remove above message?

Thank you.

Hello, I have resubmitted the model.
I will need to check the cause of the issue you've mentioned. Thank you for the report.

Thank you.

It solved..!
Thank you for your contribution.