Spaces:
Running
on
CPU Upgrade
Is there a way to download CSV?
Is there a way to download a CSV of results? I want to create a custom ranking based on specific datasets of interest eg medical. I didn't see a good way to download raw data. Thanks!
Hi cheebw1, all results can be found here. We are also currently working on developing code to more easily access the results. In a future version of mteb it will be available as something like:
# this is not yet implemented
import mteb
results = mteb.load_results() # downloads and loads in results using MTEBResults
# format will be:
results: dict[MODEL_NAME_STR, dict[REVISION_STR, list[MTEBResult]]
We will likely also add utilities for converting this format to e.g. a data frame
Adding to @KennethEnevoldsen great reply, we also plan to change the leaderboard soon to look something like this: https://github.com/embeddings-benchmark/mteb/discussions/674#discussioncomment-9404987 which would allow you to create custom rankings based on specific datasets. Help on this is very welcome if you are interested :)