xai-org

/

grok-1

like 2.17k

Follow

xAI 349

Some fun fact: the model which is not using huggingface's API/library (like transformers/diffusers/timm/datasets) will not track the # of download iirc.

that's not entirely true. It's more about what it is that you are downloading rather than how. Here is a list of what would count (must be downloaded) depending on the library:

{
    "adapter-transformers": {
        filter: [
            {
                term: { path: "adapter_config.json" },
            },
        ],
    },
    "asteroid": {
        filter: [
            {
                term: { path: "pytorch_model.bin" },
            },
        ],
    },
    "flair": {
        filter: [
            {
                term: { path: "pytorch_model.bin" },
            },
        ],
    },
    "keras": {
        filter: [
            {
                term: { path: "saved_model.pb" },
            },
        ],
    },
    "ml-agents": {
        filter: [
            {
                wildcard: { path: "*.onnx" },
            },
        ],
    },
    "nemo": {
        filter: [
            {
                wildcard: { path: "*.nemo" },
            },
        ],
    },
    "open_clip": {
        filter: [
            {
                wildcard: { path: "*pytorch_model.bin" },
            },
        ],
    },
    "sample-factory": {
        filter: [
            {
                term: { path: "cfg.json" },
            },
        ],
    },
    "paddlenlp": {
        filter: [
            {
                term: { path: "model_config.json" },
            },
        ],
    },
    "speechbrain": {
        filter: [
            {
                term: { path: "hyperparams.yaml" },
            },
        ],
    },
    "sklearn": {
        filter: [
            {
                term: { path: "sklearn_model.joblib" },
            },
        ],
    },
    "spacy": {
        filter: [
            {
                wildcard: { path: "*.whl" },
            },
        ],
    },
    "stanza": {
        filter: [
            {
                term: { path: "models/default.zip" },
            },
        ],
    },
    "stable-baselines3": {
        filter: [
            {
                wildcard: { path: "*.zip" },
            },
        ],
    },
    "timm": {
        filter: [
            {
                terms: { path: ["pytorch_model.bin", "model.safetensors"] },
            },
        ],
    },
    "diffusers": {
        /// Filter out nested safetensors and pickle weights to avoid double counting downloads from the diffusers lib
        must_not: [
            {
                wildcard: { path: "*/*.safetensors" },
            },
            {
                wildcard: { path: "*/*.bin" },
            },
        ],
        /// Include documents that match at least one of the following rules
        should: [
            /// Downloaded from diffusers lib
            {
                term: { path: "model_index.json" },
            },
            /// Direct downloads (LoRa, Auto1111 and others)
            {
                wildcard: { path: "*.safetensors" },
            },
            {
                wildcard: { path: "*.ckpt" },
            },
            {
                wildcard: { path: "*.bin" },
            },
        ],
        minimum_should_match: 1,
    },
    "peft": {
        filter: [
            {
                term: { path: "adapter_config.json" },
            },
        ],
    }
}

That said, this model's card is just a repository that contains some checkpoints. I am not sure what should be counted as a download. Once there is a way to do from_pretrained() then it should be counted as a download.

So the counter is inaccurate. I don't have the hardware for this anyway.

cc @osanseviero - we recently added download metrics for Grok as seen here: https://github.com/huggingface/huggingface.js/pull/561.

Yes, we just did the deployment and the stats should look there tomorrow or wednesday

Thank you both @nielsr and @osanseviero (I wish we could get some love for GGUF models as well 😊)

Yes, we're working in automatically adding download counts for all GGUF in the coming weeks!

BTW we would need to update this doc page: https://huggingface.co/docs/hub/en/models-download-stats to link to the actual up-to-date definition of per-library download stats rules, which is defined here:

https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/src/model-libraries.ts

The download metrics are there now btw, 31k downloads so far