Mozilla
/

granite-34b-code-instruct-llamafile

Text Generation

Inference Endpoints

Model card Files Files and versions Community

jartine commited on May 26

Commit

7a5f8e0

•

1 Parent(s): deada0d

Update README.md

Files changed (1) hide show

README.md +6 -4

README.md CHANGED Viewed

@@ -272,12 +272,14 @@ It can be changed, e.g. `--temp 0.8`.
 | hardware                                   | model\_filename                          | size       | test          | t/s             |
 | :----------------------------------------- | :--------------------------------------- | ---------: | ------------: | --------------: |
-| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | pp512         | 186.14          |
-| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | tg16          | 14.13           |
-| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | pp512         | 94.34           |
-| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | tg16          | 5.61            |
 | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | pp512         | 95.08           |
 | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | tg16          | 7.78            |
 ## About Quantization

 | hardware                                   | model\_filename                          | size       | test          | t/s             |
 | :----------------------------------------- | :--------------------------------------- | ---------: | ------------: | --------------: |
+| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | pp512         | 159.02          |
+| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | tg16          | 15.39           |
+| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | pp512         | 186.14          |
+| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | tg16          | 14.13           |
 | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | pp512         | 95.08           |
 | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | tg16          | 7.78            |
+| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | pp512         | 94.34           |
+| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | tg16          | 5.61            |
 ## About Quantization