Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,8 @@ We notice that, 1-bit quantization doesn't work well when applied directly on sm
|
|
| 15 |
Note that the weights here are unsigned 1-bit (0 or 1), <a href="https://arxiv.org/abs/2402.17764">not ternary like the recent 1.58-bit work </a>. This is a more challenging task since we lose the sign of the weights and only fine-tune a small fraction of the parameters (~94MB worth of weights).
|
| 16 |
The dequantization step can be rewriten as a 1-bit matmul which could potentially require only additions + a very low-rank matmul which is fast to compute.
|
| 17 |
|
|
|
|
|
|
|
| 18 |
## Datasets
|
| 19 |
The adapter was trained via SFT on random subsets of the following:
|
| 20 |
|
|
|
|
| 15 |
Note that the weights here are unsigned 1-bit (0 or 1), <a href="https://arxiv.org/abs/2402.17764">not ternary like the recent 1.58-bit work </a>. This is a more challenging task since we lose the sign of the weights and only fine-tune a small fraction of the parameters (~94MB worth of weights).
|
| 16 |
The dequantization step can be rewriten as a 1-bit matmul which could potentially require only additions + a very low-rank matmul which is fast to compute.
|
| 17 |
|
| 18 |
+
This versions offloads the meta-data to the CPU, so only the binary weights and the low-rank adapters are stored in the GPU memory.
|
| 19 |
+
|
| 20 |
## Datasets
|
| 21 |
The adapter was trained via SFT on random subsets of the following:
|
| 22 |
|