Commit
•
871351a
1
Parent(s):
c4fbc17
Update README.md (#5)
Browse files- Update README.md (38967b8d0952dd7ebbc5634f8933bc626773d5b7)
Co-authored-by: Vaibhav Srivastav <[email protected]>
README.md
CHANGED
|
@@ -424,6 +424,8 @@ Once a valid PyTorch version is installed, SDPA is activated by default. It can
|
|
| 424 |
+ model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True, attn_implementation="sdpa")
|
| 425 |
```
|
| 426 |
|
|
|
|
|
|
|
| 427 |
#### Torch compile
|
| 428 |
|
| 429 |
Coming soon...
|
|
|
|
| 424 |
+ model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True, attn_implementation="sdpa")
|
| 425 |
```
|
| 426 |
|
| 427 |
+
For more information about how to use the SDPA refer to the [Transformers SDPA documentation](https://huggingface.co/docs/transformers/en/perf_infer_gpu_one#pytorch-scaled-dot-product-attention).
|
| 428 |
+
|
| 429 |
#### Torch compile
|
| 430 |
|
| 431 |
Coming soon...
|