infinity usage of reranking. Implements a cohere compatible api. (#10)

- infinity usage of reranking. Implements a cohere compatible api. (231c626908c606a13aa1f64400ca9a99e03f2ac0)


Co-authored-by: Michael <[email protected]>

README.md

@@ -126,6 +126,14 @@ with torch.no_grad():
 # tensor([1.2315, 0.5923, 0.3041])
 ```
 
+Usage with infinity:
+
+[Infinity](https://github.com/michaelfeil/infinity), a MIT Licensed Inference RestAPI Server.
+```
+docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
+michaelf34/infinity:0.0.68 \
+v2 --model-id Alibaba-NLP/gte-multilingual-reranker-base --revision "main" --dtype bfloat16 --batch-size 32 --device cuda --engine torch --port 7997
+```
 
 ## Evaluation