---
license: apache-2.0
---
# MonoQwen2-VL-2B-LoRA-Reranker

## Model Overview
The **MonoQwen2-VL-2B-LoRA-Reranker** is a fine-tuned version of the Qwen2-VL-2B model, optimized for reranking image-query relevance.

## How to Use the Model
Below is a quick example to rerank a single image against a user query using this model:

```python
import torch
from PIL import Image
from transformers import AutoProcessor, Qwen2VLForConditionalGeneration

# Load processor and model
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")
model = Qwen2VLForConditionalGeneration.from_pretrained("lightonai/MonoQwen2-VL-2B-LoRA-Reranker")

# Define the query and the image
query = "What is the value of the thing in the document"
image = Image.open("path_to_image.jpg")

# Prepare the inputs
prompt = f"Assert the relevance of the previous image document to the following query, answer True or False. The query is: {query}"
inputs = processor(text=prompt, images=image, return_tensors="pt")

# Run the model and obtain results
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    logits_for_last_token = logits[:, -1, :]
    true_token_id = processor.tokenizer.convert_tokens_to_ids("True")
    false_token_id = processor.tokenizer.convert_tokens_to_ids("False")
    relevance_score = torch.softmax(logits_for_last_token[:, [true_token_id, false_token_id]], dim=-1)

# Print the True/False probabilities
true_prob = relevance_score[:, 0].item()
false_prob = relevance_score[:, 1].item()

print(f"True probability: {true_prob}, False probability: {false_prob}")
```

This example demonstrates how to use the model to assess the relevance of an image with respect to a query. It outputs the probability that the image is relevant ("True") or not relevant ("False").

## Performance Metrics

The model has been evaluated on [ViDoRe Benchmark](https://huggingface.co/spaces/vidore/vidore-leaderboard), by retrieving 10 elements with [MrLight_dse-qwen2-2b-mrl-v1](https://huggingface.co/MrLight/dse-qwen2-2b-mrl-v1) and reranking them. The table below summarizes its `ndcg@5` scores:

| Dataset                                           | NDCG@5 Before Reranking  | NDCG@5 After Reranking |
|---------------------------------------------------|--------------------------|------------------------|
| **Mean**                                          | 87.6                     | **91.8**               |
| vidore/arxivqa_test_subsampled                    | 85.6                     | 89.01                  |
| vidore/docvqa_test_subsampled                     | 57.1                     | 59.71                  |
| vidore/infovqa_test_subsampled                    | 88.1                     | 93.49                  |
| vidore/tabfquad_test_subsampled                   | 93.1                     | 95.96                  |
| vidore/shiftproject_test                          | 82.0                     | 92.98                  |
| vidore/syntheticDocQA_artificial_intelligence_test| 97.5                     | 100.00                 |
| vidore/syntheticDocQA_energy_test                 | 92.9                     | 97.65                  |
| vidore/syntheticDocQA_government_reports_test     | 96.0                     | 98.04                  |
| vidore/syntheticDocQA_healthcare_industry_test    | 96.4                     | 99.27                  |




## License

This LoRA model is licensed under the Apache 2.0 license.