FaceScanPaliGemma_Gender

from PIL import Image
import torch
from transformers import PaliGemmaProcessor, PaliGemmaForConditionalGeneration, BitsAndBytesConfig, TrainingArguments, Trainer


model = PaliGemmaForConditionalGeneration.from_pretrained('NYUAD-ComNets/FaceScanPaliGemma_Gender',torch_dtype=torch.bfloat16)

input_text = "what is the gender of the person in the image?"

processor = PaliGemmaProcessor.from_pretrained("google/paligemma-3b-pt-224")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model.to(device)


input_image = Image.open('image_path')
inputs = processor(text=input_text, images=input_image, padding="longest", do_convert_rgb=True, return_tensors="pt").to(device)
inputs = inputs.to(dtype=model.dtype)
      
with torch.no_grad():
          output = model.generate(**inputs, max_length=500)
result=processor.decode(output[0], skip_special_tokens=True)[len(input_text):].strip()

Model description

This model is a fine-tuned version of google/paligemma-3b-pt-224 on the FairFace dataset. The model aims to classify the gender of face image or image with one person into Male and Female

Model Performance

Accuracy: 95.8 %, F1 score: 96 %

Intended uses & limitations

This model is used for research purposes

Training and evaluation data

FairFace dataset was used for training and validating the model

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• num_epochs: 5

Results

The model has an accuracy of %

Framework versions

• Transformers 4.42.4
• Pytorch 2.1.2+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1

BibTeX entry and citation info

@article{aldahoul2024exploring,
  title={Exploring Vision Language Models for Facial Attribute Recognition: Emotion, Race, Gender, and Age},
  author={AlDahoul, Nouar and Tan, Myles Joshua Toledo and Kasireddy, Harishwar Reddy and Zaki, Yasir},
  journal={arXiv preprint arXiv:2410.24148},
  year={2024}
}

@misc{ComNets,
      url={https://huggingface.co/NYUAD-ComNets/FaceScanPaliGemma_Gender](https://huggingface.co/NYUAD-ComNets/FaceScanPaliGemma_Gender)},
      title={FaceScanPaliGemma_Gender},
      author={Nouar AlDahoul, Yasir Zaki}
}