README for Pathway Vision Transformer

PaViT is a Pathway Vision Transformer (PaViT)-based image recognition model developed by Ajibola Emmanuel Oluwaseun. The model is inspired by Google's PaLM (Pathways Language Model) and aims to demonstrate the potential of using few-shot learning techniques in image recognition tasks.

Model Performance

PaViT was trained on a 4GB RAM CPU using a dataset of 15000 Kaggle images of 15 classes, achieving a remarkable 88% accuracy with 4 self-attention heads. The model's accuracy further improved to 96% when trained with 12 self-attention heads and 12 linearly stacked linear layers. These results demonstrate the model's impressive performance and fast training speed on a CPU, despite being trained on a relatively small dataset.
The uploaded weight was trained on image dataset of 3 classes (Cat, Dog and Wild animal)

Usage

The model can be used for image recognition tasks by using the trained weights provided in the repository. The code can be modified to use custom datasets, and the model's performance can be further improved by adding more self-attention heads and linear layers.

Contribution

The author believes that PaViT has the potential to outperform existing Vision Transformer models and is eager to see it continue to evolve through the contributions of developers and other contributors.

Contributions to the project are welcome and can be made through pull requests. Developers can also report issues or suggest new features for the project.

License

This project is licensed under the MIT License.

How to use:

#import Libraries
!pip install huggingface_hub["tensorflow"]
import matplotlib.pyplot as plt
import cv2
from huggingface_hub import from_pretrained_keras

On inference

#load model
model=from_pretrained_keras('Ajibola/PaViT')
#load image
image=cv2.imread('image_path')
image=cv2.resize(image, (224, 224)) #224 is the default image size
image=image.image.max() #Normalize the image to [0-1]
prediction=model.predict(image)
prediction=np.argmax(prediction, axis=-1) #Get Highest probability class