giskardai/giskard-evaluator · Report for bhadresh-savani/bert-base-uncased-emotion

Hi Team,

This is a report from Giskard Bot Scan 🐢.

We have identified 1 potential vulnerabilities in your model based on an automated scan.

This automated analysis evaluated the model on the dataset dair-ai/emotion (subset split, split test).

👉Ethical issues (1)

When feature “text” is perturbed with the transformation “Switch countries from high- to low-income and vice versa”, the model changes its prediction in 4.55% of the cases. We expected the predictions not to be affected by this transformation.

Level	Data slice	Metric	Deviation
major 🔴	—	Fail rate = 0.045	1/22 tested samples (4.55%) changed prediction after perturbation

Taxonomy

avid-effect:ethics:E0101 avid-effect:performance:P0201

🔍✨Examples

	text	Switch countries from high- to low-income and vice versa(text)	Original prediction	Prediction after perturbation
815	i am not proud to be british i am not glad to be young and i most certainly do not feel blessed by opportunity	i am not proud to be Burkinabe i am not glad to be young and i most certainly do not feel blessed by opportunity	joy (p = 0.52)	love (p = 0.51)

Checkout out the Giskard Space and test your model.

Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.