MiRAGeNews: Multimodal Realistic AI-Generated News Detection
liamdugan
Abstract
The proliferation of inflammatory or misleading "fake" news content has become increasingly common in recent years. Simultaneously, it has become easier than ever to use AI tools to generate photorealistic images depicting any scene imaginable. Combining these two -- AI-generated fake news content -- is particularly potent and dangerous. To combat the spread of AI-generated fake news, we propose the MiRAGeNews Dataset, a dataset of 12,500 high-quality real and AI-generated image-caption pairs from state-of-the-art generators. We find that our dataset poses a significant challenge to humans (60% F-1) and state-of-the-art multi-modal LLMs (< 24% F-1). Using our dataset we train a multi-modal detector (MiRAGe) that improves by +5.1% F-1 over state-of-the-art baselines on image-caption pairs from out-of-domain image generators and news publishers. We release our code and data to aid future work on detecting AI-generated content.
Community
New paper from UPenn releasing a dataset of high quality generated news images from Midjourney with associated captions. News articles are based on real NYT news from TARA and are prompted by GPT-4 to be more inflammatory or controversial. Work shows that humans struggle to detect these images and that SOTA VLMs also struggle to detect these images zero-shot and proposes the MiRAGe detector that exhibits good cross-generator generalization performance.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection (2024)
- CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection (2024)
- Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs (2024)
- Detect Fake with Fake: Leveraging Synthetic Data-driven Representation for Synthetic Image Detection (2024)
- FIDAVL: Fake Image Detection and Attribution using Vision-Language Model (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper