Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
Paper
•
2404.05719
•
Published
•
80
Research on GUI Models
-
-
OS-ATLAS: A Foundation Action Model for Generalist GUI Agents
Paper • 2410.23218 • Published • 43 -
ScreenAI: A Vision-Language Model for UI and Infographics Understanding
Paper • 2402.04615 • Published • 38 -
CogAgent: A Visual Language Model for GUI Agents
Paper • 2312.08914 • Published • 29 -
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
Paper • 2401.10935 • Published • 4 -
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Paper • 2210.03347 • Published • 3 -
ScreenAgent: A Vision Language Model-driven Computer Control Agent
Paper • 2402.07945 • Published -
From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces
Paper • 2306.00245 • Published -
GUI-WORLD: A Dataset for GUI-oriented Multimodal LLM-based Agents
Paper • 2406.10819 • Published -
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
Paper • 2209.08199 • Published