Model Card for ADELIE-DPO

We introduce ADELIE (Aligning large language moDELs on Information Extraction), an aligned LLM that effectively solves various IE tasks, including closed IE, open IE, and on-demand IE. We first collect and construct a high-quality alignment corpus IEInstruct for IE. Then we train ADELIE_SFT using instruction tuning on IEInstruct. We further train ADELIE_SFT with direct preference optimization (DPO) objective, resulting in ADELIE_DPO. Extensive experiments on various held-out IE datasets demonstrate that our models (ADELIE_SFT and ADELIE_DPO) achieve state-of-the-art (SoTA) performance among open-source models. We further explore the general capabilities of ADELIE, and experimental results reveal that their general capabilities do not exhibit a noticeable decline.

• 📖 Paper: ADELIE: Aligning Large Language Models on Information Extraction

• 🐧 Github: THU/ADELIE

Model Performance

The table below presents the average F1 scores (%) of the ADELIE model across closed IE, open IE, and on-demand IE tasks, as well as its overall performance (%) on general benchmarks. For dataset details, please refer to the paper.

Model	Closed IE	Open IE	On-demand IE	General Average Score
Llama2 7B	5.7	5.6	22.4	52.2
ADELIE-SFT	42.6	46.9	60.4	53.5
ADELIE-DPO	42.7	47.6	60.5	53.8
-----------------	-----------	---------	--------------	-----------------------
Llama3.2 3B	19.1	18.5	20.8	55.5
ADELIE-SFT-3B	41.8	47.6	60.8	55.6
ADELIE-DPO-3B	39.2	47.8	60.7	55.6
-----------------	-----------	---------	--------------	-----------------------
Qwen2.5 1.5B	16.5	14.2	20.5	54.6
ADELIE-SFT-1.5B	37.7	44.6	58.9	55.0
ADELIE-DPO-1.5B	38.5	45.6	59.2	55.1

Model Description

• Developed by: Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
• Model type: Text Generation
• Language(s) (NLP): English
• License: LLaMA2 License for the base model.
• Finetuned from model [optional]: LLaMA2-7B