beomi (Lee Junbum)

upvoted an article 5 days ago

Article

Efficient Deep Learning: A Comprehensive Overview of Optimization Techniques 👐 📚

By

•

Aug 26

• 34

upvoted an article 18 days ago

Article

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

Sep 18

• 198

upvoted a paper 23 days ago

Addition is All You Need for Energy-efficient Language Models

Paper • 2410.00907 • Published Oct 1 • 143

upvoted an article 24 days ago

Article

Fixing Gradient Accumulation

25 days ago

• 39

upvoted a paper 26 days ago

StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization

Paper • 2410.08815 • Published 30 days ago • 41

upvoted a paper 27 days ago

SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights

Paper • 2410.09008 • Published 30 days ago • 16

upvoted a paper about 1 month ago

Differential Transformer

Paper • 2410.05258 • Published Oct 7 • 165

upvoted a collection about 1 month ago

Moshi v0.1 Release

Collection

MLX, Candle & PyTorch model checkpoints released as part of the Moshi release from Kyutai. Run inference via: https://github.com/kyutai-labs/moshi • 13 items • Updated Sep 18 • 215

upvoted a paper about 2 months ago

Training Language Models to Self-Correct via Reinforcement Learning

Paper • 2409.12917 • Published Sep 19 • 134

upvoted a collection about 2 months ago

DataGemma Release

Collection

A series of pioneering open models that help ground LLMs in real-world data through Data Commons. • 2 items • Updated Sep 12 • 77

upvoted a paper 2 months ago

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Paper • 2406.14491 • Published Jun 20 • 85

upvoted a paper 3 months ago

To Code, or Not To Code? Exploring Impact of Code in Pre-training

Paper • 2408.10914 • Published Aug 20 • 40

upvoted an article 3 months ago

Article

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 65

upvoted 7 papers 4 months ago

The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism

Paper • 2407.10457 • Published Jul 15 • 22

Lee Junbum PRO

AI & ML interests

Organizations

beomi's activity

Efficient Deep Learning: A Comprehensive Overview of Optimization Techniques 👐 📚

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

Addition is All You Need for Energy-efficient Language Models

Fixing Gradient Accumulation

StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization

SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights

Differential Transformer

Moshi v0.1 Release

Training Language Models to Self-Correct via Reinforcement Learning

DataGemma Release

Instruction Pre-Training: Language Models are Supervised Multitask Learners

To Code, or Not To Code? Exploring Impact of Code in Pre-training

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism

GRUtopia: Dream General Robots in a City at Scale

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated

Learning to Refuse: Towards Mitigating Privacy Risks in LLMs

Qwen2 Technical Report

Better & Faster Large Language Models via Multi-token Prediction

Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems