Post
836
๐๐ง๐๐ฉ๐๐ฃ๐ ๐ฉ๐ค๐ค๐ก ๐๐๐ก๐ก๐จ ๐๐ฃ ๐๐ค๐๐ ๐๐ช๐จ๐ฉ ๐ฌ๐ค๐ง๐ ๐จ ๐๐๐ฉ๐ฉ๐๐ง ๐ฉ๐๐๐ฃ ๐
๐๐๐ ๐ช
I was really happy to learn today by @sergeipetrov that paper ๐๐น๐ฆ๐ค๐ถ๐ต๐ข๐ฃ๐ญ๐ฆ ๐๐ฐ๐ฅ๐ฆ ๐๐ค๐ต๐ช๐ฐ๐ฏ๐ด ๐๐ญ๐ช๐ค๐ช๐ต ๐๐ฆ๐ต๐ต๐ฆ๐ณ ๐๐๐ ๐๐จ๐ฆ๐ฏ๐ต๐ด was accepted at ICLR 2024!
As a reminder, an agent is a system in which you embed a LLM engine, to let it call tools.
These tools are meant like an IronMan suit, to supplement the LLM in areas that it isn't good at.
๐งโ๐ป For instance your friendly LLM may be terrible at calculating powers of floating numbers ("What is X ^0.2947 ?"), so it should use a calculator.
๐It may be terrible at knowing precise facts ("What was the date of the Golden Bull?") so it should use a web browser.
So the agent system will prompt an agent with "Now you can use these tools: calculator, search,..."
But ๐๐ค๐ฌ ๐จ๐๐ค๐ช๐ก๐ ๐ฉ๐๐ ๐๐๐๐ฃ๐ฉ ๐๐ญ๐ฅ๐ง๐๐จ๐จ ๐๐ฉ๐จ ๐๐๐ฉ๐๐ค๐ฃ๐จ?
All well known frameworks let agents write their actions as JSON strings.
We ๐ฝ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฟ๐ฒ๐ฑ ๐๐ผ ๐ด๐ผ ๐๐ถ๐๐ต ๐ณ๐ผ๐ฟ๐บ๐๐น๐ฎ๐๐ถ๐ป๐ด ๐ฎ๐ฐ๐๐ถ๐ผ๐ป๐ ๐ถ๐ป ๐๐ผ๐ฑ๐ฒ, ๐๐ต๐ถ๐ฐ๐ต ๐ถ๐ ๐บ๐๐ฐ๐ต ๐บ๐ผ๐ฟ๐ฒ ๐๐ฒ๐ฟ๐๐ฎ๐๐ถ๐น๐ฒ ๐ฎ๐ป๐ฑ ๐ฐ๐ผ๐ป๐ฐ๐ถ๐๐ฒ, ๐ฎ๐ป๐ฑ ๐ฎ๐น๐น๐ผ๐๐ ๐๐ผ ๐ฐ๐ต๐ฎ๐ถ๐ป ๐ฎ๐ฐ๐๐ถ๐ผ๐ป๐ ๐๐ฒ๐ฎ๐บ๐น๐ฒ๐๐๐น๐: see the picture attached for an example where Code formulation really shines.
And the paper confirms our choice: researchers show that ๐ฐ๐ผ๐บ๐ฝ๐ฎ๐ฟ๐ฒ๐ฑ ๐๐ผ ๐๐ฆ๐ข๐ก ๐ผ๐ฟ ๐ฝ๐น๐ฎ๐ถ๐ป ๐๐ฒ๐ ๐, ๐๐ผ๐ฑ๐ฒ ๐ถ๐ ๐ฏ๐ฒ๐๐๐ฒ๐ฟ ๐ฏ๐ผ๐๐ต ๐ถ๐ป ๐ฐ๐ผ๐ป๐ฐ๐ถ๐๐ฒ๐ป๐ฒ๐๐ ๐ฎ๐ป๐ฑ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ:
โค Up to 30% fewer steps for the same actions (much more concise)
โค Up to 20% higher performance on benchmarks
And we find additional benefits, for instance a natural handling of variables.
Read the paper here ๐ Executable Code Actions Elicit Better LLM Agents (2402.01030)
Get your ReactCodeAgent running with our Agents framework! ๐ https://huggingface.co/learn/cookbook/agents
I was really happy to learn today by @sergeipetrov that paper ๐๐น๐ฆ๐ค๐ถ๐ต๐ข๐ฃ๐ญ๐ฆ ๐๐ฐ๐ฅ๐ฆ ๐๐ค๐ต๐ช๐ฐ๐ฏ๐ด ๐๐ญ๐ช๐ค๐ช๐ต ๐๐ฆ๐ต๐ต๐ฆ๐ณ ๐๐๐ ๐๐จ๐ฆ๐ฏ๐ต๐ด was accepted at ICLR 2024!
As a reminder, an agent is a system in which you embed a LLM engine, to let it call tools.
These tools are meant like an IronMan suit, to supplement the LLM in areas that it isn't good at.
๐งโ๐ป For instance your friendly LLM may be terrible at calculating powers of floating numbers ("What is X ^0.2947 ?"), so it should use a calculator.
๐It may be terrible at knowing precise facts ("What was the date of the Golden Bull?") so it should use a web browser.
So the agent system will prompt an agent with "Now you can use these tools: calculator, search,..."
But ๐๐ค๐ฌ ๐จ๐๐ค๐ช๐ก๐ ๐ฉ๐๐ ๐๐๐๐ฃ๐ฉ ๐๐ญ๐ฅ๐ง๐๐จ๐จ ๐๐ฉ๐จ ๐๐๐ฉ๐๐ค๐ฃ๐จ?
All well known frameworks let agents write their actions as JSON strings.
We ๐ฝ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฟ๐ฒ๐ฑ ๐๐ผ ๐ด๐ผ ๐๐ถ๐๐ต ๐ณ๐ผ๐ฟ๐บ๐๐น๐ฎ๐๐ถ๐ป๐ด ๐ฎ๐ฐ๐๐ถ๐ผ๐ป๐ ๐ถ๐ป ๐๐ผ๐ฑ๐ฒ, ๐๐ต๐ถ๐ฐ๐ต ๐ถ๐ ๐บ๐๐ฐ๐ต ๐บ๐ผ๐ฟ๐ฒ ๐๐ฒ๐ฟ๐๐ฎ๐๐ถ๐น๐ฒ ๐ฎ๐ป๐ฑ ๐ฐ๐ผ๐ป๐ฐ๐ถ๐๐ฒ, ๐ฎ๐ป๐ฑ ๐ฎ๐น๐น๐ผ๐๐ ๐๐ผ ๐ฐ๐ต๐ฎ๐ถ๐ป ๐ฎ๐ฐ๐๐ถ๐ผ๐ป๐ ๐๐ฒ๐ฎ๐บ๐น๐ฒ๐๐๐น๐: see the picture attached for an example where Code formulation really shines.
And the paper confirms our choice: researchers show that ๐ฐ๐ผ๐บ๐ฝ๐ฎ๐ฟ๐ฒ๐ฑ ๐๐ผ ๐๐ฆ๐ข๐ก ๐ผ๐ฟ ๐ฝ๐น๐ฎ๐ถ๐ป ๐๐ฒ๐ ๐, ๐๐ผ๐ฑ๐ฒ ๐ถ๐ ๐ฏ๐ฒ๐๐๐ฒ๐ฟ ๐ฏ๐ผ๐๐ต ๐ถ๐ป ๐ฐ๐ผ๐ป๐ฐ๐ถ๐๐ฒ๐ป๐ฒ๐๐ ๐ฎ๐ป๐ฑ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ:
โค Up to 30% fewer steps for the same actions (much more concise)
โค Up to 20% higher performance on benchmarks
And we find additional benefits, for instance a natural handling of variables.
Read the paper here ๐ Executable Code Actions Elicit Better LLM Agents (2402.01030)
Get your ReactCodeAgent running with our Agents framework! ๐ https://huggingface.co/learn/cookbook/agents