Description
Is there some equivalent to AutoGen's reflect_on_tool_use=False
? That is: Some way for an agent to use the output of one of its tools as its own output, without needing another roundtrip to the LLM?
Worked example:
# Untested code, just illustrative
class LoremModel(BaseModel):
paragraph: str
@function_tool
def generate_lorem(n: int) -> List[LoremModel]:
'''Generate `n` paragraphs of lorem ipsum'''
# omitted
agent = Agent(
name='Lorem Agent',
tools=[generate_lorem],
output_type=List[LoremModel],
)
await Runner.run(agent, 'Generate 1000 paragraphs of lorem ipsum')
I believe the way this works is:
- Prompt gets sent up to the LLM
- LLM responds requesting invocation of
generate_lorem(1000)
- We execute that function and now have 1000 paragraphs of lorem ipsum in memory
- We send these 1000 paragraphs back up to the LLM
- The LLM generates a final response to the user prompt, which is just those same paragraphs of lorem ipsum again.
In practice we already have our answer after step 3, but then have this extra roundtrip in steps 4&5, which can get expensive in terms of both tokens and time if the output from our tool is large.
Is there functionality or design pattern to work around this? I think it would really help push the composability of agents even further than you already have with this SDK!
Real world usecase: I have an agent whose goal it is to retrieve a chunky piece of content. It has ~5 tools at its disposal. The "main" tool actually retrieves the data, the other tools help the agent determine the correct parameters with which to invoke the main tool. I would like to then use this "data retrieval agent" inside other agents which can then use this dataset to do interesting things.
I can work around this by making my "data retrieval agent" only return the correct parameters with which to call the main tool, and leave the actual retrieval to other agents, but then it feels like my agent isn't encapsulating a nice isolated chunk of work.
Love the library so far, thanks for your work!
Activity
TimoVink commentedon Mar 13, 2025
Apologies if there's mistakes in my understanding of how AutoGen works, or how this library works. New to the space (as many of us are, I imagine!)
rm-openai commentedon Mar 13, 2025
That's a cool feature. Currently don't have support for it, but we'll talk about it and see if we can add it!
LeonG7 commentedon Mar 14, 2025
Sometimes I query a lot of web data through an agent, but the agent summarizes the web information, resulting in missing information. I hope to receive the complete information obtained through web search. Perhaps using an intermediate variable can achieve this task, but this solution is cool
rm-openai commentedon Mar 17, 2025
@TimoVink - implemented in #203. Chime in there if you have any feedback!
TimoVink commentedon Mar 17, 2025
Wasn't expecting anything so quickly. Thanks!
Had a play, and this definitely solves my usecase. Left 2 small comments on the PR but even if it was merged exactly as-is I'd be a happy user.
SeeknnDestroy commentedon Mar 19, 2025
Hey @TimoVink @rm-openai
Thanks so much for raising and implementing this feature — it’s super helpful! 🙌
I’d love to try this out in my own project. Could you share any details or code samples on how to use the new tool_use_behavior feature with an agent? A quick example or some guidance on how to set it up would be really appreciated!
Thanks again for the quick turnaround and the awesome work!
Introduce tool_use_behavior on agents (#203)