Short term memory

[amri369]

Support for Short-term Memory in OpeAI Agents SDK

The current SDK doesn't support short-term memory, which is needed at two levels:

• Tools output
• User requests

For tools, managing this at the server side (e.g., caching at the API or database level) seems like a reasonable approach.

When it comes to caching for LLM calls, what would be the ideal approach?

I’d love to get feedback and suggestions on how best to tackle this.

[rm-openai]

What do you mean by caching? Like for the same user input, you want to cache the LLM response and use the same output?

[amri369]

Yes exactly

[rm-openai]

Generally it's not the best idea to cache the LLM calls because:

1. The inputs from the user for even the same query will be subtly different, so you wont get cache hits very often. e.g. "best restaurants near me" vs "good restaurants nearby", plus typos, etc etc.
2. Even if the input was identical, you might have different tools, conversation history, personalized info etc.

Assuming you still want to do it, you can create a new instance of Model, wrap the existing model class, and use @functools.cache or similar to cache calls.

[amri369]

Many thanks @rm-openai . Aligned on the approach.