Skip to content

Commit d7172a5

Browse files
authoredMar 27, 2025
Python: Common agent invocation API updates (microsoft#11224)
### Motivation and Context After this week's release of the common agent invocation API, there are some things that we can do as further improvements. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR includes updates for: - Right now we force any agent invocation to provide some type of input, whether a str, a CMC, or a list of str | CMC. At times, there could be a reason, based on using an existing thread or just providing instructions to the agent, that one doesn't need to provide a message to invoke the agent. Updating to make messages optional. Updating the ABC contracts as well. - For `invoke_stream` calls on agents, there's no need to call `thread.on_new_message` that contains a streaming chunks -- once we move to support memory, this is where the "hook" will be. Removing this call. - The `get_messages(...)` methods on the `AutoGenConversableAgentThread` and the `ChatHistoryAgentThread` returned concrete `ChatHistory` objects, whereas the `AssistantAgentThread` and `AzureAIAgentThread` returned `AsyncIterable[ChatMessageContent]`. To align to a common API, the `AutoGenConversableAgentThread` and `ChatHistoryAgentThread`'s `get_messages(...)` methods were moved to return `AsyncIterable[ChatMessageContent]`. - Removing a public facing `output_messages` for streaming invoke, and replacing it with a callback to get a chat history back of "full" messages. Two samples are added in `samples/concepts/agents`: - `azure_ai_agent/azure_ai_agent_streaming_chat_history_callback.py` - `openai_assistant_streaming_chat_history_callback.py` - Update the README for OpenAI Assistants to showcase new thread abstraction. - Include unit tests for chat history (`on_complete`) callback. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄
1 parent 8875f9f commit d7172a5

File tree

17 files changed

+429
-84
lines changed

17 files changed

+429
-84
lines changed
 

‎python/pyproject.toml

+1-2
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ dependencies = [
2929
"pydantic-settings ~= 2.0",
3030
"defusedxml ~= 0.7",
3131
# azure identity
32-
"azure-identity ~= 1.13",
32+
"azure-identity >= 1.13",
3333
# embeddings
3434
"numpy >= 1.25.0; python_version < '3.12'",
3535
"numpy >= 1.26.0; python_version >= '3.12'",
@@ -62,7 +62,6 @@ azure = [
6262
"azure-ai-projects >= 1.0.0b7",
6363
"azure-core-tracing-opentelemetry >= 1.0.0b11",
6464
"azure-search-documents >= 11.6.0b4",
65-
"azure-identity ~= 1.13",
6665
"azure-cosmos ~= 4.7"
6766
]
6867
chroma = [

‎python/samples/concepts/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
- [Azure AI Agent as Kernel Function](./agents/azure_ai_agent/azure_ai_agent_as_kernel_function.py)
1010
- [Azure AI Agent with Azure AI Search](./agents/azure_ai_agent/azure_ai_agent_azure_ai_search.py)
1111
- [Azure AI Agent File Manipulation](./agents/azure_ai_agent/azure_ai_agent_file_manipulation.py)
12+
- [Azure AI Agent Chat History Callback](./agents/azure_ai_agent/azure_ai_agent_streaming_chat_history_callback.py)
1213
- [Azure AI Agent Streaming](./agents/azure_ai_agent/azure_ai_agent_streaming.py)
1314

1415
#### [Bedrock Agent](../../semantic_kernel/agents/bedrock/bedrock_agent.py)
@@ -50,6 +51,7 @@
5051
- [OpenAI Assistant File Manipulation](./agents/openai_assistant/openai_assistant_file_manipulation.py)
5152
- [OpenAI Assistant File Manipulation Streaming](./agents/openai_assistant/openai_assistant_file_manipulation_streaming.py)
5253
- [OpenAI Assistant Retrieval](./agents/openai_assistant/openai_assistant_retrieval.py)
54+
- [OpenAI Assistant Streaming Chat History Callback](./agents/openai_assistant/openai_assistant_streaming_chat_history_callback.py)
5355
- [OpenAI Assistant Streaming](./agents/openai_assistant/openai_assistant_streaming.py)
5456
- [OpenAI Assistant Structured Outputs](./agents/openai_assistant/openai_assistant_structured_outputs.py)
5557
- [OpenAI Assistant Templating Streaming](./agents/openai_assistant/openai_assistant_templating_streaming.py)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
from typing import Annotated
5+
6+
from azure.identity.aio import DefaultAzureCredential
7+
8+
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings, AzureAIAgentThread
9+
from semantic_kernel.contents import ChatHistory, FunctionCallContent, FunctionResultContent
10+
from semantic_kernel.functions import kernel_function
11+
12+
"""
13+
The following sample demonstrates how to create an Azure AI Agent
14+
and use it with streaming responses. Additionally, the invoke_stream
15+
configures a chat history callback to receive the conversation history
16+
once the streaming invocation is complete. The agent is configured to use
17+
a plugin that provides a list of specials from the menu and the price
18+
of the requested menu item.
19+
"""
20+
21+
22+
# Define a sample plugin for the sample
23+
class MenuPlugin:
24+
"""A sample Menu Plugin used for the concept sample."""
25+
26+
@kernel_function(description="Provides a list of specials from the menu.")
27+
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
28+
return """
29+
Special Soup: Clam Chowder
30+
Special Salad: Cobb Salad
31+
Special Drink: Chai Tea
32+
"""
33+
34+
@kernel_function(description="Provides the price of the requested menu item.")
35+
def get_item_price(
36+
self, menu_item: Annotated[str, "The name of the menu item."]
37+
) -> Annotated[str, "Returns the price of the menu item."]:
38+
return "$9.99"
39+
40+
41+
final_chat_history = ChatHistory()
42+
43+
44+
def handle_stream_completion(history: ChatHistory) -> None:
45+
final_chat_history.messages.extend(history.messages)
46+
47+
48+
async def main() -> None:
49+
ai_agent_settings = AzureAIAgentSettings.create()
50+
51+
async with (
52+
DefaultAzureCredential() as creds,
53+
AzureAIAgent.create_client(
54+
credential=creds,
55+
conn_str=ai_agent_settings.project_connection_string.get_secret_value(),
56+
) as client,
57+
):
58+
AGENT_NAME = "Host"
59+
AGENT_INSTRUCTIONS = "Answer questions about the menu."
60+
61+
# Create agent definition
62+
agent_definition = await client.agents.create_agent(
63+
model=ai_agent_settings.model_deployment_name,
64+
name=AGENT_NAME,
65+
instructions=AGENT_INSTRUCTIONS,
66+
)
67+
68+
# Create the AzureAI Agent
69+
agent = AzureAIAgent(
70+
client=client,
71+
definition=agent_definition,
72+
plugins=[MenuPlugin()], # add the sample plugin to the agent
73+
)
74+
75+
# Create a thread for the agent
76+
# If no thread is provided, a new thread will be
77+
# created and returned with the initial response
78+
thread: AzureAIAgentThread = None
79+
80+
user_inputs = [
81+
"Hello",
82+
"What is the special soup?",
83+
"How much does that cost?",
84+
"Thank you",
85+
]
86+
87+
try:
88+
for user_input in user_inputs:
89+
print(f"# User: '{user_input}'")
90+
first_chunk = True
91+
async for response in agent.invoke_stream(
92+
messages=user_input,
93+
thread=thread,
94+
on_complete=handle_stream_completion,
95+
):
96+
if first_chunk:
97+
print(f"# {response.role}: ", end="", flush=True)
98+
first_chunk = False
99+
print(response.content, end="", flush=True)
100+
thread = response.thread
101+
print()
102+
finally:
103+
# Cleanup: Delete the thread and agent
104+
await thread.delete() if thread else None
105+
await client.agents.delete_agent(agent.id)
106+
107+
# Print the final chat history
108+
print("\nFinal chat history:")
109+
for msg in final_chat_history.messages:
110+
if any(isinstance(item, FunctionResultContent) for item in msg.items):
111+
for fr in msg.items:
112+
if isinstance(fr, FunctionResultContent):
113+
print(f"Function Result:> {fr.result} for function: {fr.name}")
114+
elif any(isinstance(item, FunctionCallContent) for item in msg.items):
115+
for fcc in msg.items:
116+
if isinstance(fcc, FunctionCallContent):
117+
print(f"Function Call:> {fcc.name} with arguments: {fcc.arguments}")
118+
else:
119+
print(f"{msg.role}: {msg.content}")
120+
121+
122+
if __name__ == "__main__":
123+
asyncio.run(main())

‎python/samples/concepts/agents/openai_assistant/README.md

+10-12
Original file line numberDiff line numberDiff line change
@@ -47,15 +47,14 @@ agent = OpenAIAssistantAgent(
4747
definition=definition,
4848
)
4949

50-
# Define a thread and invoke the agent with the user input
51-
thread = await agent.client.beta.threads.create()
52-
53-
# Add a message to the thread
54-
await agent.add_chat_message(thread_id=thread.id, message="Why is the sky blue?")
50+
# Define a thread
51+
thread = None
5552

5653
# Invoke the agent
57-
async for content in agent.invoke(thread_id=thread.id):
54+
async for content in agent.invoke(messages="user input", thread=thread):
5855
print(f"# {content.role}: {content.content}")
56+
# Grab the thread from the response to continue with the current context
57+
thread = response.thread
5958
```
6059

6160
### Semantic Kernel Azure Assistant Agents
@@ -89,13 +88,12 @@ agent = AzureAssistantAgent(
8988
definition=definition,
9089
)
9190

92-
# Define a thread and invoke the agent with the user input
93-
thread = await agent.client.beta.threads.create()
94-
95-
# Add a message to the thread
96-
await agent.add_chat_message(thread_id=thread.id, message="Why is the sky blue?")
91+
# Define a thread
92+
thread = None
9793

9894
# Invoke the agent
99-
async for content in agent.invoke(thread_id=thread.id):
95+
async for content in agent.invoke(messages="user input", thread=thread):
10096
print(f"# {content.role}: {content.content}")
97+
# Grab the thread from the response to continue with the current context
98+
thread = response.thread
10199
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
import asyncio
3+
from typing import Annotated
4+
5+
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
6+
from semantic_kernel.contents import AuthorRole, ChatHistory, FunctionCallContent, FunctionResultContent
7+
from semantic_kernel.functions import kernel_function
8+
9+
"""
10+
The following sample demonstrates how to create an OpenAI
11+
assistant using either Azure OpenAI or OpenAI. OpenAI Assistants
12+
allow for function calling, the use of file search and a
13+
code interpreter. Assistant Threads are used to manage the
14+
conversation state, similar to a Semantic Kernel Chat History.
15+
Additionally, the invoke_stream configures a chat history callback
16+
to receive the conversation history once the streaming invocation
17+
is complete. This sample also demonstrates the Assistants Streaming
18+
capability and how to manage an Assistants chat history.
19+
"""
20+
21+
22+
# Define a sample plugin for the sample
23+
class MenuPlugin:
24+
"""A sample Menu Plugin used for the concept sample."""
25+
26+
@kernel_function(description="Provides a list of specials from the menu.")
27+
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
28+
return """
29+
Special Soup: Clam Chowder
30+
Special Salad: Cobb Salad
31+
Special Drink: Chai Tea
32+
"""
33+
34+
@kernel_function(description="Provides the price of the requested menu item.")
35+
def get_item_price(
36+
self, menu_item: Annotated[str, "The name of the menu item."]
37+
) -> Annotated[str, "Returns the price of the menu item."]:
38+
return "$9.99"
39+
40+
41+
final_chat_history = ChatHistory()
42+
43+
44+
def handle_stream_completion(history: ChatHistory) -> None:
45+
final_chat_history.messages.extend(history.messages)
46+
47+
48+
async def main():
49+
# Create the client using Azure OpenAI resources and configuration
50+
client, model = AzureAssistantAgent.setup_resources()
51+
52+
# Define the assistant definition
53+
definition = await client.beta.assistants.create(
54+
model=model,
55+
name="Host",
56+
instructions="Answer questions about the menu.",
57+
)
58+
59+
# Create the AzureAssistantAgent instance using the client and the assistant definition and the defined plugin
60+
agent = AzureAssistantAgent(
61+
client=client,
62+
definition=definition,
63+
plugins=[MenuPlugin()],
64+
)
65+
66+
# Create a new thread for use with the assistant
67+
# If no thread is provided, a new thread will be
68+
# created and returned with the initial response
69+
thread: AssistantAgentThread = None
70+
71+
user_inputs = ["Hello", "What is the special soup?", "What is the special drink?", "How much is that?", "Thank you"]
72+
73+
try:
74+
for user_input in user_inputs:
75+
print(f"# {AuthorRole.USER}: '{user_input}'")
76+
77+
first_chunk = True
78+
async for response in agent.invoke_stream(
79+
messages=user_input,
80+
thread=thread,
81+
on_complete=handle_stream_completion,
82+
):
83+
thread = response.thread
84+
if first_chunk:
85+
print(f"# {response.role}: ", end="", flush=True)
86+
first_chunk = False
87+
print(response.content, end="", flush=True)
88+
print()
89+
finally:
90+
await thread.delete() if thread else None
91+
await client.beta.assistants.delete(assistant_id=agent.id)
92+
93+
# Print the final chat history
94+
print("\nFinal chat history:")
95+
for msg in final_chat_history.messages:
96+
if any(isinstance(item, FunctionResultContent) for item in msg.items):
97+
for fr in msg.items:
98+
if isinstance(fr, FunctionResultContent):
99+
print(f"Function Result:> {fr.result} for function: {fr.name}")
100+
elif any(isinstance(item, FunctionCallContent) for item in msg.items):
101+
for fcc in msg.items:
102+
if isinstance(fcc, FunctionCallContent):
103+
print(f"Function Call:> {fcc.name} with arguments: {fcc.arguments}")
104+
else:
105+
print(f"{msg.role}: {msg.content}")
106+
107+
108+
if __name__ == "__main__":
109+
asyncio.run(main())

‎python/samples/getting_started_with_agents/azure_ai_agent/README.md

+7-7
Original file line numberDiff line numberDiff line change
@@ -59,13 +59,13 @@ You can pass in a connection string (shown above) to create the client:
5959

6060
```python
6161
async with (
62-
DefaultAzureCredential() as creds,
63-
AzureAIAgent.create_client(
64-
credential=creds,
65-
conn_str=ai_agent_settings.project_connection_string.get_secret_value(),
66-
) as client,
67-
):
68-
# operational logic
62+
DefaultAzureCredential() as creds,
63+
AzureAIAgent.create_client(
64+
credential=creds,
65+
conn_str=ai_agent_settings.project_connection_string.get_secret_value(),
66+
) as client,
67+
):
68+
# operational logic
6969
```
7070

7171
### Creating an Agent Definition

‎python/semantic_kernel/agents/agent.py

+8-4
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,7 @@ async def _as_kernel_function(
236236
def get_response(
237237
self,
238238
*,
239-
messages: str | ChatMessageContent | list[str | ChatMessageContent],
239+
messages: str | ChatMessageContent | list[str | ChatMessageContent] | None = None,
240240
thread: AgentThread | None = None,
241241
**kwargs,
242242
) -> Awaitable[AgentResponseItem[ChatMessageContent]]:
@@ -266,7 +266,7 @@ def get_response(
266266
def invoke(
267267
self,
268268
*,
269-
messages: str | ChatMessageContent | list[str | ChatMessageContent],
269+
messages: str | ChatMessageContent | list[str | ChatMessageContent] | None = None,
270270
thread: AgentThread | None = None,
271271
**kwargs,
272272
) -> AsyncIterable[AgentResponseItem[ChatMessageContent]]:
@@ -291,7 +291,7 @@ def invoke(
291291
def invoke_stream(
292292
self,
293293
*,
294-
messages: str | ChatMessageContent | list[str | ChatMessageContent],
294+
messages: str | ChatMessageContent | list[str | ChatMessageContent] | None = None,
295295
thread: AgentThread | None = None,
296296
**kwargs,
297297
) -> AsyncIterable[AgentResponseItem[StreamingChatMessageContent]]:
@@ -379,12 +379,16 @@ def _merge_arguments(self, override_args: KernelArguments | None) -> KernelArgum
379379

380380
async def _ensure_thread_exists_with_messages(
381381
self,
382-
messages: str | ChatMessageContent | Sequence[str | ChatMessageContent],
382+
*,
383+
messages: str | ChatMessageContent | Sequence[str | ChatMessageContent] | None = None,
383384
thread: AgentThread | None,
384385
construct_thread: Callable[[], TThreadType],
385386
expected_type: type[TThreadType],
386387
) -> TThreadType:
387388
"""Ensure the thread exists with the provided message(s)."""
389+
if messages is None:
390+
messages = []
391+
388392
if isinstance(messages, (str, ChatMessageContent)):
389393
messages = [messages]
390394

0 commit comments

Comments
 (0)
Please sign in to comment.