Skip to content

Commit f001f61

Browse files
unsafecodemoonbox3
andauthoredMar 20, 2025
Python: Add Copilot Studio Agents and Copilot Studio Skill demos (microsoft#11006)
### Motivation and Context Adds two new Python demos demonstrating integration with Microsoft Copilot Studio (MCS). ### Description - `copilot_studio_agent` features a new subclass of `Agent` connecting to a Microsoft Copilot Studio agent via DirectLine API - `copilot_studio_skill` showcases how to extend MCS to connect directly to a Semantic Kernel Agent, by defining a _skill_ ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]>
1 parent e48a975 commit f001f61

29 files changed

+1302
-0
lines changed
 

‎python/samples/demos/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ Demonstration applications that leverage the usage of one or many SK features
66
| ----------------- | ----------------------------------------------- |
77
| assistants_group_chat | A sample Agent demo that shows a chat functionality with an OpenAI Assistant agent. |
88
| booking_restaurant | A sample chat bot that leverages the Microsoft Graph and Bookings API as a Semantic Kernel plugin to make a fake booking at a restaurant. |
9+
| copilot_studio_agent | A sample that shows how to invoke Microsoft Copilot Studio agents as first-party Agent in Semantic Kernel|
10+
| copilot_studio_skill | A sample demonstrating how to extend Microsoft Copilot Studio to invoke Semantic Kernel agents |
911
| guided_conversations | A sample showing a framework for a pattern of use cases referred to as guided conversations. |
1012
| processes_with_dapr | A sample showing the Semantic Kernel process framework used with the Python Dapr runtime. |
1113
| telemetry_with_application_insights | A sample project that shows how a Python application can be configured to send Semantic Kernel telemetry to Application Insights. |
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
BOT_SECRET="copy from Copilot Studio Agent, under Settings > Security > Web Channel"
2+
BOT_ENDPOINT="https://europe.directline.botframework.com/v3/directline"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Copilot Studio Agents interaction
2+
3+
This is a simple example of how to interact with Copilot Studio Agents as they were first-party agents in Semantic Kernel.
4+
5+
![alt text](image.png)
6+
7+
## Rationale
8+
9+
Semantic Kernel already features many different types of agents, including `ChatCompletionAgent`, `AzureAIAgent`, `OpenAIAssistantAgent` or `AutoGenConversableAgent`. All of them though involve code-based agents.
10+
11+
Instead, [Microsoft Copilot Studio](https://learn.microsoft.com/en-us/microsoft-copilot-studio/fundamentals-what-is-copilot-studio) allows you to create declarative, low-code, and easy-to-maintain agents and publish them over multiple channels.
12+
13+
This way, you can create any amount of agents in Copilot Studio and interact with them along with code-based agents in Semantic Kernel, thus being able to use the best of both worlds.
14+
15+
## Implementation
16+
17+
The implementation is quite simple, since Copilot Studio can publish agents over DirectLine API, which we can use in Semantic Kernel to define a new subclass of `Agent` named [`DirectLineAgent`](src/direct_line_agent.py).
18+
19+
Additionally, we do enforce [authentication to the DirectLine API](https://learn.microsoft.com/en-us/microsoft-copilot-studio/configure-web-security).
20+
21+
## Usage
22+
23+
> [!NOTE]
24+
> Working with Copilot Studio Agents requires a [subscription](https://learn.microsoft.com/en-us/microsoft-copilot-studio/requirements-licensing-subscriptions) to Microsoft Copilot Studio.
25+
26+
> [!TIP]
27+
> In this case, we suggest to start with a simple Q&A Agent and supply a PDF to answer some questions. You can find a free sample like [Microsoft Surface Pro 4 User Guide](https://download.microsoft.com/download/2/9/B/29B20383-302C-4517-A006-B0186F04BE28/surface-pro-4-user-guide-EN.pdf)
28+
29+
1. [Create a new agent](https://learn.microsoft.com/en-us/microsoft-copilot-studio/fundamentals-get-started?tabs=web) in Copilot Studio
30+
2. [Publish the agent](https://learn.microsoft.com/en-us/microsoft-copilot-studio/publication-fundamentals-publish-channels?tabs=web)
31+
3. Turn off default authentication under the agent Settings > Security
32+
4. [Setup web channel security](https://learn.microsoft.com/en-us/microsoft-copilot-studio/configure-web-security) and copy the secret value
33+
34+
Once you're done with the above steps, you can use the following code to interact with the Copilot Studio Agent:
35+
36+
1. Copy the `.env.sample` file to `.env` and set the `BOT_SECRET` environment variable to the secret value
37+
2. Run the following code:
38+
39+
```bash
40+
python -m venv .venv
41+
42+
# On Mac/Linux
43+
source .venv/bin/activate
44+
# On Windows
45+
.venv\Scripts\Activate.ps1
46+
47+
pip install -r requirements.txt
48+
49+
chainlit run --port 8081 .\chat.py
50+
```
Loading
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import logging
4+
import os
5+
6+
import chainlit as cl
7+
from direct_line_agent import DirectLineAgent
8+
from dotenv import load_dotenv
9+
10+
from semantic_kernel.contents.chat_history import ChatHistory
11+
12+
load_dotenv(override=True)
13+
14+
logging.basicConfig(level=logging.INFO)
15+
logging.getLogger("direct_line_agent").setLevel(logging.DEBUG)
16+
logger = logging.getLogger(__name__)
17+
18+
agent = DirectLineAgent(
19+
id="copilot_studio",
20+
name="copilot_studio",
21+
description="copilot_studio",
22+
bot_secret=os.getenv("BOT_SECRET"),
23+
bot_endpoint=os.getenv("BOT_ENDPOINT"),
24+
)
25+
26+
27+
@cl.on_chat_start
28+
async def on_chat_start():
29+
cl.user_session.set("chat_history", ChatHistory())
30+
31+
32+
@cl.on_message
33+
async def on_message(message: cl.Message):
34+
chat_history: ChatHistory = cl.user_session.get("chat_history")
35+
36+
chat_history.add_user_message(message.content)
37+
38+
response = await agent.get_response(history=chat_history)
39+
40+
cl.user_session.set("chat_history", chat_history)
41+
42+
logger.info(f"Response: {response}")
43+
44+
await cl.Message(content=response.content, author=agent.name).send()
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
import logging
5+
import sys
6+
from collections.abc import AsyncIterable
7+
from typing import Any
8+
9+
if sys.version_info >= (3, 12):
10+
from typing import override # pragma: no cover
11+
else:
12+
from typing_extensions import override # pragma: no cover
13+
import aiohttp
14+
15+
from semantic_kernel.agents import Agent
16+
from semantic_kernel.contents.chat_history import ChatHistory
17+
from semantic_kernel.contents.chat_message_content import ChatMessageContent
18+
from semantic_kernel.exceptions.agent_exceptions import AgentInvokeException
19+
from semantic_kernel.utils.telemetry.agent_diagnostics.decorators import (
20+
trace_agent_get_response,
21+
trace_agent_invocation,
22+
)
23+
24+
logger = logging.getLogger(__name__)
25+
26+
27+
class DirectLineAgent(Agent):
28+
"""
29+
An Agent subclass that connects to a DirectLine Bot from Microsoft Bot Framework.
30+
Instead of directly supplying a secret and conversation ID, the agent queries a token_endpoint
31+
to retrieve the token and then starts a conversation.
32+
"""
33+
34+
token_endpoint: str | None = None
35+
bot_secret: str | None = None
36+
bot_endpoint: str
37+
conversation_id: str | None = None
38+
directline_token: str | None = None
39+
session: aiohttp.ClientSession = None
40+
41+
async def _ensure_session(self) -> None:
42+
"""
43+
Lazily initialize the aiohttp ClientSession.
44+
"""
45+
if self.session is None:
46+
self.session = aiohttp.ClientSession()
47+
48+
async def _fetch_token_and_conversation(self) -> None:
49+
"""
50+
Retrieve the DirectLine token either by using the bot_secret or by querying the token_endpoint.
51+
If bot_secret is provided, it posts to "https://directline.botframework.com/v3/directline/tokens/generate".
52+
"""
53+
await self._ensure_session()
54+
try:
55+
if self.bot_secret:
56+
url = f"{self.bot_endpoint}/tokens/generate"
57+
headers = {"Authorization": f"Bearer {self.bot_secret}"}
58+
async with self.session.post(url, headers=headers) as resp:
59+
if resp.status == 200:
60+
data = await resp.json()
61+
self.directline_token = data.get("token")
62+
if not self.directline_token:
63+
logger.error("Token generation response missing token: %s", data)
64+
raise AgentInvokeException("No token received from token generation.")
65+
else:
66+
logger.error("Token generation endpoint error status: %s", resp.status)
67+
raise AgentInvokeException("Failed to generate token using bot_secret.")
68+
else:
69+
async with self.session.get(self.token_endpoint) as resp:
70+
if resp.status == 200:
71+
data = await resp.json()
72+
self.directline_token = data.get("token")
73+
if not self.directline_token:
74+
logger.error("Token endpoint returned no token: %s", data)
75+
raise AgentInvokeException("No token received.")
76+
else:
77+
logger.error("Token endpoint error status: %s", resp.status)
78+
raise AgentInvokeException("Failed to fetch token from token endpoint.")
79+
except Exception as ex:
80+
logger.exception("Exception fetching token: %s", ex)
81+
raise AgentInvokeException("Exception occurred while fetching token.") from ex
82+
83+
@trace_agent_get_response
84+
@override
85+
async def get_response(
86+
self,
87+
history: ChatHistory,
88+
arguments: dict[str, Any] | None = None,
89+
**kwargs: Any,
90+
) -> ChatMessageContent:
91+
"""
92+
Get a response from the DirectLine Bot.
93+
"""
94+
responses = []
95+
async for response in self.invoke(history, arguments, **kwargs):
96+
responses.append(response)
97+
98+
if not responses:
99+
raise AgentInvokeException("No response from DirectLine Bot.")
100+
101+
return responses[0]
102+
103+
@trace_agent_invocation
104+
@override
105+
async def invoke(
106+
self,
107+
history: ChatHistory,
108+
arguments: dict[str, Any] | None = None,
109+
**kwargs: Any,
110+
) -> AsyncIterable[ChatMessageContent]:
111+
"""
112+
Send the latest message from the chat history to the DirectLine Bot
113+
and yield responses. This sends the payload after ensuring that:
114+
1. The token is fetched.
115+
2. A conversation is started.
116+
3. The activity payload is posted.
117+
4. Activities are polled until an event "DynamicPlanFinished" is received.
118+
"""
119+
payload = self._build_payload(history, arguments, **kwargs)
120+
response_data = await self._send_message(payload)
121+
if response_data is None or "activities" not in response_data:
122+
raise AgentInvokeException(f"Invalid response from DirectLine Bot.\n{response_data}")
123+
124+
logger.debug("DirectLine Bot response: %s", response_data)
125+
126+
# NOTE DirectLine Activities have different formats
127+
# than ChatMessageContent. We need to convert them and
128+
# remove unsupported activities.
129+
for activity in response_data["activities"]:
130+
if activity.get("type") != "message" or activity.get("from", {}).get("role") == "user":
131+
continue
132+
role = activity.get("from", {}).get("role", "assistant")
133+
if role == "bot":
134+
role = "assistant"
135+
message = ChatMessageContent(
136+
role=role,
137+
content=activity.get("text", ""),
138+
name=activity.get("from", {}).get("name", self.name),
139+
)
140+
yield message
141+
142+
def _build_payload(
143+
self,
144+
history: ChatHistory,
145+
arguments: dict[str, Any] | None = None,
146+
**kwargs: Any,
147+
) -> dict[str, Any]:
148+
"""
149+
Build the message payload for the DirectLine Bot.
150+
Uses the latest message from the chat history.
151+
"""
152+
latest_message = history.messages[-1] if history.messages else None
153+
text = latest_message.content if latest_message else "Hello"
154+
payload = {
155+
"type": "message",
156+
"from": {"id": "user"},
157+
"text": text,
158+
}
159+
# Optionally include conversationId if available.
160+
if self.conversation_id:
161+
payload["conversationId"] = self.conversation_id
162+
return payload
163+
164+
async def _send_message(self, payload: dict[str, Any]) -> dict[str, Any] | None:
165+
"""
166+
1. Ensure the token is fetched.
167+
2. Start a conversation by posting to the bot_endpoint /conversations endpoint (without a payload)
168+
3. Post the payload to /conversations/{conversationId}/activities
169+
4. Poll GET /conversations/{conversationId}/activities every 1s using a watermark
170+
to fetch only the latest messages until an activity with type="event"
171+
and name="DynamicPlanFinished" is found.
172+
"""
173+
await self._ensure_session()
174+
if not self.directline_token:
175+
await self._fetch_token_and_conversation()
176+
177+
headers = {
178+
"Authorization": f"Bearer {self.directline_token}",
179+
"Content-Type": "application/json",
180+
}
181+
182+
# Step 2: Start a conversation if one hasn't already been started.
183+
if not self.conversation_id:
184+
start_conv_url = f"{self.bot_endpoint}/conversations"
185+
async with self.session.post(start_conv_url, headers=headers) as resp:
186+
if resp.status not in (200, 201):
187+
logger.error("Failed to start conversation. Status: %s", resp.status)
188+
raise AgentInvokeException("Failed to start conversation.")
189+
conv_data = await resp.json()
190+
self.conversation_id = conv_data.get("conversationId")
191+
if not self.conversation_id:
192+
raise AgentInvokeException("Conversation ID not found in start response.")
193+
194+
# Step 3: Post the message payload.
195+
activities_url = f"{self.bot_endpoint}/conversations/{self.conversation_id}/activities"
196+
async with self.session.post(activities_url, json=payload, headers=headers) as resp:
197+
if resp.status != 200:
198+
logger.error("Failed to post activity. Status: %s", resp.status)
199+
raise AgentInvokeException("Failed to post activity.")
200+
_ = await resp.json() # Response from posting activity is ignored.
201+
202+
# Step 4: Poll for new activities using watermark until DynamicPlanFinished event is found.
203+
finished = False
204+
collected_data = None
205+
watermark = None
206+
while not finished:
207+
url = activities_url if watermark is None else f"{activities_url}?watermark={watermark}"
208+
async with self.session.get(url, headers=headers) as resp:
209+
if resp.status == 200:
210+
data = await resp.json()
211+
watermark = data.get("watermark", watermark)
212+
activities = data.get("activities", [])
213+
if any(
214+
activity.get("type") == "event" and activity.get("name") == "DynamicPlanFinished"
215+
for activity in activities
216+
):
217+
collected_data = data
218+
finished = True
219+
break
220+
else:
221+
logger.error("Error polling activities. Status: %s", resp.status)
222+
await asyncio.sleep(0.3)
223+
224+
return collected_data
225+
226+
async def close(self) -> None:
227+
"""
228+
Clean up the aiohttp session.
229+
"""
230+
await self.session.close()
231+
232+
# NOTE not implemented yet, possibly use websockets
233+
@trace_agent_invocation
234+
@override
235+
async def invoke_stream(self, *args, **kwargs):
236+
return super().invoke_stream(*args, **kwargs)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
chainlit>=2.0.1
2+
python-dotenv>=1.0.1
3+
aiohttp>=3.10.5
4+
semantic-kernel>=1.22.0

0 commit comments

Comments
 (0)
Please sign in to comment.