Skip to content

Commit 1bf520a

Browse files
SergeyMenshykhCopilotrogerbarretowestey-m
authored
.NET: Add support for background responses (#1501)
* add support for background responses * Update dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunResponseUpdate.cs Co-authored-by: Copilot <[email protected]> * fix broken link * fix xml comments and background responses properties override funcitonity * change ai model provider * use Run{Streaming}Async overloads that don't require messages * stop using m: prefix in cref attribute of <see/> element. * reject input messages provided with continuation token + don't extract messages from message store and context provide if continuation token is provided * use agent thread for background-responses sample * require agent thread for background responses * Update dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs Co-authored-by: Roger Barreto <[email protected]> * Update dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs Co-authored-by: Roger Barreto <[email protected]> * remove CA1200 * Update dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunOptions.cs Co-authored-by: westey <[email protected]> * Update dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunResponse.cs Co-authored-by: westey <[email protected]> * Update dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunResponse.cs Co-authored-by: westey <[email protected]> * address pr review comments * Update dotnet/samples/GettingStarted/Agents/Agent_Step17_BackgroundResponses/Program.cs Co-authored-by: westey <[email protected]> --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Roger Barreto <[email protected]> Co-authored-by: westey <[email protected]>
1 parent 699149c commit 1bf520a

File tree

17 files changed

+2499
-39
lines changed

17 files changed

+2499
-39
lines changed

‎docs/decisions/0007-python-subpackages.md‎ renamed to ‎docs/decisions/0008-python-subpackages.md‎

File renamed without changes.

‎docs/decisions/0009-support-long-running-operations.md‎

Lines changed: 1689 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/design/python-package-setup.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ Sub-packages are comprised of two parts, the code itself and the dependencies, t
175175
- Subpackage naming should also follow this, so in principle a package name is `<vendor/folder>-<feature/brand>`, so `google-gemini`, `azure-purview`, `microsoft-copilotstudio`, etc. For smaller vendors, where it's less likely to have a multitude of connectors, we can skip the feature/brand part, so `mem0`, `redis`, etc.
176176
- For Microsoft services we will have two vendor folders, `azure` and `microsoft`, where `azure` contains all Azure services, while `microsoft` contains other Microsoft services, such as Copilot Studio Agents.
177177

178-
This setup was discussed at length and the decision is captured in [ADR-0007](../decisions/0007-python-subpackages.md).
178+
This setup was discussed at length and the decision is captured in [ADR-0008](../decisions/0008-python-subpackages.md).
179179

180180
#### Evolving the package structure
181181
For each of the advanced components, we have two reason why we may split them into a folder, with an `__init__.py` and optionally a `_files.py`:

‎dotnet/agent-framework-dotnet.slnx‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@
5757
<Project Path="samples/GettingStarted/Agents/Agent_Step14_Middleware/Agent_Step14_Middleware.csproj" />
5858
<Project Path="samples/GettingStarted/Agents/Agent_Step15_Plugins/Agent_Step15_Plugins.csproj" />
5959
<Project Path="samples/GettingStarted/Agents/Agent_Step16_ChatReduction/Agent_Step16_ChatReduction.csproj" />
60+
<Project Path="samples/GettingStarted/Agents/Agent_Step17_BackgroundResponses/Agent_Step17_BackgroundResponses.csproj" />
6061
</Folder>
6162
<Folder Name="/Samples/GettingStarted/AgentWithOpenAI/">
6263
<File Path="samples/GettingStarted/AgentWithOpenAI/README.md" />
@@ -162,8 +163,14 @@
162163
<Folder Name="/Solution Items/docs/" />
163164
<Folder Name="/Solution Items/docs/decisions/">
164165
<File Path="../docs/decisions/0001-agent-run-response.md" />
165-
<File Path="../docs/decisions/0001-agent-tools.md" />
166-
<File Path="../docs/decisions/0002-agent-opentelemetry-instrumentation.md" />
166+
<File Path="../docs/decisions/0002-agent-tools.md" />
167+
<File Path="../docs/decisions/0003-agent-opentelemetry-instrumentation.md" />
168+
<File Path="../docs/decisions/0004-foundry-sdk-extensions.md" />
169+
<File Path="../docs/decisions/0005-python-naming-conventions.md" />
170+
<File Path="../docs/decisions/0006-userapproval.md" />
171+
<File Path="../docs/decisions/0007-agent-filtering-middleware.md" />
172+
<File Path="../docs/decisions/0008-python-subpackages.md" />
173+
<File Path="../docs/decisions/0009-support-long-running-operations.md" />
167174
<File Path="../docs/decisions/adr-short-template.md" />
168175
<File Path="../docs/decisions/adr-template.md" />
169176
<File Path="../docs/decisions/README.md" />

‎dotnet/samples/GettingStarted/Agents/Agent_Step17_BackgroundResponses/Agent_Step17_BackgroundResponses.csproj‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<OutputType>Exe</OutputType>
5+
<TargetFramework>net9.0</TargetFramework>
6+
7+
<Nullable>enable</Nullable>
8+
<ImplicitUsings>enable</ImplicitUsings>
9+
</PropertyGroup>
10+
11+
<ItemGroup>
12+
<PackageReference Include="Azure.AI.OpenAI" />
13+
<PackageReference Include="Azure.Identity" />
14+
</ItemGroup>
15+
16+
<ItemGroup>
17+
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
18+
</ItemGroup>
19+
20+
</Project>

‎dotnet/samples/GettingStarted/Agents/Agent_Step17_BackgroundResponses/Program.cs‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
// Copyright (c) Microsoft. All rights reserved.
2+
3+
// This sample shows how to use background responses with ChatClientAgent and OpenAI Responses.
4+
5+
using Azure.AI.OpenAI;
6+
using Azure.Identity;
7+
using Microsoft.Agents.AI;
8+
using OpenAI;
9+
10+
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
11+
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
12+
13+
AIAgent agent = new AzureOpenAIClient(
14+
new Uri(endpoint),
15+
new AzureCliCredential())
16+
.GetOpenAIResponseClient(deploymentName)
17+
.CreateAIAgent(instructions: "You are good at telling jokes.", name: "Joker");
18+
19+
// Enable background responses (only supported by OpenAI Responses at this time).
20+
AgentRunOptions options = new() { AllowBackgroundResponses = true };
21+
22+
AgentThread thread = agent.GetNewThread();
23+
24+
// Start the initial run.
25+
AgentRunResponse response = await agent.RunAsync("Tell me a joke about a pirate.", thread, options);
26+
27+
// Poll until the response is complete.
28+
while (response.ContinuationToken is { } token)
29+
{
30+
// Wait before polling again.
31+
await Task.Delay(TimeSpan.FromSeconds(2));
32+
33+
// Continue with the token.
34+
options.ContinuationToken = token;
35+
36+
response = await agent.RunAsync(thread, options);
37+
}
38+
39+
// Display the result.
40+
Console.WriteLine(response.Text);
41+
42+
// Reset options and thread for streaming.
43+
options = new() { AllowBackgroundResponses = true };
44+
thread = agent.GetNewThread();
45+
46+
AgentRunResponseUpdate? lastReceivedUpdate = null;
47+
// Start streaming.
48+
await foreach (AgentRunResponseUpdate update in agent.RunStreamingAsync("Tell me a joke about a pirate.", thread, options))
49+
{
50+
// Output each update.
51+
Console.Write(update.Text);
52+
53+
// Track last update.
54+
lastReceivedUpdate = update;
55+
56+
// Simulate connection loss after first piece of content received.
57+
if (update.Text.Length > 0)
58+
{
59+
break;
60+
}
61+
}
62+
63+
// Resume from interruption point.
64+
options.ContinuationToken = lastReceivedUpdate?.ContinuationToken;
65+
66+
await foreach (AgentRunResponseUpdate update in agent.RunStreamingAsync(thread, options))
67+
{
68+
// Output each update.
69+
Console.Write(update.Text);
70+
}

‎dotnet/samples/GettingStarted/Agents/Agent_Step17_BackgroundResponses/README.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# What This Sample Shows
2+
3+
This sample demonstrates how to use background responses with ChatCompletionAgent and OpenAI Responses for long-running operations. Background responses support:
4+
5+
- **Polling for completion** - Non-streaming APIs can start a background operation and return a continuation token. Poll with the token until the response completes.
6+
- **Resuming after interruption** - Streaming APIs can be interrupted and resumed from the last update using the continuation token.
7+
8+
> **Note:** Background responses are currently only supported by OpenAI Responses.
9+
10+
# Prerequisites
11+
12+
Before you begin, ensure you have the following prerequisites:
13+
14+
- .NET 8.0 SDK or later
15+
- OpenAI api key
16+
17+
Set the following environment variables:
18+
19+
```powershell
20+
$env:AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/" # Replace with your Azure OpenAI resource endpoint
21+
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
22+
```

‎dotnet/samples/GettingStarted/Agents/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ Before you begin, ensure you have the following prerequisites:
4242
|[Using middleware with an agent](./Agent_Step14_Middleware/)|This sample demonstrates how to use middleware with an agent|
4343
|[Using plugins with an agent](./Agent_Step15_Plugins/)|This sample demonstrates how to use plugins with an agent|
4444
|[Reducing chat history size](./Agent_Step16_ChatReduction/)|This sample demonstrates how to reduce the chat history to constrain its size, where chat history is maintained locally|
45+
|[Background responses](./Agent_Step17_BackgroundResponses/)|This sample demonstrates how to use background responses for long-running operations with polling and resumption support|
4546

4647
## Running the samples from the console
4748

‎dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunOptions.cs‎

Lines changed: 43 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,6 @@ namespace Microsoft.Agents.AI;
1010
/// </summary>
1111
/// <remarks>
1212
/// <para>
13-
/// This class currently has no options, but may be extended in the future to include additional configuration settings.
14-
/// </para>
15-
/// <para>
1613
/// Implementations of <see cref="AIAgent"/> may provide subclasses of <see cref="AgentRunOptions"/> with additional options specific to that agent type.
1714
/// </para>
1815
/// </remarks>
@@ -33,5 +30,48 @@ public AgentRunOptions()
3330
public AgentRunOptions(AgentRunOptions options)
3431
{
3532
_ = Throw.IfNull(options);
33+
this.ContinuationToken = options.ContinuationToken;
34+
this.AllowBackgroundResponses = options.AllowBackgroundResponses;
3635
}
36+
37+
/// <summary>
38+
/// Gets or sets the continuation token for resuming and getting the result of the agent response identified by this token.
39+
/// </summary>
40+
/// <remarks>
41+
/// This property is used for background responses that can be activated via the <see cref="AllowBackgroundResponses"/>
42+
/// property if the <see cref="AIAgent"/> implementation supports them.
43+
/// Streamed background responses, such as those returned by default by <see cref="AIAgent.RunStreamingAsync(AgentThread?, AgentRunOptions?, System.Threading.CancellationToken)"/>
44+
/// can be resumed if interrupted. This means that a continuation token obtained from the <see cref="AgentRunResponseUpdate.ContinuationToken"/>
45+
/// of an update just before the interruption occurred can be passed to this property to resume the stream from the point of interruption.
46+
/// Non-streamed background responses, such as those returned by <see cref="AIAgent.RunAsync(AgentThread?, AgentRunOptions?, System.Threading.CancellationToken)"/>,
47+
/// can be polled for completion by obtaining the token from the <see cref="AgentRunResponse.ContinuationToken"/> property
48+
/// and passing it via this property on subsequent calls to <see cref="AIAgent.RunAsync(AgentThread?, AgentRunOptions?, System.Threading.CancellationToken)"/>.
49+
/// </remarks>
50+
public object? ContinuationToken { get; set; }
51+
52+
/// <summary>
53+
/// Gets or sets a value indicating whether the background responses are allowed.
54+
/// </summary>
55+
/// <remarks>
56+
/// <para>
57+
/// Background responses allow running long-running operations or tasks asynchronously in the background that can be resumed by streaming APIs
58+
/// and polled for completion by non-streaming APIs.
59+
/// </para>
60+
/// <para>
61+
/// When this property is set to true, non-streaming APIs may start a background operation and return an initial
62+
/// response with a continuation token. Subsequent calls to the same API should be made in a polling manner with
63+
/// the continuation token to get the final result of the operation.
64+
/// </para>
65+
/// <para>
66+
/// When this property is set to true, streaming APIs may also start a background operation and begin streaming
67+
/// response updates until the operation is completed. If the streaming connection is interrupted, the
68+
/// continuation token obtained from the last update that has one should be supplied to a subsequent call to the same streaming API
69+
/// to resume the stream from the point of interruption and continue receiving updates until the operation is completed.
70+
/// </para>
71+
/// <para>
72+
/// This property only takes effect if the implementation it's used with supports background responses.
73+
/// If the implementation does not support background responses, this property will be ignored.
74+
/// </para>
75+
/// </remarks>
76+
public bool? AllowBackgroundResponses { get; set; }
3777
}

‎dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunResponse.cs‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ public AgentRunResponse(ChatResponse response)
7474
this.RawRepresentation = response;
7575
this.ResponseId = response.ResponseId;
7676
this.Usage = response.Usage;
77+
this.ContinuationToken = response.ContinuationToken;
7778
}
7879

7980
/// <summary>
@@ -159,6 +160,23 @@ public IList<ChatMessage> Messages
159160
/// </value>
160161
public string? ResponseId { get; set; }
161162

163+
/// <summary>
164+
/// Gets or sets the continuation token for getting the result of a background agent response.
165+
/// </summary>
166+
/// <remarks>
167+
/// <see cref="AIAgent"/> implementations that support background responses will return
168+
/// a continuation token if background responses are allowed in <see cref="AgentRunOptions.AllowBackgroundResponses"/>
169+
/// and the result of the response has not been obtained yet. If the response has completed and the result has been obtained,
170+
/// the token will be <see langword="null"/>.
171+
/// <para>
172+
/// This property should be used in conjunction with <see cref="AgentRunOptions.ContinuationToken"/> to
173+
/// continue to poll for the completion of the response. Pass this token to
174+
/// <see cref="AgentRunOptions.ContinuationToken"/> on subsequent calls to <see cref="AIAgent.RunAsync(AgentThread?, AgentRunOptions?, System.Threading.CancellationToken)"/>
175+
/// to poll for completion.
176+
/// </para>
177+
/// </remarks>
178+
public object? ContinuationToken { get; set; }
179+
162180
/// <summary>
163181
/// Gets or sets the timestamp indicating when this response was created.
164182
/// </summary>
@@ -234,7 +252,7 @@ public AgentRunResponseUpdate[] ToAgentRunResponseUpdates()
234252
{
235253
extra = new AgentRunResponseUpdate
236254
{
237-
AdditionalProperties = this.AdditionalProperties
255+
AdditionalProperties = this.AdditionalProperties,
238256
};
239257

240258
if (this.Usage is { } usage)

0 commit comments

Comments
 (0)