Skip to content
This repository was archived by the owner on Jul 9, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
129 changes: 69 additions & 60 deletions docs/preview features/orchestrator.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,15 @@ Read more about Orchestrator [here](https://aka.ms/bf-orchestrator).
- State of the art - Extensive pre-trained transformer-based models for conversational applications
- Extremely fast inference time: Written in C++ for performance
- Cross-platform support: Runs on Windows, Linux and Mac OS X, including Cloud platforms
- Supports a [large number of languages](https://github.com/microsoft/botframework-sdk/blob/main/Orchestrator/docs/NLRModels.md#pretrained20210205microsoftdte0006unicoder_multilingualonnx) out of the box
- Free to use: Language models are deployed alongside your bot and runs in memory. No external services required

## Getting started

### Limitations
For this preview release, please be aware of these limitations:

1. Orchestrator has to be deployed on an x64 platform. If you are deploying your bot to a cloud service, please be aware that you need to publish your bot with a `win-x64` [runtime identifier](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish), or use Composer's built-in Azure deployment.

1. We are supporting English-only for this release.
1. It is recommended to deploy Orchestrator on the `x86-64` platform. On Azure, it is recommended to publish your bot application with the `win-x64` [runtime identifier](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish), or use Composer's built-in Azure deployment to do this automatically.

1. Orchestrator does not support entity extraction for this release. Any entities declared in your LU file will be ignored.

Expand All @@ -30,16 +29,24 @@ Please see the [roadmap](https://github.com/microsoft/botframework-sdk/blob/main


### Adding Orchestrator to a new bot in Composer
1. Enable Orchestrator from the preview features list:
Starting in Composer v1.4, Orchestrator is available as a downloadable package for your bot. To enable Composer's new Package Manager and Orchestrator preview features, please follow these steps:
1. Enable the 'New Creation Flow' from the preview features list:
- Navigate to `Application Settings` (
<img src="orchestrator-assets/application-settings.png" alt="drawing" width="30"/> icon)
- Enable Orchestrator preview feature under `Application Updates -> Preview features`

- Enable the `Application Updates -> Preview features`

<img src="orchestrator-assets/preview-flag-orchestrator.png" alt="preview flag orchestrator" width="600">

1. Create a new bot or open an existing bot project. Select a dialog in the bot project that you would like to try with Orchestrator. Select `Orchestrator recognizer` from the `Recognizer Type` drop-down menu.
1. Create a new bot in Composer (`Home -> New`). Select any bot template.

1. After the bot project has been initialized, navigate to the `Package Manager` tab. Download and install the `Microsoft.Bot.Components.Orchestrator` package.

<img src="orchestrator-assets/package-manager.png" alt="package manager">

<img src="orchestrator-assets/orchestrator-recognizer-dropdown.png" alt="recognizer dropdown options" width="300">
1. After the package has been successfully installed, navigate back to the `Design` tab. Select a dialog in the bot project that you would like to try with Orchestrator. Select `Orchestrator recognizer` from the `Recognizer Type` drop-down menu.

<img src="orchestrator-assets/orchestrator-recognizer-dropdown.png" alt="recognizer dropdown options" width="300">

1. Create an intent trigger
- Click on the waffle menu on a dialog to add a new trigger
Expand All @@ -53,6 +60,14 @@ Please see the [roadmap](https://github.com/microsoft/botframework-sdk/blob/main
- Add a response to the new `Intent Recognized` trigger to test that Orchestrator is working
- In the flow diagram for the newly created trigger, click on the `+` button and select `Send a Response` from the dropdown, then add a sample response in the `Language Generation` textbox.

1. Click the `Start Bot` button to start the bot locally and test the bot in the emulator. When the dialog is hit and Orchestrator is loaded successfully, test your bot in the Emulator. Orchestrator will show up in the logs as `Orchestrator Recognition`.
![orchestrator recognizer in emulator](orchestrator-assets/emulator.png)

## Publishing an Orchestrator-based bot to Azure
While it is possible to deploy your bot using your own scripts, it is strongly recommended to use Composer to publish your bot to Azure. Please see the documentation [here](https://docs.microsoft.com/en-us/composer/how-to-publish-bot).

The Composer deployment will create an S1 Tier 64-bit WebApp, bundle the Orchestrator language models, update bot settings to use these models, and upload the artifacts automatically to Azure.

1. Modify the runtime to support Orchestrator
1. Eject the runtime
1. Navigate to `Application Settings` (
Expand All @@ -61,68 +76,62 @@ Please see the [roadmap](https://github.com/microsoft/botframework-sdk/blob/main
![eject-runtime](orchestrator-assets/eject-runtime.png)
1. The runtime will be ejected into the `<Bot Project Folder>/runtime/azurewebapp` folder.

1. Perform these modifications to add Orchestrator libraries to your newly ejected runtime:
1. Open `<Bot Project Folder>/runtime/azurewebapp/Microsoft.BotFramework.Composer.WebApp.csproj`:

- Add `Microsoft.Bot.Builder.AI.Orchestrator` after `Microsoft.Bot.Builder.AI.Luis`
```xml
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="3.1.2" />
<PackageReference Include="Microsoft.Bot.Builder" Version="4.11.0" />
<PackageReference Include="Microsoft.Bot.Builder.AI.Luis" Version="4.11.0" />
<PackageReference Include="Microsoft.Bot.Builder.AI.Orchestrator" Version="4.11.0-preview" />
...
```
> [!Important]
> All packages starting with `Microsoft.Bot.Builder` must have the same version number. For example, if your ejected runtime `Microsoft.Bot.Builder` version is `4.11.0` like above, set the `Microsoft.Bot.Builder.AI.Orchestrator` version to `4.11.0-preview`.**

- Restrict platform target to the `x64` platform by adding these lines:
```xml
<PropertyGroup>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
```
1. Open `<Bot Project Folder>/runtime/azurewebapp/Startup.cs` and add these lines:
- Add the Orchestrator dependency after `Microsoft.Bot.Builder.AI.Luis`:
```csharp
using Microsoft.Bot.Builder.AI.Orchestrator;
```
- Add registration for Orchestrator after `ComponentRegistration.Add(new LuisComponentRegistration());`:
```csharp
ComponentRegistration.Add(new OrchestratorComponentRegistration());
```
1. Open `<Bot Project Folder>/runtime/core/ComposerSettingsExtensions.cs` and add these lines at the bottom of the file, before returning the `builder` object:
```csharp
var orchestratorSettingsPath = Path.GetFullPath(Path.Combine(botRoot, "generated", "orchestrator.settings.json"));
var orchestratorSettingsFile = new FileInfo(orchestratorSettingsPath);
if (orchestratorSettingsFile.Exists)
{
builder.AddJsonFile(orchestratorSettingsFile.FullName, optional: false, reloadOnChange: true);
}
```
1. Return back to Composer. Click the `Start Bot` button to start the bot locally and test the bot in the emulator. When the dialog is hit and Orchestrator is loaded successfully, Orchestrator will show up in the logs as `Orchestrator Recognition`.
## Adding Orchestrator to an Existing Bot
You can add Orchestrator to a bot created prior to Composer v1.4 by following these steps:

![orchestrator recognizer in emulator](orchestrator-assets/emulator.png)
1. Perform these modifications to add Orchestrator libraries to your newly ejected runtime:
1. Open `<Bot Project Folder>/runtime/azurewebapp/Microsoft.BotFramework.Composer.WebApp.csproj`:

## Publishing an Orchestrator-based bot to Azure
While it is possible to deploy your bot using your own scripts, it is strongly recommended to use Composer to publish your bot to Azure. Please see the documentation [here](https://docs.microsoft.com/en-us/composer/how-to-publish-bot).

The Composer deployment will create an S1 Tier 64-bit WebApp, bundle the Orchestrator language models, update bot settings to use these models, and upload the artifacts automatically to Azure.
- Add `Microsoft.Bot.Builder.AI.Orchestrator` after `Microsoft.Bot.Builder.AI.Luis`
```xml
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="3.1.2" />
<PackageReference Include="Microsoft.Bot.Builder" Version="4.11.0" />
<PackageReference Include="Microsoft.Bot.Builder.AI.Luis" Version="4.11.0" />
<PackageReference Include="Microsoft.Bot.Builder.AI.Orchestrator" Version="4.11.0-preview" />
...
```
> [!Important]
> All packages starting with `Microsoft.Bot.Builder` must have the same version number. For example, if your ejected runtime `Microsoft.Bot.Builder` version is `4.11.0` like above, set the `Microsoft.Bot.Builder.AI.Orchestrator` version to `4.11.0-preview`.**

- Restrict platform target to the `x64` platform by adding these lines:
```xml
<PropertyGroup>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
```
1. Open `<Bot Project Folder>/runtime/azurewebapp/Startup.cs` and add these lines:
- Add the Orchestrator dependency after `Microsoft.Bot.Builder.AI.Luis`:
```csharp
using Microsoft.Bot.Builder.AI.Orchestrator;
```
- Add registration for Orchestrator after `ComponentRegistration.Add(new LuisComponentRegistration());`:
```csharp
ComponentRegistration.Add(new OrchestratorComponentRegistration());
```
1. Open `<Bot Project Folder>/runtime/core/ComposerSettingsExtensions.cs` and add these lines at the bottom of the file, before returning the `builder` object:
```csharp
var orchestratorSettingsPath = Path.GetFullPath(Path.Combine(botRoot, "generated", "orchestrator.settings.json"));
var orchestratorSettingsFile = new FileInfo(orchestratorSettingsPath);
if (orchestratorSettingsFile.Exists)
{
builder.AddJsonFile(orchestratorSettingsFile.FullName, optional: false, reloadOnChange: true);
}
```

1. Finally, update the bot schema:
```ps1
cd Shemas
./update-schema.ps1
```
1. Return back to Composer. Re-open your bot project. The `Orchestrator Recognizer` will now be available in the dialog dropdown.

## Troubleshooting guide

1. When I use Orchestrator, clicking `Start Bot` in Composer, or deploying to Azure takes a lot longer. Why?

Orchestrator runs completely offline. Its language models are anywhere from 200MB-300MB, and they are automatically downloaded and packaged with your bot.

1. I am getting a `BadImageFormatException` when I click `Start Bot`.

Orchestrator has to run on a 64-bit platform. A `BadImageFormatException` either indicates that a 32-bit dotnet SDK was used to compile the bot, or that the bot was run on a 32-bit or unsupported platform.

Please *uninstall* any 32-bit dotnet SDKs that are on your local machine. Download the latest [v3.1 x64 SDK](https://dotnet.microsoft.com/download/dotnet-core/3.1) for your platform.

Finally, be sure that you're deploying and running your bot on a 64-bit capable machine. If you've deployed your bot via Composer to Azure, this is automatically taken care of for you.

1. How do I run Orchestrator on a Node.js-based bot?

Please see the instructions [here](https://aka.ms/bf-orchestrator#in-a-javascript-bot). Then eject a `Node.js` runtime instead of a `C#` one.
Expand Down