Deployment CLI tools

[zidaneymar]

08/28 Update:

Refer to #699

create.ps1

• Accept a bot name, environment name (single word, all lowercase), 16-character password, an Azure region (location), and a LUIS authoring key as parameters
• For any missing parameter, prompt the user interactively for the value
• for LUIS authoring key from EU, make sure LUIS apps are created only in EU regions.
• create all resources within the same Azure region
• Create descriptive names for all resources based on the bot name - for example, (botname)-(environment)-webapp (botname)-(environment)-group (botname)-(environment)-serviceplan
• Create app registration
• capture the appID resulting from the app registration, report it back to the user on console and log it
• create a new Resource Group
• create App Service plan
• create web app service
• create CosmosDB
• create Azure Storage
• Upon any error, display the error in full detail, and stop process
  • capture common error: access denied
  • capture common error: resource already exists
• create LUIS Cognitive Service key, in given bot region
• create LUIS apps for specified environment and capture resulting app IDs
  • LUIS app names should be generated in the same way they are inside Composer, botname(environment)-dialogname
  • if environment already created inside Composer, use existing LUIS apps
  • P0 assume Composer ran its language creation and orchestration process. The create and publish scripts should use LUIS JSON flies from the generated folder
  • P1 Extract the orcanstration for managed LU process from Composer into a CLI lib that can be used from the script and run directly on LU files
• assign LUIS Cognitive Service keys to LUIS apps
• Run the necessary ludown/luparse commands to process the .lu files
• Publish all apps to LUIS
• If error encountered in publish to luis, stop script and display error message
• Write values to appsettings.json in appropriate place
• Upon success, display the resulting values, including: app id, app password, resource group name, web app name, web app API endpoint url

deploy.ps1

• Accept bot name, environment name, and LUIS authoring key as parameters
• Run the necessary ludown/luparse commands to process the .lu files
• Publish to LUIS
• If error encountered in publish to luis, stop script and display error message
• Make any necessary changes to appsettings.json (add additional LUIS app ids?)
• Create zipfile of project folder
• Deploy the zipfile to web app hosting your bot
• If an error is encountered in deploy of zipfile, display error message

Usage:

.\Scripts\create.ps1 -name TestBotProject0828 -location westus -appPassword qikangqikang123$ -environment testenv

.\Scripts\deploy.ps1 -name TestBotProject0828 -environment testenv -botPath ....\SampleBots\ToDoBot

For now it doesn't include luis publishing.

---

Deprecated:

Deployment Process

• Deploy process (first time to deploy by using “deploy.ps1” script))

• First time to deploy a bot runtime and create a bunch of azure resources (channel registration, web app bot) on azure
  Deploy Azure Resource
  Publish bot runtime (.dll ) and dialog files (including .lu and .lg)
• Instruction :
  1. Open PowerShell console(at least PowerShell 6.0).
  2. Run “az login”
  3. Run “az account set --subscription ""” to set the default subscription, you can use “az account list” command to show all subscriptions.
  4. Navigate to the BotFramework-Composer\BotProject\CSharp
  5. .\Scripts\deploy.ps1 -name TestBotProject0823 -resourceGroup TestBotProject0823 -botPath ....\SampleBots\Cards_Samples -location westus -appPassword qikangqikang123$

Name: the web app bot name
ResourceGroup: the resource group name
BotPath: the bot you want to deploy, this bot will be set as the start up bot
Location: the location of the resource
AppPassword: the app password of your bot resource, the password should be valid

Publish process (update bot file “publish.ps1” script)

• If you have already have a bot runtime instance on Azure, for example if you have run the “deploy.ps1” script before, and you want to update the bot file, then use the “publish.ps1” script.
  Publish bot runtime (.dll ) and dialog files (including .lu and .lg)
• Instruction
  1. Open PowerShell console(at least PowerShell 6.0).
  2. Run “az login”
  3. Run “az account set --subscription ""” to set the default subscription, you can use “az account list” command to show all subscriptions.
  4. Navigate to the BotFramework-Composer\BotProject\CSharp
  5. .\Scripts\publish.ps1 -name TestBotProject0823 -resourceGroup TestBotProject0823 -botPath ....\SampleBots\ToDoBot

for luis settting, need more discussions.

Task Item

#699

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Code refactor (non-breaking change which improve code quality, clean up, add tests, etc)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Doc update (document update)

Checklist

• I have added tests that prove my fix is effective or that my feature works
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have functionally tested my change

Screenshots

Please include screenshots or gifs if your PR include UX changes.

[benbrown]

I have done some thinking about this as well, here is the doc. Our thinking is closely aligned - one script for creating resources, another for doing repeated deploys.

I would like to minimize the number of parameters required from a user -- I believe we can do this with just a bot name and a 16-character password. The resource names can be generated based on the bot name.

https://microsoft-my.sharepoint-df.com/:w:/r/personal/benbro_microsoft_com1/_layouts/15/doc2.aspx?sourcedoc=%7Baf893fd0-e221-43b5-b675-6310c0964af5%7D&action=editnew&wdPreviousSession=2a3d7cc2%2D4d58%2D42cf%2Db3c6%2Df424620014dd&wdNewAndOpenCt=1566329763229&wdo=4&wdOrigin=wacFileNew&wdTpl=blank&wdLcid=1033&wdPreviousCorrelation=017e5ba8%2D3c67%2D4c98%2Dbfa5%2De6a8a689d1a2&cid=b730d061-90cf-401c-8144-c6513795a5ce

Refer to the details in this ticket:
#699

[cwhitten]

@benbrown @yochay @zidaneymar We knew this going in, but got additional feedback from SDK & Tools group that the Powershell script here will cause friction that we'll need to address eventually. We should consider porting this script to run from a node runtime early. Is there a specific reason this is out of scope?

[yochay]

The main reason for keeping this out of scope is to leverage exiting, already working solutions, in the form of the VA scripts. I talked to Lauren and we agree we can update both project based on Composer needs. Another reasons, is the lack of JS runtime will cause even a larger friction, specially since this design calls for choosing your runtime during new bot creation. With that said, PowerShell 6 runs on Mac and is currently the only ship vehicle for Virtual Assistant.

I agree that when Composer GA and fully support both JS and C# bot runtime, we need to provide a more generic CLI experience. Could be in the form of node, or native Az CLI, which is Python

[luhan2017]

RunningInstance [](start = 70, length = 15)

wenyi is working on editing environment variables in composer and post them to bot project, how could those settings passed to this deployment script??

[zidaneymar]

now the script search the appsettings.json file in bot and merge with the main appsettings.json in BotProject root folder.

[yochay]

@zidaneymar , this is great. I'll test the scripts shortly. Let's keep this item and #699 open until we complete the work around for provisioning LUIS keys, having Adaptive work with Cosmos and add support for storage middleware (for transcript logging).

[benbrown]

After running the create script, my appsettings.json file is updated with the app id and secret, but the resource group, name, location and other fields are set to null:

{
  "microsoftAppId": "xxxxxx",
  "microsoftAppPassword": "somepasswwd123!",
  "bot": "RunningInstance",
  "name": null,
  "properties": null,
  "id": null,
  "resourceGroup": null,
  "location": null,
  "type": null
}

Is this intentional? I think it makes sense to write these values to the file.

[benbrown]

DEPLOY:

• Once the deployable runtime is ready and we can create new projects with the runtime code, composer files and these scripts all together, the deploy script should be updated to deprecate the botPath parameter in favor of using the local folder
• There is a typo in the prompt for path: Reletive should be Relative
• The deploy script should output a success message if the deploy was successful.
• The deploy script should catch failed deploys. There is no error message for invalid bot name or invalid environment name.

CREATE:

• The resources all need to include the environment name. Currently only the group name includes the environment.

[zidaneymar]

DEPLOY:

• Once the deployable runtime is ready and we can create new projects with the runtime code, composer files and these scripts all together, the deploy script should be updated to deprecate the botPath parameter in favor of using the local folder
• There is a typo in the prompt for path: Reletive should be Relative
• The deploy script should output a success message if the deploy was successful.
• The deploy script should catch failed deploys. There is no error message for invalid bot name or invalid environment name.

CREATE:

• The resources all need to include the environment name. Currently only the group name includes the environment.

Fixed in #833