Skip to content

Commit b98393b

Browse files
leniatghLeiwang3SQL
authored andcommitted
update api scenario documentation (Azure#16086)
* add operation step * fix schema * restore OpTest * add cleanUpSteps & remove type * use draft-07 schema * refine propertyNames * add description * remove merge * fix copy/move * use property name * variable type * restore plain style * define Example and file reference in v1.1 * remove raw Example support * fix * rename to API scenario * restore entry * fix ref
1 parent b2a1ea6 commit b98393b

File tree

19 files changed

+573
-496
lines changed

19 files changed

+573
-496
lines changed

documentation/test-scenario/how-to/QuickStart.md renamed to documentation/api-scenario/how-to/QuickStart.md

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,15 @@
1414
```sh
1515
npm install -g oav@latest
1616
```
17+
1718
### OAV Features
19+
1820
- Very easy to use and run.
1921
- Support postman collection format. Debug easily.
2022
- Request response validation. `oav` implement a powerful validation algorithm and help developer to detect service issue in the early phase.
21-
- Validation result report. After each run test scenario, developer will get a validation report which contains detect issue in api test.
23+
- Validation result report. After each run API scenario, developer will get a validation report which contains detect issue in api test.
2224
- Integrate everywhere. Easily integrate with azure-pipeline, cloud-test.
2325

24-
2526
## Create AAD app
2627

2728
To run API test, first please prepare an AAD app which is used for provisioning Azure resource. Please grant subscription contributor permission to this AAD app.
@@ -30,24 +31,24 @@ For how to create AAD app, please follow this doc https://docs.microsoft.com/en-
3031

3132
## Authoring steps
3233

33-
We will write test scenario file for SignalR service as an example.
34+
We will write API scenario file for SignalR service as an example.
3435

35-
#### 1. Write your first test scenario file
36+
#### 1. Write your first API scenario file
3637

37-
First, create a folder `scenarios` under the api version folder. All test scenario files under the `scenarios` folder should bind with the api version.
38+
First, create a folder `scenarios` under the api version folder. All API scenario files under the `scenarios` folder should bind with the api version.
3839

3940
![folder-structure](./folder-structure.png)
4041

41-
Now write your basic test scenario. For more detail about test scenario file format, please refer to
42-
[Test Scenario Definition Reference](../references/TestDefinitionReference.md).
42+
Now write your basic API scenario. For more detail about API scenario file format, please refer to
43+
[API Scenario Definition Reference](../references/ApiScenarioDefinition.md).
4344

4445
```yaml
45-
# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/documentation/test-scenario/references/v1.0/schema.json
46+
# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/documentation/api-scenario/references/v1.1/schema.json
4647

47-
contentVersion: 1.0.0
4848
scope: ResourceGroup
49-
testScenarios:
50-
- description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
49+
scenarios:
50+
- scenario: quickStart
51+
description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
5152
steps:
5253
- step: SignalR_CreateOrUpdate
5354
exampleFile: ../examples/SignalR_CreateOrUpdate.json
@@ -57,7 +58,7 @@ testScenarios:
5758
5859
#### 2. create your env file
5960
60-
The `env.json` file contains required test scenario variables such as, subscriptionId, AAD applicationId, AAD applicationSecret.
61+
The `env.json` file contains required API scenario variables such as, subscriptionId, AAD applicationId, AAD applicationSecret.
6162

6263
```json
6364
{
@@ -77,9 +78,9 @@ oav run /home/user/azure-rest-api-specs/specification/signalr/resource-manager/M
7778

7879
#### 4. Debug with postman
7980

80-
Sometimes the command `oav run` may fail due to non 2xx HTTP status code. Now you need to debug the test scenario with postman.
81+
Sometimes the command `oav run` may fail due to non 2xx HTTP status code. Now you need to debug the API scenario with postman.
8182

82-
When run `run`, it automatically generate postman collection and postman env in `generated/<providerNamespace>/<testScenarioFile>/<runId>/<testScenario>` folder. Here is the generated file folder structure. The `collection.json` and `env.json` is generated postman collection file and environment file. `202105120922-5c3x5` is current runId. For each run command it will generated unique runId.
83+
When run `run`, it automatically generate postman collection and postman env in `generated/<providerNamespace>/<apiScenarioFile>/<runId>/<apiScenario>` folder. Here is the generated file folder structure. The `collection.json` and `env.json` is generated postman collection file and environment file. `202105120922-5c3x5` is current runId. For each run command it will generated unique runId.
8384

8485
```
8586
generated
@@ -104,7 +105,7 @@ After you import postman collection, you will get such requests. Now you could d
104105
105106
#### 5. manual update example value
106107
107-
After debug with postman, you need to rewrite back all the updated values and run `oav run <test-scenario-file> -e <env.json>` again. The result should be successful.
108+
After debug with postman, you need to rewrite back all the updated values and run `oav run <api-scenario-file> -e <env.json>` again. The result should be successful.
108109
109110
## Feedback
110111

documentation/test-scenario/how-to/testScenarioWithARMTemplate.md renamed to documentation/api-scenario/how-to/apiScenarioWithARMTemplate.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Test scenario integrate with armTemplate
1+
# API scenario integrate with armTemplate
22

33
## Background
44

@@ -13,7 +13,7 @@ Here is an example about `generate unique resource name for signalR service`
1313

1414
#### Generate unique resource name
1515

16-
We use `armTemplate output` to overwrite `resourceName` variable and following `createResource` step will use this variable. Below is generate unique name armTemplate. This armTemplate output `resourceName` variables, so test scenario following step will using the output variable.
16+
We use `armTemplate output` to overwrite `resourceName` variable and following `createResource` step will use this variable. Below is generate unique name armTemplate. This armTemplate output `resourceName` variables, so API scenario following step will using the output variable.
1717

1818
```json
1919
{
@@ -38,7 +38,7 @@ We use `armTemplate output` to overwrite `resourceName` variable and following `
3838
}
3939
```
4040

41-
After we have this armTemplate, we could define current test scenario file. We defined `resourceName` variable globally. `./generate_unique_string.json` is armTemplate.
41+
After we have this armTemplate, we could define current API scenario file. We defined `resourceName` variable globally. `./generate_unique_string.json` is armTemplate.
4242

4343
`SignalR_CreateOrUpdate.json`
4444

@@ -80,8 +80,9 @@ After we have this armTemplate, we could define current test scenario file. We d
8080
scope: ResourceGroup
8181
variables:
8282
resourceName: ""
83-
testScenarios:
84-
- description: Microsoft.SignalRService/signalR CRUD
83+
scenarios:
84+
- scenario: quickStart
85+
description: Microsoft.SignalRService/signalR CRUD
8586
steps:
8687
- step: Generate_Unique_string
8788
armTemplateDeployment: ./generate_unique_string.json
File renamed without changes.

documentation/test-scenario/how-to/folder-structure.png renamed to documentation/api-scenario/how-to/folder-structure.png

File renamed without changes.

documentation/test-scenario/how-to/genTestScenario.gif renamed to documentation/api-scenario/how-to/genTestScenario.gif

File renamed without changes.
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Generate a basic API scenario file
2+
3+
## Prerequisite
4+
5+
We use `oav` tools to generate basic API scenario. `oav` analyze swagger file and use swagger example as API scenario steps. So first, you need to install the latest oav.
6+
7+
## Introduction
8+
9+
`oav` support rule based API scenario file generation. We use this command to generate API scenario file.
10+
11+
`oav generate-static-api-scenario --readme <readme> --tag <tag> --rules <generated-rules>`
12+
13+
- readme: swagger readme file.
14+
- tag: which tag to generate. oav will analyze swagger file under the tag and generate API scenario.
15+
- rules: Currently support two types. `resource-put-delete`, `operations-list`. Default: `resource-put-delete`
16+
- `resource-put-delete`: generate resource put and delete API scenario.
17+
- `operations-list`: generate operations list API scenario. `operations-list` is the simplest API which must be defined in swagger.
18+
19+
Example:
20+
21+
![](./genTestScenario.gif)
22+
23+
This command will load and analyze swagger and generate a basic API scenario file (`resource-put-delete`).
24+
25+
Result: the output contains two files
26+
27+
- scenarios/signalR.yaml: The API scenario file.
28+
- readme.test.md: The entry for SDK test generation
29+
30+
The generated API scenario file: The generated API scenario file contains two steps. Create signalR and delete it. It's a basic API scenario and developer can add more step based on the basic API scenario file.
31+
32+
```
33+
scope: ResourceGroup
34+
testScenarios:
35+
- description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
36+
steps:
37+
- step: SignalR_CreateOrUpdate
38+
exampleFile: ../examples/SignalR_CreateOrUpdate.json
39+
- step: SignalR_Delete
40+
exampleFile: ../examples/SignalR_Delete.json
41+
```
42+
43+
If you pass rule option `operations-list`, you will get such API scenario file.
44+
45+
```
46+
scope: ResourceGroup
47+
testScenarios:
48+
- description: operationsList
49+
steps:
50+
- step: operationsList
51+
exampleFile: ../examples/Operations_List.json
52+
53+
```
54+
55+
## Reference
56+
57+
- [oav](https://github.com/Azure/oav/tree/develop)

documentation/test-scenario/how-to/import-postman-collection.png renamed to documentation/api-scenario/how-to/import-postman-collection.png

File renamed without changes.

documentation/test-scenario/how-to/postman-collection-signalr.PNG renamed to documentation/api-scenario/how-to/postman-collection-signalr.PNG

File renamed without changes.
File renamed without changes.
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# API Scenario Documentation
2+
3+
API Scenario is a YAML file defining RESTful API usage scenarios of your service with a series of API calls. API scenario can be used for service functional test, API quality validation and SDK/CLIs test generation.
4+
5+
_**Caution**: This project is in early preview phase, hence breaking changes should be expected._
6+
7+
## Features
8+
9+
- Simple to use: Intuitive step definition based on Swagger examples and raw REST call.
10+
- ARM Template integration: Support creating external Azure resources with ARM Template and executing Azure Powershell or Azure CLI scripts with ARM Template deployment script.
11+
- Implementation independent: [oav](https://github.com/Azure/oav) is the default API scenario runner, and more runners will be supported, like SDKs in different languages.
12+
13+
### Demo gif
14+
15+
![demo](./how-to/runApiTest.gif)
16+
17+
## Quick start
18+
19+
- [Example: Write and run your first API scenario file](./how-to/QuickStart.md)
20+
- [Example: Generate a basic API scenario file](./how-to/generateABasicApiScenario.md)
21+
- [Example: use armTemplate to generate unique resourceName](./how-to/apiScenarioWithARMTemplate.md)
22+
- [API scenario file sample](../samplefiles/Microsoft.YourServiceName/stable/YYYY-MM-DD/scenarios/quickStart.yaml)
23+
24+
## References
25+
26+
- [API Scenario Definition Reference](./references/ApiScenarioDefinition.md)
27+
- [API Scenario Variable Definition Reference](./references/Variables.md)
28+
- [API Scenario Runner Reference](./references/Runner.md)
29+
- [API Scenario JSON Schema](./references/v1.1/schema.json)

0 commit comments

Comments
 (0)