[K6 Generator] various enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #11106

mostafa · 2021-12-13T14:45:21Z

This PR builds on top of the PR #10614 by @ideas-into-software and fixes an issue I found while reviewing the code, and that is the empty lines in the output script.

@agilob @wing328 @ideas-into-software I'd be happy to have your feedback.

PR checklist

Read the contribution guidelines.
Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
Run the following to build the project and update samples:
```
./mvnw clean package 
./bin/generate-samples.sh
./bin/utils/export_docs_generators.sh
```
Commit all changed files.
This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
These must match the expectations made by your contribution.
You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
For Windows users, please run the script in Git BASH.
File the PR against the correct branch: master (5.3.0), 6.0.x
If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

* request body example data extraction * request grouping and ordering * response visibility * request data extraction for chaining requests Signed-off-by: Michael H. Siemaszko <[email protected]>

- regenerated samples Signed-off-by: Michael H. Siemaszko <[email protected]>

* resolved merge conflicts * replaced deprecated dependency `org.apache.commons.lang3.StringEscapeUtils` Signed-off-by: Michael H. Siemaszko <[email protected]>

agilob · 2021-12-13T18:07:23Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

@@ -506,33 +634,47 @@ public void preprocessOpenAPI(OpenAPI openAPI) {
                                break;
                        }
                    }
-                } catch (NullPointerException e) {
+                } catch (NullPointerException ignored) {


let's log this exception and learn what's broken

agilob · 2021-12-13T18:08:43Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenOperation.java

@@ -70,6 +70,10 @@ private static boolean nonempty(List<?> params) {
        return params != null && params.size() > 0;
    }

+    private static boolean nonempty(Map<?, ?> params) {


agilob · 2021-12-13T18:10:02Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+
+                if (cgOperation.getHasVendorExtensions()
+                        && cgOperation.vendorExtensions.containsKey(X_OPERATION_DATAEXTRACT)
+                        && cgOperation.vendorExtensions.get(X_OPERATION_DATAEXTRACT) instanceof java.util.Map) {


instanceof java.util.Map, don't we have Map already imported to do intanceof Map?

That's a boolean check.

agilob · 2021-12-13T18:13:19Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+            Map<String, DataExtractSubstituteParameter> dataExtractSubstituteParams, HTTPRequestGroup requestGroup) {
+
+        if (!dataExtractSubstituteParams.isEmpty()) {
+            List<String> existingVariablesNames = requestGroup.variables.stream().map(v -> v.key)


List<String> existingVariablesNames = requestGroup.variables.stream().map(v -> v.key) .collect(Collectors.toList()); Set<DataExtractSubstituteParameter> initializeVariables = dataExtractSubstituteParams.values().stream() .filter(p -> !existingVariablesNames.contains(toVarName(p.paramName))).collect(Collectors.toSet());

this filter condition will be O(1) fast if existingVariablesNames is a Set, when existingVariablesNames is a List filter will be O(n)

agilob · 2021-12-13T18:13:51Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+            Set<DataExtractSubstituteParameter> initializeVariables = dataExtractSubstituteParams.values().stream()
+                    .filter(p -> !existingVariablesNames.contains(toVarName(p.paramName))).collect(Collectors.toSet());
+
+            if (!initializeVariables.isEmpty()) {


check if (!initializeVariables.isEmpty()) { not required when using for-each loop

agilob · 2021-12-13T18:14:14Z

samples/client/petstore/k6/script.js

-        check(request, {
+        {
+          let url = BASE_URL + `/pet/findByStatus?status=${status}`;
+


still triple empty lines :P

I think I haven't regenerated the file.

Interesting, the pipeline passed on 5082527 when the sample project was out of date.

That was strange to me, too.

agilob · 2021-12-13T18:14:29Z

samples/client/petstore/k6/script.js

+
+
+          let request = http.del(url);
+


4 empty lines!

agilob · 2021-12-13T18:16:23Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+
+                String operationId = operation.getOperationId();
+
+                boolean hasRequestBodyExample = false;


initialisation of this can go to line 558

An outer block needs this variable if I move it inside the if block.

agilob · 2021-12-13T18:17:05Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+
+                final CodegenOperation cgOperation = super.fromOperation(path, method.name(), operation, null);
+
+                String operationId = operation.getOperationId();


this line can go a lot lower than here

agilob · 2021-12-13T18:17:36Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

@@ -416,26 +512,54 @@ public void preprocessOpenAPI(OpenAPI openAPI) {

                final Operation operation = methodOperation.getValue();
                final PathItem.HttpMethod method = methodOperation.getKey();
+                OptionalInt operationGroupingOrder = OptionalInt.empty();


had no idea OptionalInt is a thing 👍

agilob · 2021-12-13T18:23:13Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+            List<Parameter> bodyOrFormParams) {
+
+        Optional<Map.Entry<String, Example>> requestBodyExampleEntry = requestBody.getContent().get(contentTypeValue)
+                .getExamples().entrySet().stream().findFirst();


this chain map->set->stream->optional is quite slow, how often is this method call? might be worth to new ArrayList(getExamples().entrySet()).get(0)?

Only once per request. It doesn't matter much.

mostafa · 2021-12-14T10:38:19Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenOperation.java

@@ -66,7 +66,11 @@
     *
     * @return true if parameter exists, false otherwise
     */
-    private static boolean nonempty(List<?> params) {
+    private static boolean nonEmpty(List<?> params) {


@agilob This is unrelated to our code, but I changed it nevertheless.

…nhancements

agilob · 2021-12-14T12:36:30Z

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/K6ClientCodegen.java

+            for (Map.Entry<?, ?> respExtensionEntry : respExtensions.entrySet()) {
+
+                if (X_OPERATION_RESPONSE_HIDE.equals(String.valueOf(respExtensionEntry.getKey()))) {
+                    hideOperationResponse = Boolean.parseBoolean(respExtensionEntry.getValue().toString());


should this have a break statement after setting this to true? effectively, now hideOperationResponse will have the last value of respExtensionEntry.getValue(), and if you want the last value, then we don't need a loop here, and since we're iterating on respExtensions.entrySet() a Set, the last value is not deterministic, so method might produce different value on every run.

agilob

nice, thanks 👍

mostafa · 2021-12-14T13:46:40Z

@agilob Thanks to you for the meticulous review! Finally mergeable?

agilob · 2021-12-14T18:07:27Z

There is no technical commette for k6, so it now depends on when @wing328 or @jimschubert can merge it

wing328 · 2021-12-17T02:17:21Z

bin/configs/k6.yaml

+inputSpec: modules/openapi-generator/src/test/resources/3_0/petstore-with-fake-endpoints-models-for-testing.yaml
+templateDir: modules/openapi-generator/src/main/resources/k6
+additionalProperties:
+  appName: PetstoreClient


There's already one in ./bin/configs/other/k6.yaml.

I'll remove it after merging this PR.

DevMrRober · 2024-07-03T11:48:35Z

Hello, good afternoon,

I have detected that in the version change from 5.3.0 to 5.3.1 there is a bug that is dragging to the latest current version v7.7.0.

The bug itself is to generate more than 2 methods per path, ie if I have a path "/users" with a "GET, POST, PUT, PATCH" I will only generate 2 of the methods leaving the others.

I pass code

V5.3.0

group("/users/{userId}", () => {
let locale = 'TODO_EDIT_THE_LOCALE'; // specify value as there is no example value for this parameter in OpenAPI spec
let userId = 'TODO_EDIT_THE_USERID'; // specify value as there is no example value for this parameter in OpenAPI spec
let url = BASE_URL + /users/${userId}?locale=${locale};
// Request No. 1
let request = http.get(url);
check(request, {
"User found. Resource obtained successfully.": (r) => r.status === 200
});
sleep(SLEEP_DURATION);

    // Request No. 2
    // TODO: edit the parameters of the request body.
    body = {"user": {"ids": "useridmodel", "firstName": "string", "lastName": "string", "email": "string", "dateOfBirth": "date", "isEmailVerified": "boolean", "creationDate": "date"}};
    params = {headers: {"Content-Type": "application/json"}};
    request = http.put(url, body, params);
    check(request, {
        "User full information updated.": (r) => r.status === 200
    });
    sleep(SLEEP_DURATION);

    // Request No. 3
    request = http.del(url);
    check(request, {
        "Resource deleted successfully.": (r) => r.status === 204
    });
    sleep(SLEEP_DURATION);

    // Request No. 4
    // TODO: edit the parameters of the request body.
    body = {"user": {"ids": "useridmodel", "firstName": "string", "lastName": "string", "email": "string", "dateOfBirth": "date", "isEmailVerified": "boolean", "creationDate": "date"}};
    params = {headers: {"Content-Type": "application/json"}};
    request = http.patch(url, body, params);
    check(request, {
        "User partial information updated successfully.": (r) => r.status === 200
    });
    sleep(SLEEP_DURATION);
});


V5.3.1

group("/users/{userId}", () => {
let locale = 'TODO_EDIT_THE_LOCALE'; // specify value as there is no example value for this parameter in OpenAPI spec
let userId = 'TODO_EDIT_THE_USERID'; // specify value as there is no example value for this parameter in OpenAPI spec

    // Request No. 1
    {
        let url = BASE_URL + `/users/${userId}?locale=${locale}`;
        let request = http.get(url);

        check(request, {
            "User found. Resource obtained successfully.": (r) => r.status === 200
        });

        sleep(SLEEP_DURATION);
    }

    // Request No. 2
    {
        let url = BASE_URL + `/users/${userId}?locale=${locale}`;
        // TODO: edit the parameters of the request body.
        let body = {"user": {"ids": "useridmodel", "firstName": "string", "lastName": "string", "email": "string", "dateOfBirth": "date", "isEmailVerified": "boolean", "creationDate": "date"}};
        let params = {headers: {"Content-Type": "application/json", "Accept": "application/json"}};
        let request = http.patch(url, JSON.stringify(body), params);

        check(request, {
            "User partial information updated successfully.": (r) => r.status === 200
        });
    }
});


im using this commands:

docker run --rm -v "${PWD}:/local" openapitools/openapi-generator-cli:v5.3.0 generate -i /local/users.swagger.v2.0.0.json -g k6 -o /local/k6-test/ --skip-validate-spec

docker run --rm -v "${PWD}:/local" openapitools/openapi-generator-cli:v5.3.1 generate -i /local/users.swagger.v2.0.0.json -g k6 -o /local/k6-test/ --skip-validate-spec

ideas-into-software and others added 6 commits October 17, 2021 20:14

Further K6 OpenAPI generator enhancements

67eac90

* request body example data extraction * request grouping and ordering * response visibility * request data extraction for chaining requests Signed-off-by: Michael H. Siemaszko <[email protected]>

Further K6 OpenAPI generator enhancements

809dadd

- regenerated samples Signed-off-by: Michael H. Siemaszko <[email protected]>

Further K6 OpenAPI generator enhancements

ed3f00c

* resolved merge conflicts * replaced deprecated dependency `org.apache.commons.lang3.StringEscapeUtils` Signed-off-by: Michael H. Siemaszko <[email protected]>

Fix suggested changes by linter

66dc3ed

Fix extra spaces in the template

a79d0c7

Merge branch 'master' into k6-openapi-extract-examples-enhancements

5082527

agilob reviewed Dec 13, 2021

View reviewed changes

mostafa added 6 commits December 14, 2021 10:19

Log exception

5d28180

Rename function signature to camelCase

d9e10b8

Address comments on Big-O

6e106df

Move declaration of variable near the usage

4cb1819

Add config file for generating k6 script

fd19b33

Regenerate k6 script

a42d924

mostafa commented Dec 14, 2021

View reviewed changes

mostafa added 2 commits December 14, 2021 12:10

Merge branch 'OpenAPITools:master' into k6-openapi-extract-examples-e…

86ffce3

…nhancements

Regenerate samples

070f730

agilob reviewed Dec 14, 2021

View reviewed changes

mostafa added 2 commits December 14, 2021 14:36

Fix predicate

8788910

Fix missing import

e79063b

agilob approved these changes Dec 14, 2021

View reviewed changes

wing328 reviewed Dec 17, 2021

View reviewed changes

wing328 added the Client: k6 label Dec 17, 2021

wing328 added this to the 5.3.1 milestone Dec 17, 2021

wing328 added the Enhancement: General label Dec 17, 2021

wing328 merged commit fedc54a into OpenAPITools:master Dec 17, 2021

wing328 mentioned this pull request Dec 17, 2021

[ K6 Generator ] Further K6 OpenAPI generator enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #10614

Closed

5 tasks

mostafa deleted the k6-openapi-extract-examples-enhancements branch December 17, 2021 09:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[K6 Generator] various enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #11106

[K6 Generator] various enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #11106

mostafa commented Dec 13, 2021 •

edited

Loading

agilob Dec 13, 2021 •

edited

Loading

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021 •

edited

Loading

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

mostafa Dec 14, 2021

agilob Dec 14, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

agilob Dec 13, 2021

agilob Dec 13, 2021

mostafa Dec 14, 2021

mostafa Dec 14, 2021

agilob Dec 14, 2021

mostafa Dec 14, 2021

agilob left a comment

mostafa commented Dec 14, 2021 •

edited

Loading

agilob commented Dec 14, 2021

wing328 Dec 17, 2021

DevMrRober commented Jul 3, 2024 •

edited

Loading


		String operationId = operation.getOperationId();

		boolean hasRequestBodyExample = false;


		final CodegenOperation cgOperation = super.fromOperation(path, method.name(), operation, null);

		String operationId = operation.getOperationId();

[K6 Generator] various enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #11106

[K6 Generator] various enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #11106

Conversation

mostafa commented Dec 13, 2021 • edited Loading

PR checklist

agilob Dec 13, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agilob Dec 13, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agilob left a comment

Choose a reason for hiding this comment

mostafa commented Dec 14, 2021 • edited Loading

agilob commented Dec 14, 2021

Choose a reason for hiding this comment

DevMrRober commented Jul 3, 2024 • edited Loading

mostafa commented Dec 13, 2021 •

edited

Loading

agilob Dec 13, 2021 •

edited

Loading

agilob Dec 13, 2021 •

edited

Loading

mostafa commented Dec 14, 2021 •

edited

Loading

DevMrRober commented Jul 3, 2024 •

edited

Loading