Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix python generation when custom files and templates are specified #9572

Merged
merged 2 commits into from
Jun 2, 2021
Merged

Fix python generation when custom files and templates are specified #9572

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea0a8bb
fix python generation when custom files and templates are specified
shylasrinivas May 24, 2021
2b1bcb0
add test for processUserDefinedTemplates
shylasrinivas May 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/generators/tiny-cpp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ These options may be applied as additional-properties (cli) or configOptions (pl
|Maps|✗|ToolingExtension
|CollectionFormat|✓|OAS2
|CollectionFormatMulti|✓|OAS2
|Enum|✓|OAS2,OAS3
|Enum|✗|OAS2,OAS3
|ArrayOfEnum|✓|ToolingExtension
|ArrayOfModel|✓|ToolingExtension
|ArrayOfCollectionOfPrimitives|✓|ToolingExtension
Expand Down
2 changes: 1 addition & 1 deletion modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultGenerator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -938,7 +938,7 @@ private void processUserDefinedTemplates() {
// TODO: initial behavior is "merge" user defined with built-in templates. consider offering user a "replace" option.
if (userDefinedTemplates != null && !userDefinedTemplates.isEmpty()) {
Map<String, SupportingFile> supportingFilesMap = config.supportingFiles().stream()
.collect(Collectors.toMap(TemplateDefinition::getTemplateFile, Function.identity()));
.collect(Collectors.toMap(TemplateDefinition::getTemplateFile, Function.identity(), (oldValue, newValue) -> oldValue));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this lambda doing? Why is it returning the oldValue?
I have no way to tell from this PR if this code works or not.
If this feature is changed in the future, we will not have any test that fails and the feature will break.
How about adding a test of processUserDefinedTemplates to ensure that this feature will keep working?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spacether Absolutely valid points.
Collectors.toMap throws IllegalStateException when dup keys are found, hence the logic is to return a single value when dups are encountered. oldValue is an existing key in the map, whereas newValue is a key found during later iterations of the list (in this case config.supportingFiles()). The code simply retains the existing key (oldValue) in the map when a dup (newValue) is encountered. Based on my testing, I have found oldValue to be user defined templates, which is desired to be retained in the map.
I will look into adding a test. Agreed there is no way to tell if the code works, how do you typically ensure PR quality?

Copy link
Contributor

@spacether spacether May 25, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for explaining what this code is doing.
Most of our most basic tests are in: https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/test/java/org/openapitools/codegen/DefaultCodegenTest.java
We typically add tests that can avoid generating a new output client/server sample if we can. If you have to, then generating a new sample is okay too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spacether Thank you for the pointers. I have added a test, but it looks like the config (line 684 of DefaultCodegenTest.java) I want to specify is not supported in CodegenConfigurator unless, I am missing something. Lmk if the test looks ok. I can remove the commented check based on your input.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for explaining that and adding the test. Your test looks good.


// TemplateFileType.SupportingFiles
userDefinedTemplates.stream()
Expand Down
59 changes: 58 additions & 1 deletion modules/openapi-generator/src/test/java/org/openapitools/codegen/DefaultGeneratorTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -662,5 +662,62 @@ public void testHandlesTrailingSlashInServers() {
Assert.assertEquals(servers.get(1).url, "http://trailingshlash.io:80/v1");
Assert.assertEquals(servers.get(2).url, "http://notrailingslash.io:80/v2");
}
}

@Test
public void testProcessUserDefinedTemplatesWithConfig() throws IOException {
Path target = Files.createTempDirectory("test");
Path templates = Files.createTempDirectory("templates");
File output = target.toFile();
try {
// Create custom template
File customTemplate = new File(templates.toFile(), "README.mustache");
new File(customTemplate.getParent()).mkdirs();
Files.write(customTemplate.toPath(),
"# {{someKey}}".getBytes(StandardCharsets.UTF_8),
StandardOpenOption.CREATE);

final CodegenConfigurator configurator = new CodegenConfigurator()
.setGeneratorName("python")
.setInputSpec("src/test/resources/3_0/petstore.yaml")
.setPackageName("io.something")
.setTemplateDir(templates.toAbsolutePath().toString())
.addAdditionalProperty("files", "src/test/resources/sampleConfig.json:\n\t folder: supportingjson "+
"\n\t destinationFilename: supportingconfig.json \n\t templateType: SupportingFiles")
.setSkipOverwrite(false)
.setOutputDir(target.toAbsolutePath().toString());

final ClientOptInput clientOptInput = configurator.toClientOptInput();
DefaultGenerator generator = new DefaultGenerator(false);

generator.setGeneratorPropertyDefault(CodegenConstants.MODELS, "true");
generator.setGeneratorPropertyDefault(CodegenConstants.MODEL_DOCS, "false");
generator.setGeneratorPropertyDefault(CodegenConstants.APIS, "true");
generator.setGeneratorPropertyDefault(CodegenConstants.SUPPORTING_FILES, "true");
generator.setGeneratorPropertyDefault(CodegenConstants.API_DOCS, "false");
generator.setGeneratorPropertyDefault(CodegenConstants.MODEL_TESTS, "false");
generator.setGeneratorPropertyDefault(CodegenConstants.API_TESTS, "false");

List<File> files = generator.opts(clientOptInput).generate();

// remove commented code based on review - files does not seem to be supported in CodegenConfigurator
// supporting files sanity check
// TestUtils.ensureContainsFile(files, output, "sampleConfig.json");
// Assert.assertTrue(new File(output, "sampleConfig.json").exists());

// Generator should report api_client.py as a generated file
TestUtils.ensureContainsFile(files, output, "io/something/api_client.py");

// Generated file should exist on the filesystem after generation
File apiClient = new File(output, "io/something/api_client.py");
Assert.assertTrue(apiClient.exists());

// Generated file should contain our custom packageName
TestUtils.assertFileContains(apiClient.toPath(),
"from io.something import rest"
);
} finally {
output.delete();
templates.toFile().delete();
}
}
}