Skip to content

Commit

Permalink
Generate features (#674)
Browse files Browse the repository at this point in the history 
* Add Generate Features prototype and implement the necessary interfaces.

Signed-off-by: Paul Gooderham <[email protected]>

* Add the generateFeatures command line and build file option.

Signed-off-by: Paul Gooderham <[email protected]>

* Add comment to existing server.xml and scan gradle classes dirs.

Signed-off-by: Paul Gooderham <[email protected]>

* Code cleanup.

Signed-off-by: Paul Gooderham <[email protected]>

* Make generateFeatures option command line only. Clean up streams code.

Signed-off-by: Paul Gooderham <[email protected]>

* Allow generateFeatures=false

* Add classFile option to GenerateFeaturesTask (#640)

Signed-off-by: Kathryn Kodama <[email protected]>

* Get binary scanner jar from connected repos

* Fix name of API method.

Signed-off-by: Paul Gooderham <[email protected]>

* Use gradle convention for latest release version

* Update message

* Change generated file name to generated-features.xml (#644)

Signed-off-by: Kathryn Kodama <[email protected]>

* Fix NPE and remove getAllServerFeatures

* Fix IO error when server is not yet setup.

Signed-off-by: Paul Gooderham <[email protected]>

* Quick fix

Signed-off-by: Paul Gooderham <[email protected]>

* Remove binary scanner jar parameter

* Handle optimize and non-optimize cases

* Add a system where the binary scanner handler detects an error generated
by the scanner and calls it again with fewer restrictions in order to
generate a list of suggested features that should work. This error
and suggested solution is then displayed to the user.

* Improved error reporting after code review.

Signed-off-by: Paul Gooderham <[email protected]>

* After writing server.xml ensure <server> element is on a new line.

Signed-off-by: Paul Gooderham <[email protected]>

* Move the xml document rewrite code into ci.common.

Signed-off-by: Paul Gooderham <[email protected]>

* Remove unused import.

Signed-off-by: Paul Gooderham <[email protected]>

* Use the binary scanner support in ci.common.

Signed-off-by: Paul Gooderham <[email protected]>

* Remove unused imports and class variable.

Signed-off-by: Paul Gooderham <[email protected]>

* Allow info messages to appear in output by default and without setting log level to Info.

Signed-off-by: Paul Gooderham <[email protected]>

* Get existing features for feature gen from the source config dir (#661)

* Get existing features for feature gen from the source config dir

Signed-off-by: Kathryn Kodama <[email protected]>

* Call generate features on dev mode startup and restart

Signed-off-by: Kathryn Kodama <[email protected]>

* Simplify getServerFeatureUtil method

Signed-off-by: Kathryn Kodama <[email protected]>

* Update to use latest snapshot from ci.common

Signed-off-by: Kathryn Kodama <[email protected]>

* Add binary scanner sonatype repo to dev test

Signed-off-by: Kathryn Kodama <[email protected]>

* Update sonatype repo link in dev tests

Signed-off-by: Kathryn Kodama <[email protected]>

* Diable feature generation in dev tests

Signed-off-by: Kathryn Kodama <[email protected]>

* Added optimize parameter for generating features

* Added optimize parameter to generateFeatures task

* Feature generation, create generated xml file with empty feature list (#664)

* When no features left to generate create generated xml file with empty feature list

Signed-off-by: Kathryn Kodama <[email protected]>

* Improve comment in generated-features.xml

Signed-off-by: Kathryn Kodama <[email protected]>

* Improve formatting for comments

Signed-off-by: Kathryn Kodama <[email protected]>

* Allow optimize to be passed in as a String parameter

* Use binary scanner Maven coordinates promoted by the scanner team.

Signed-off-by: Paul Gooderham <[email protected]>

* Fix compileJava call in recompileBuildFile

* Put snapshot repo on project level for dev tests

Signed-off-by: Kathryn Kodama <[email protected]>

* Updated debug statements

* Use new method downloadBuildArtifact() to search buildscript portion of build.gradle

Signed-off-by: Paul Gooderham <[email protected]>

* Exception handling for generateFeatures (#672)

* Exception handling for generateFeatures

Signed-off-by: Kathryn Kodama <[email protected]>

* Updated generateFeatures error message and generated-features.xml comment

Signed-off-by: Kathryn Kodama <[email protected]>

* Remove unnecessary imports and adjust copyright header

Signed-off-by: Kathryn Kodama <[email protected]>

* Update dev test project build.gradle

Signed-off-by: Kathryn Kodama <[email protected]>

* Improve comment for default generateFeatures behaviour

Signed-off-by: Kathryn Kodama <[email protected]>

* Improve exception handling message when generate features fails (#675)

* Improve exception handling message when generate features fails

Signed-off-by: Kathryn Kodama <[email protected]>

* Add return boolean for libertyGenerateFeatures method

Signed-off-by: Kathryn Kodama <[email protected]>

* Add missing return statement for createNewInstallFeatureUtil (#678)

Signed-off-by: Kathryn Kodama <[email protected]>

Co-authored-by: Paul Gooderham <[email protected]>
Co-authored-by: Eric Lau <[email protected]>
Co-authored-by: Kathryn Kodama <[email protected]>
  • Loading branch information
@TrevCraw @turkeylurkey
@ericglau @kathrynkodama
4 people committed Dec 15, 2021
1 parent 98dc9d3 commit bad4ec7
Show file tree
Hide file tree
Showing 8 changed files with 625 additions and 156 deletions.
4 changes: 3 additions & 1 deletion src/main/groovy/io/openliberty/tools/gradle/LibertyTaskFactory.groovy
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import io.openliberty.tools.gradle.tasks.DebugTask
import io.openliberty.tools.gradle.tasks.DeployTask
import io.openliberty.tools.gradle.tasks.UndeployTask
import io.openliberty.tools.gradle.tasks.InstallFeatureTask
import io.openliberty.tools.gradle.tasks.GenerateFeaturesTask
import io.openliberty.tools.gradle.tasks.PrepareFeatureTask
import io.openliberty.tools.gradle.tasks.InstallLibertyTask
import io.openliberty.tools.gradle.tasks.UninstallFeatureTask
Expand Down Expand Up @@ -62,7 +63,8 @@ class LibertyTaskFactory {
project.tasks.create('deploy', DeployTask)
project.tasks.create('undeploy', UndeployTask)
project.tasks.create('installFeature', InstallFeatureTask)
project.tasks.create('prepareFeature', PrepareFeatureTask)
project.tasks.create('generateFeatures', GenerateFeaturesTask)
project.tasks.create('prepareFeature', PrepareFeatureTask)
project.tasks.create('uninstallFeature', UninstallFeatureTask)
project.tasks.create('cleanDirs', CleanTask)
project.tasks.create('configureArquillian', ConfigureArquillianTask)
Expand Down
225 changes: 124 additions & 101 deletions src/main/groovy/io/openliberty/tools/gradle/tasks/AbstractFeatureTask.groovy
View file
Original file line number Diff line number Diff line change
@@ -1,74 +1,104 @@
/**
* (C) Copyright IBM Corporation 2021.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
* (C) Copyright IBM Corporation 2021.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.openliberty.tools.gradle.tasks

import java.util.Set

import org.gradle.api.artifacts.ResolveException
import org.gradle.api.logging.LogLevel
import org.gradle.api.tasks.Internal
import org.gradle.api.tasks.TaskAction
import org.gradle.api.tasks.options.Option
import org.gradle.api.tasks.Input
import org.gradle.api.Project
import org.gradle.testfixtures.ProjectBuilder

import io.openliberty.tools.common.plugins.util.InstallFeatureUtil
import io.openliberty.tools.common.plugins.util.InstallFeatureUtil.ProductProperties
import io.openliberty.tools.common.plugins.util.PluginExecutionException
import io.openliberty.tools.common.plugins.util.PluginScenarioException
import io.openliberty.tools.common.plugins.util.ServerFeatureUtil
import io.openliberty.tools.gradle.utils.ArtifactDownloadUtil
import org.gradle.api.Project
import org.gradle.api.logging.LogLevel
import org.gradle.api.tasks.Internal
import org.gradle.api.tasks.options.Option
import org.gradle.testfixtures.ProjectBuilder

public class AbstractFeatureTask extends AbstractServerTask {

// DevMode uses this option to provide the location of the
// temporary serverDir it uses after a change to the server.xml
private String serverDirectoryParam;

public boolean installFeaturesFromAnt;

private InstallFeatureUtil util;

@Internal
Project newProject = project;
Project newProject = project;

private ServerFeatureUtil servUtil;

@Option(option = 'serverDir', description = '(Optional) Server directory to get the list of features from.')
void setServerDirectoryParam(String serverDir) {
this.serverDirectoryParam = serverDir;
}

private class ServerFeatureTaskUtil extends ServerFeatureUtil {

@Override
void debug(String msg) {
logger.debug(msg);
}

@Override
void error(String msg, Throwable throwable) {
logger.error(msg, e);
}

@Override
void debug(String msg, Throwable throwable) {
logger.debug(msg, (Throwable) e);
}

@Override
void debug(Throwable throwable) {
logger.debug("Throwable exception received: " + e.getMessage(), (Throwable) e);
}

@Override
void warn(String msg) {
logger.warn(msg);
}

@Override
void info(String msg) {
logger.lifecycle(msg);
}
}

private class InstallFeatureTaskUtil extends InstallFeatureUtil {


public InstallFeatureTaskUtil(File installDir, String from, String to, Set<String> pluginListedEsas, List<ProductProperties> propertiesList, String openLibertyVerion, String containerName, List<String> additionalJsons) throws PluginScenarioException, PluginExecutionException {
super(installDir, from, to, pluginListedEsas, propertiesList, openLibertyVerion, containerName, additionalJsons)
}

@Override
public void debug(String msg) {
logger.debug(msg)
logger.debug(msg)
}

@Override
public void debug(String msg, Throwable e) {
logger.debug(msg, (Throwable) e)
logger.debug(msg, (Throwable) e)
}

@Override
public void debug(Throwable e) {
logger.debug("Throwable exception received: "+e.getMessage(), (Throwable) e)
logger.debug("Throwable exception received: " + e.getMessage(), (Throwable) e)
}

@Override
Expand Down Expand Up @@ -98,28 +128,7 @@ public class AbstractFeatureTask extends AbstractServerTask {

@Override
public File downloadArtifact(String groupId, String artifactId, String type, String version) throws PluginExecutionException {

String coordinates = groupId + ":" + artifactId + ":" + version + "@" + type

def dep = newProject.dependencies.create(coordinates)
def config = newProject.configurations.detachedConfiguration(dep)

Set<File> files = new HashSet<File>()
try {
config.resolvedConfiguration.resolvedArtifacts.each { artifact ->
File artifactFile = artifact.file
files.add(artifactFile)
debug(artifactFile.toString())
}
} catch (ResolveException e) {
throw new PluginExecutionException("Could not find artifact with coordinates " + coordinates, e)
}

if (!files) {
throw new PluginExecutionException("Could not find artifact with coordinates " + coordinates)
}
return files.iterator().next()

return ArtifactDownloadUtil.downloadArtifact(project, groupId, artifactId, type, version);
}
}

Expand All @@ -145,43 +154,32 @@ public class AbstractFeatureTask extends AbstractServerTask {
}
return features
}

@Internal
protected List<String> getAdditionalJsonList() {
List<String> result = new ArrayList<String>()
project.configurations.featuresBom.dependencies.each { dep ->
def coordinate = dep.group + ":" + "features" + ":" + dep.version
logger.debug("feature Json: " + coordinate)
result.add(coordinate)
}
return result;
}

protected Set<String> getSpecifiedFeatures(String containerName) throws PluginExecutionException {
if (util == null) {
def pluginListedEsas = getPluginListedFeatures(true)
def propertiesList = null;
def openLibertyVersion = null;
if (containerName == null) {
propertiesList = InstallFeatureUtil.loadProperties(getInstallDir(project))
openLibertyVersion = InstallFeatureUtil.getOpenLibertyVersion(propertiesList)
}
def additionalJsons = getAdditionalJsonList()
createNewInstallFeatureUtil(pluginListedEsas, propertiesList, openLibertyVersion, containerName, additionalJsons)
@Internal
protected List<String> getAdditionalJsonList() {
List<String> result = new ArrayList<String>()
project.configurations.featuresBom.dependencies.each { dep ->
def coordinate = dep.group + ":" + "features" + ":" + dep.version
logger.debug("feature Json: " + coordinate)
result.add(coordinate)
}
return result;
}

protected Set<String> getSpecifiedFeatures(String containerName) throws PluginExecutionException {
InstallFeatureUtil util = getInstallFeatureUtil(null, containerName);
// if createNewInstallFeatureUtil failed to create a new InstallFeatureUtil instance, then features are installed via ant
if(installFeaturesFromAnt) {
if (installFeaturesFromAnt) {
Set<String> featuresInstalledFromAnt;
if(server.features.name != null) {
if (server.features.name != null) {
featuresInstalledFromAnt = new HashSet<String>(server.features.name);
return featuresInstalledFromAnt;
}
else {
} else {
featuresInstalledFromAnt = new HashSet<String>();
return featuresInstalledFromAnt;
}
}

def pluginListedFeatures = getPluginListedFeatures(false)
def dependencyFeatures = getDependencyFeatures()
def serverFeatures = null;
Expand All @@ -194,40 +192,65 @@ public class AbstractFeatureTask extends AbstractServerTask {
}

Set<String> featuresToInstall = InstallFeatureUtil.combineToSet(pluginListedFeatures, dependencyFeatures, serverFeatures)
return featuresToInstall
return featuresToInstall
}

protected ServerFeatureUtil getServerFeatureUtil() {
if (servUtil == null) {
servUtil = new ServerFeatureTaskUtil();
}
return servUtil;
}

private void createNewInstallFeatureUtil(Set<String> pluginListedEsas, List<ProductProperties> propertiesList, String openLibertyVerion, String containerName, List<String> additionalJsons) throws PluginExecutionException {
try {
util = new InstallFeatureTaskUtil(getInstallDir(project), server.features.from, server.features.to, pluginListedEsas, propertiesList, openLibertyVerion, containerName, additionalJsons)
} catch (PluginScenarioException e) {
logger.debug("Exception received: "+e.getMessage(),(Throwable)e)
logger.debug("Exception received: " + e.getMessage(), (Throwable) e)
logger.debug("Installing features from installUtility.")
installFeaturesFromAnt = true
return
}
}

protected InstallFeatureUtil getInstallFeatureUtil(Set<String> pluginListedEsas, String containerName) throws PluginExecutionException {
if (util == null) {
if (pluginListedEsas == null) {
pluginListedEsas = getPluginListedFeatures(true);
}
def propertiesList = null;
def openLibertyVersion = null;
if (containerName == null) {
propertiesList = InstallFeatureUtil.loadProperties(getInstallDir(project))
openLibertyVersion = InstallFeatureUtil.getOpenLibertyVersion(propertiesList)
}
def additionalJsons = getAdditionalJsonList()
createNewInstallFeatureUtil(pluginListedEsas, propertiesList, openLibertyVersion, containerName, additionalJsons)
}
return util;
}

protected InstallFeatureUtil getInstallFeatureUtil(Set<String> pluginListedEsas, List<ProductProperties> propertiesList, String openLibertyVerion, String containerName, List<String> additionalJsons) throws PluginExecutionException {
//if installing userFeature, recompile gradle to find mavenLocal artifacts created by prepareFeature task.
if(project.configurations.featuresBom.dependencies) {
try {
ProjectBuilder builder = ProjectBuilder.builder();
newProject = builder
.withProjectDir(project.rootDir)
.withGradleUserHomeDir(project.gradle.gradleUserHomeDir)
.withName(project.name)
.build();

// need this for gradle to evaluate the project
// and load the different plugins and extensions
newProject.evaluate();
} catch (Exception e) {
throw new PluginExecutionException("Could not parse build.gradle " + e.getMessage());
}
}
createNewInstallFeatureUtil(pluginListedEsas, propertiesList, openLibertyVerion, containerName, additionalJsons)
//if installing userFeature, recompile gradle to find mavenLocal artifacts created by prepareFeature task.
if (project.configurations.featuresBom.dependencies) {
try {
ProjectBuilder builder = ProjectBuilder.builder();
newProject = builder
.withProjectDir(project.rootDir)
.withGradleUserHomeDir(project.gradle.gradleUserHomeDir)
.withName(project.name)
.build();

// need this for gradle to evaluate the project
// and load the different plugins and extensions
newProject.evaluate();
} catch (Exception e) {
throw new PluginExecutionException("Could not parse build.gradle " + e.getMessage());
}
}
createNewInstallFeatureUtil(pluginListedEsas, propertiesList, openLibertyVerion, containerName, additionalJsons)
return util
}


}
4 changes: 2 additions & 2 deletions src/main/groovy/io/openliberty/tools/gradle/tasks/AbstractServerTask.groovy
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ import org.apache.commons.io.FilenameUtils
import org.apache.commons.io.filefilter.FileFilterUtils

import io.openliberty.tools.ant.ServerTask
import io.openliberty.tools.common.plugins.config.ServerConfigDropinXmlDocument;
import io.openliberty.tools.common.plugins.config.ServerConfigXmlDocument;

import java.util.ArrayList
import java.util.List
Expand Down Expand Up @@ -927,7 +927,7 @@ abstract class AbstractServerTask extends AbstractLibertyTask {

private void writeConfigDropinsServerVariables(File file, Properties varProps, Properties varProjectProps, boolean isDefaultVar) throws IOException, TransformerException, ParserConfigurationException {

ServerConfigDropinXmlDocument configDocument = ServerConfigDropinXmlDocument.newInstance()
ServerConfigXmlDocument configDocument = ServerConfigXmlDocument.newInstance()

configDocument.createComment(HEADER)

Expand Down
Loading

0 comments on commit bad4ec7

Please sign in to comment.