Cannot install passwordUtilities-1.0 on WLP, only on OL-runtime #1236

bmarwell · 2021-08-24T13:02:28Z

Hi everyone,

Plugin setup

I have the following feature in my src/main/liberty/config/server.xml:

<features>
  <feature>passwordUtilities-1.0</features>
<features>

Expected behaviour

in the install-feature step, the feature is being downloaded and installed independent of my current runtime artifact (e.g. ol-kernel, wlp-kernel, ol-runtime). Note, there is not wlp-runtime nor wlp-base (which you can download as customer from IBM).

Actual behavour

When I configure the com.ibm.wlp artifact, like so:

      <plugin>
        <groupId>io.openliberty.tools</groupId>
        <artifactId>liberty-maven-plugin</artifactId>
        <version>${liberty-maven-plugin.version}</version>
        <configuration>
          <serverName>cfggenerator</serverName>
          <libertyRuntimeVersion>${wlp.version}</libertyRuntimeVersion>

          <runtimeArtifact>
            <groupId>com.ibm.websphere.appserver.runtime</groupId>
            <artifactId>wlp-javaee7</artifactId>
            <version>${wlp.version}</version>
            <type>zip</type>
          </runtimeArtifact>
          <features>
            <acceptLicense>true</acceptLicense>
            <feature>passwordUtilities-1.0</feature>
          </features>
          <deployPackages>dependencies</deployPackages>
        </configuration>
        <executions>
          <execution>
            <id>test-start</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>create</goal>
              <goal>deploy</goal>
              <goal>install-feature</goal>
              <goal>start</goal>
            </goals>
          </execution>
          <execution>
            <id>test-stop</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

I get this exception:

CWWKF1221E: The io.openliberty.autoPasswordUtilities1.0.javaee asset depends on com.ibm.websphere.appserver.eeCompatible-6.0, which is not available in the IBM WebSphere Liberty Repository. If the required asset is a feature that you previously downloaded, install the missing feature by specifying the location of the feature ESA file on the featureManager install --location option, and try to install io.openliberty.autoPasswordUtilities1.0.javaee again.

This happens on wlp.version = 21.0.0.6 and even if I import the features-bom.

known workarounds

The setup process will work flawlessly when I use the artifact io.openliberty:openliberty-runtime.

From what I can see, only this artifact has servlet-3.0 integrated and also supports com.ibm.websphere.appserver.eeCompatible-6.0 which (for unknown reasons) cannot be installed via feature utility

Things I looked up and tried

I haven't found any hint that passwordUtility-1.0 is excluded from installation via featureUtility or this plugin.
Installing baseBundle in any WLP (not OL) also works. But I haven't found how to install baseBundle via this plugin. However, I only want to install the required features anyway. I would like to use wlp-kernel and then install passwordUtility-1.0

Any help what gets wrong here is highly appreciated.
Best regards,
Ben

The text was updated successfully, but these errors were encountered:

cherylking · 2021-08-24T13:17:20Z

@bmarwell Which version of liberty-maven-plugin are you using?

bmarwell · 2021-08-24T14:16:33Z

Hi cheryl,

I am using version 3.4.
However, I do not think this might make a difference, as features are loaded from the repositories. Although I think it always loads from an /io/openliberty path.

cherylking · 2021-08-24T14:22:16Z

@bmarwell I asked about the plugin version because a fix did go into 3.4 related to resolving features (OpenLiberty/ci.common#277). It was related to OpenLiberty 21.0.0.7 however. We will look into your issue and let you know what we find out. Thanks.

cherylking · 2021-08-25T15:53:19Z

@bmarwell So I set up a test project locally with the configuration you provided above, and I was not able to recreate the error you reported. Can you provide a sample project to recreate the error? Here is what I see in my terminal window when I run my test project:

[INFO] [INFO] The runtimeArtifact version 21.0.0.6 is overwritten by the liberty.runtime.version value 21.0.0.6.
[INFO] [INFO] CWWKM2102I: Using artifact based assembly archive : com.ibm.websphere.appserver.runtime:wlp-javaee7:null:21.0.0.6:zip.
[INFO] [INFO] CWWKM2102I: Using installDirectory : /Users/xxx/test-project/target/liberty/wlp.
[INFO] [INFO] CWWKM2102I: Using serverName : test.
[INFO] [INFO] CWWKM2102I: Using serverDirectory : /Users/xxx/test-project/target/liberty/wlp/usr/servers/test.
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/features/21.0.0.6/features-21.0.0.6.json
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/features/21.0.0.6/features-21.0.0.6.json (1.3 MB at 4.4 MB/s)
[INFO] [INFO] Parsing the server file /Users/xxx/test-project/target/liberty/wlp/usr/servers/test/server.xml for features and includes.
[INFO] [INFO] Parsing the server file /Users/xxx/test-project/target/liberty/wlp/usr/servers/test/configDropins/overrides/liberty-plugin-variable-config.xml for features and includes.
[INFO] [INFO] Installing features: [passwordUtilities-1.0]
[INFO] 
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/servlet-3.0/21.0.0.6/servlet-3.0-21.0.0.6.esa
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/servlet-3.0/21.0.0.6/servlet-3.0-21.0.0.6.esa (3.3 MB at 5.2 MB/s)
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.jaxrsApiStub-1.1/21.0.0.6/com.ibm.websphere.appserver.jaxrsApiStub-1.1-21.0.0.6.esa
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.jaxrsApiStub-1.1/21.0.0.6/com.ibm.websphere.appserver.jaxrsApiStub-1.1-21.0.0.6.esa (16 kB at 235 kB/s)
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.internal.jaxrs-1.1/21.0.0.6/com.ibm.websphere.appserver.internal.jaxrs-1.1-21.0.0.6.esa
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.internal.jaxrs-1.1/21.0.0.6/com.ibm.websphere.appserver.internal.jaxrs-1.1-21.0.0.6.esa (3.1 MB at 5.0 MB/s)
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/jaxrs-1.1/21.0.0.6/jaxrs-1.1-21.0.0.6.esa
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/jaxrs-1.1/21.0.0.6/jaxrs-1.1-21.0.0.6.esa (25 kB at 332 kB/s)
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.restConnectorjaxrs-1.0/21.0.0.6/com.ibm.websphere.appserver.restConnectorjaxrs-1.0-21.0.0.6.esa
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.restConnectorjaxrs-1.0/21.0.0.6/com.ibm.websphere.appserver.restConnectorjaxrs-1.0-21.0.0.6.esa (14 kB at 203 kB/s)
[INFO] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.jaxrsEJB-1.1/21.0.0.6/com.ibm.websphere.appserver.jaxrsEJB-1.1-21.0.0.6.esa
[INFO] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/com/ibm/websphere/appserver/features/com.ibm.websphere.appserver.jaxrsEJB-1.1/21.0.0.6/com.ibm.websphere.appserver.jaxrsEJB-1.1-21.0.0.6.esa (25 kB at 346 kB/s)
[INFO] [INFO] Product validation completed successfully.
[INFO] [INFO] The following features have been installed: servlet-3.0 appSecurity-1.0 passwordUtilities-1.0 servlet-4.0 jaxrs-1.1

bmarwell · 2021-08-25T16:16:41Z

Hi, I will provide a sample project.

One more thing. I have my company pom and a mirror of Central in my .m2/settings.xml. I saw there was no download attempt from that repo. I'll test it without those, too.

bmarwell · 2021-08-26T08:28:11Z

I deleted my $HOME/.m2/repository/com/ibm directory, and this is what happens now:

[INFO] --- liberty-maven-plugin:3.4:install-feature (test-start) @ openliberty ---
[INFO] The runtimeArtifact version 21.0.0.6 is overwritten by the liberty.runtime.version value 21.0.0.6.
[INFO] CWWKM2102I: Using artifact based assembly archive : com.ibm.websphere.appserver.runtime:wlp-javaee7:null:21.0.0.6:zip.
[INFO] CWWKM2102I: Using installDirectory : /Users/$USER/git/myproject/myproject-integrationtests/openliberty/target/liberty/wlp.
[INFO] CWWKM2102I: Using serverName : myproject..................................
[INFO] CWWKM2102I: Using serverDirectory : /Users/$USER/git/myproject/myproject-integrationtests/openliberty/target/liberty/wlp/usr/servers/myproject.
Downloading from ext-release: https://repository.mirror/artifactory/ext-release-local/io/openliberty/features/install-map/maven-metadata.xml
Downloading from snapshots: https://repository.mirror/artifactory/libs-snapshot/io/openliberty/features/install-map/maven-metadata.xml
Downloading from maven-central: https://repository.mirror/artifactory/maven-central/io/openliberty/features/install-map/maven-metadata.xml
Downloading from openjdk-api-v3: http://repository.mirror/artifactory/openjdk-api-v3/io/openliberty/features/install-map/maven-metadata.xml
Downloading from int-release: https://repository.mirror/artifactory/libs-release-local/io/openliberty/features/install-map/maven-metadata.xml
Downloading from int-snapshots: https://repository.mirror/artifactory/libs-snapshot-local/io/openliberty/features/install-map/maven-metadata.xml
Downloading from myproject-local-repo: file:///Users/$USER/git/myproject/repository/io/openliberty/features/install-map/maven-metadata.xml
Downloading from artifacatory: https://repository.mirror/webdeployment/io/openliberty/features/install-map/maven-metadata.xml
Downloaded from maven-central: https://repository.mirror/artifactory/maven-central/io/openliberty/features/install-map/maven-metadata.xml (350 B at 2.1 kB/s)
Downloaded from snapshots: https://repository.mirror/artifactory/libs-snapshot/io/openliberty/features/install-map/maven-metadata.xml (350 B at 556 B/s)
Downloading from openjdk-api-v3: http://repository.mirror/artifactory/openjdk-api-v3/com/ibm/websphere/appserver/features/features/21.0.0.6/features-21.0.0.6.json
Downloading from maven-central: https://repository.mirror/artifactory/maven-central/com/ibm/websphere/appserver/features/features/21.0.0.6/features-21.0.0.6.json
Downloaded from maven-central: https://repository.mirror/artifactory/maven-central/com/ibm/websphere/appserver/features/features/21.0.0.6/features-21.0.0.6.json (1.3 MB at 4.3 MB/s)
[INFO] Parsing the server file /Users/$USER/git/myproject/myproject-integrationtests/openliberty/target/liberty/wlp/usr/servers/myproject/server.xml for features and includes.
[INFO] Parsing the server file /Users/$USER/git/myproject/myproject-integrationtests/openliberty/targgit/myprojectet/liberty/wlp/usr/servers/myproject/configDropins/overrides/liberty-plugin-variable-config.xml for features and includes.
[INFO] Installing features: [transportsecurity-1.0, servlet-3.1, jndi-1.0, passwordUtilities-1.0, jdbc-4.1, jaxrs-2.0]
Downloading from openjdk-api-v3: http://repository.mirror/artifactory/openjdk-api-v3/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloading from maven-central: https://repository.mirror/artifactory/maven-central/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloading from snapshots: https://repository.mirror/artifactory/libs-snapshot/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloading from ext-release: https://repository.mirror/artifactory/ext-release-local/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloading from int-release: https://repository.mirror/artifactory/libs-release-local/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloading from int-snapshots: https://repository.mirror/artifactory/libs-snapshot-local/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloading from myproject-local-repo: file:///Users/$USER/git/myproject/repository/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloaded from maven-central: https://repository.mirror/artifactory/maven-central/io/openliberty/features/repository-resolver/maven-metadata.xml (358 B at 6.4 kB/s)
Downloading from artifacatory: https://repository.mirror/webdeployment/io/openliberty/features/repository-resolver/maven-metadata.xml
Downloaded from snapshots: https://repository.mirror/artifactory/libs-snapshot/io/openliberty/features/repository-resolver/maven-metadata.xml (358 B at 2.2 kB/s)

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Configuration Generator 2.3.2-SNAPSHOT:
[INFO]
[INFO] MY_PROJECT :: ITs .................................. SUCCESS [  0.046 s]
[INFO] MY_PROJECT :: ITs :: OpenLiberty ................... FAILURE [ 38.191 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  51.339 s
[INFO] Finished at: 2021-08-26T10:18:52+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.openliberty.tools:liberty-maven-plugin:3.4:install-feature (test-start) on project openliberty: CWWKF1221E: The io.openliberty.autoPasswordUtilities1.0.javaee asset depends on com.ibm.websphere.appserver.eeCompatible-6.0, which is not available in the IBM WebSphere Liberty Repository. If the required asset is a feature that you previously downloaded, install the missing feature by specifying the location of the feature ESA file on the featureManager install --location option, and try to install io.openliberty.autoPasswordUtilities1.0.javaee again. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :openliberty

bmarwell · 2021-08-26T08:33:55Z

In -X (debug) mode, I found this message:

Is installing only Open Liberty features? true

This seems to be the cause, because in your case features are downloaded from com/ibm/ rather than io/openliberty with the appropriate artefact.

bmarwell · 2021-08-26T11:02:29Z

Found it. Not too happy about the error message, though :)

Here is the root cause I found using -X:

MissingRequirement [requirementName=com.ibm.websphere.appserver.eeCompatible-6.0, owningResource=com.ibm.websphere.appserver.javax.connector-1.6]
Conflict for com.ibm.websphere.appserver.eeCompatible
 - ROOT->com.ibm.websphere.appserver.jaxrs-2.0->com.ibm.websphere.appserver.internal.jaxrs-2.0->[com.ibm.websphere.appserver.eeCompatible-7.0] 7.0.0
 - ROOT->io.openliberty.autoPasswordUtilities1.0.javaee->com.ibm.websphere.appserver.javax.connector-1.6->[com.ibm.websphere.appserver.eeCompatible-6.0] 6.0.0

This is what I have in my server.xml:

  <featureManager>
    <feature>servlet-3.1</feature>
    <feature>jdbc-4.1</feature>
    <feature>jndi-1.0</feature>
    <feature>transportSecurity-1.0</feature>
    <feature>jaxrs-2.0</feature>
    <feature>passwordUtilities-1.0</feature>
  </featureManager>

When I switch to servlet-4.0 and jaxrs-2.1, the installation process works flawlessly. Not sure why, but in any case the root cause is only visible via -X and does not even provide any information HOW to solve the missing requirement problem. I also do not understand why it works using openliberty-runtime. The features were unchanged.

To me, this looks like a bug in the installUtility or featureUtility, WDYT?

cherylking · 2021-08-26T13:10:15Z

To me, this looks like a bug in the installUtility or featureUtility, WDYT?

@bmarwell Wow, good debugging. I can see why you are not happy with the error message. Nothing points you towards the root cause. I definitely agree that you should open an issue against Open Liberty for this feature installation problem.

I found the line of code in our plugin responsible for the Is installing only Open Liberty features? true message (https://github.com/OpenLiberty/ci.common/blob/main/src/main/java/io/openliberty/tools/common/plugins/util/InstallFeatureUtil.java#L424). It seems if the features getting installed are all present in OpenLiberty, then that flag is set. I'm not sure why using openliberty-runtime causes the same set of features to work though. I think someone in Open Liberty install team needs to look at this.

Can you recreate this from the command line with installUtility or featureUtility?

bmarwell · 2021-08-26T16:06:54Z

@bmarwell Wow, good debugging. I can see why you are not happy with the error message. Nothing points you towards the root cause. I definitely agree that you should open an issue against Open Liberty for this feature installation problem.

Haha, I should now as a maven committer. ;-) Good thing your maven plugin does have excellent logging! :)

I found the line of code in our plugin responsible for the Is installing only Open Liberty features? true message (https://github.com/OpenLiberty/ci.common/blob/main/src/main/java/io/openliberty/tools/common/plugins/util/InstallFeatureUtil.java#L424). It seems if the features getting installed are all present in OpenLiberty, then that flag is set. I'm not sure why using openliberty-runtime causes the same set of features to work though. I think someone in Open Liberty install team needs to look at this.

Can you recreate this from the command line with installUtility or featureUtility?

Yes this is what exactly came to my mind as well. I will create a script which downloads wlp-kernel to a temporary location, sets WLP_ dirs and calls the install utility. That should do as a minimal reproducible test.

I will leave this issue open for a few more days for the unlikely case I cannot rproduce this behaviour with one of the provided utilities.

Thanks for your supprt!
Ben.

bmarwell · 2021-08-26T19:25:49Z

Hi @cherylking,

I cannot reproduce the issue with any of those tools: featureUtility, featureManager nor installUtility.

My current workaround is to use JavaEE8 features (most of them are compatible anyway), but I'd still like to get to the cause of this just in case we have an invalid setup.
Next idea would be to build a version of this plugin where isOnlyLibertyFeatures always returns false...

cherylking · 2021-08-26T19:47:03Z

I cannot reproduce the issue with any of those tools: featureUtility, featureManager nor installUtility.

@bmarwell When you tried from the command line, did it fetch the features from com/ibm instead of io/openliberty?

Next idea would be to build a version of this plugin where isOnlyLibertyFeatures always returns false...

I don't think that would make a difference. That line of code only used isOnlyLibertyFeatures to set the acceptLicense flag to true, which you already set to true anyway. It did not pass it along to the install utility. This is quite puzzling. I'll try to get some other eyes on this internally and see if we can figure out what is going awry.

bmarwell · 2021-08-26T19:58:37Z

Alright. Will try to set up an example project and will also post the full stack trace tomorrow.

bmarwell · 2021-08-27T06:51:01Z

@cherylking I was able to set up an example project:

https://github.com/bmarwell/liberty.maven-1236

Just run mvn verify.

It is a LOT of debug, but the error message is the same I get:
https://github.com/bmarwell/liberty.maven-1236/runs/3440493350?check_suite_focus=true

As soon as you either change features (servlet 4, jars-2.1) OR use openliberty-runtime or your local wlp-base artifact, it will work flawlessy..

cherylking · 2021-08-30T14:17:53Z

@bmarwell After talking to Open Liberty development, the set of features you are trying to install on openliberty-kernel should work, but there is some bug that they are tracking down with the autoPasswordUtilities feature dependency on com.ibm.websphere.appserver.eeCompatible-6.0. A workaround was mentioned to include the jca-1.7 feature in your server.xml. I have confirmed that this does get around this issue.

I am following up on whether an issue is currently open for this on Open Liberty and will add a reference here.

cherylking · 2021-08-30T14:22:51Z

Related Open Liberty issue: OpenLiberty/open-liberty#17843

bmarwell · 2021-08-30T14:22:56Z

Thanks @cherylking!

Good findings 🥇, your help is VERY much appreciated! :) 👍🏻

bmarwell · 2021-08-31T12:39:15Z

I added <feature>jca-1.7</feature> to the liberty-maven-plugin, but left only <feature>passwordUtilities-1.0</feature> in the actual server.xml. Works as suggested. Thank you for the workaround!

bmarwell · 2022-01-13T06:41:14Z

Hi @cherylking. As far as I can tell, this has long been resolved. Will close this issue. If you think there is anything else to discuss, just ping me. :)

bmarwell closed this as completed Jan 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot install passwordUtilities-1.0 on WLP, only on OL-runtime #1236

Cannot install passwordUtilities-1.0 on WLP, only on OL-runtime #1236

bmarwell commented Aug 24, 2021 •

edited

Loading

cherylking commented Aug 24, 2021

bmarwell commented Aug 24, 2021

cherylking commented Aug 24, 2021

cherylking commented Aug 25, 2021

bmarwell commented Aug 25, 2021

bmarwell commented Aug 26, 2021

bmarwell commented Aug 26, 2021

bmarwell commented Aug 26, 2021 •

edited

Loading

cherylking commented Aug 26, 2021 •

edited

Loading

bmarwell commented Aug 26, 2021

bmarwell commented Aug 26, 2021

cherylking commented Aug 26, 2021

bmarwell commented Aug 26, 2021

bmarwell commented Aug 27, 2021

cherylking commented Aug 30, 2021

cherylking commented Aug 30, 2021

bmarwell commented Aug 30, 2021

bmarwell commented Aug 31, 2021

bmarwell commented Jan 13, 2022

Cannot install passwordUtilities-1.0 on WLP, only on OL-runtime #1236

Cannot install passwordUtilities-1.0 on WLP, only on OL-runtime #1236

Comments

bmarwell commented Aug 24, 2021 • edited Loading

Plugin setup

Expected behaviour

Actual behavour

known workarounds

Things I looked up and tried

cherylking commented Aug 24, 2021

bmarwell commented Aug 24, 2021

cherylking commented Aug 24, 2021

cherylking commented Aug 25, 2021

bmarwell commented Aug 25, 2021

bmarwell commented Aug 26, 2021

bmarwell commented Aug 26, 2021

bmarwell commented Aug 26, 2021 • edited Loading

cherylking commented Aug 26, 2021 • edited Loading

bmarwell commented Aug 26, 2021

bmarwell commented Aug 26, 2021

cherylking commented Aug 26, 2021

bmarwell commented Aug 26, 2021

bmarwell commented Aug 27, 2021

cherylking commented Aug 30, 2021

cherylking commented Aug 30, 2021

bmarwell commented Aug 30, 2021

bmarwell commented Aug 31, 2021

bmarwell commented Jan 13, 2022

bmarwell commented Aug 24, 2021 •

edited

Loading

bmarwell commented Aug 26, 2021 •

edited

Loading

cherylking commented Aug 26, 2021 •

edited

Loading