[Airgap] maven could not calculate build plan - settings.xml not used

[ghost]

I am running che 7.2.0 and the environment information:

mvn --version
Picked up JAVA_TOOL_OPTIONS: -XX:MaxRAMPercentage=50 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z)
Maven home: /usr/share/maven
Java version: 11.0.3, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-11-openjdk-amd64
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.5.1.el7.x86_64", arch: "amd64", family: "unix"

Any suggestion?

[tolusha]

Memory issue?
Could you show the whole error message?

[tolusha]

I tested the project and wasn't able to reproduce the issue.
https://github.com/rbcminghou/java-maven-helloworld

[tolusha]

devfile might help

[ghost]

I guess my environment is behind the firewall, so that I copy settings.xml (contains the proxy settings) to /home/users/.m2 fold.

How do I do in the devfile?

[tolusha]

@rbcminghou
The whole error message should give a clue.
Could you show it?

[ghost]

CoreException: Could not calculate build plan: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.8.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-compiler-plugin:jar:3.8.1: ArtifactResolutionException: Could not transfer artifact org.apache.maven.plugins:maven-compiler-plugin:pom:3.8.1 from/to central (https://repo.maven.apache.org/maven2): Connection reset

[nickboldt]

well, there's your reason: you're behind a firewall and can't see repo.maven.apache.org/maven2.

Can you set up settings.xml to redirect requests for maven central to your internal copy?

[ghost]

The "settings.xml" file must be stored in /home/user/.m2 and that is where Maven picks up by default.

[nickboldt]

so if you're using a settings.xml with an override for maven central -> internal nexus... then why is your build trying to go to MC directly?

Are you doing this commandline in console, or via a command?

If the latter, seems like the issue is that the maven runtime container (or sidecar?) isn't respecting the settings.xml file in the user's workspace, and is instead defaulting to normal "on the internet where Maven Central is accessible" behaviour, rather than the air-gapped, settings.xml-overridden scenario.

@tolusha Do we need some way to inject it into the rt/sidecar container so that it's seen and used?

cc: @l0rd

[ghost]

I think settings.xml is used for Maven widely.... there is not just set for proxy but for other purposes also.

I think if Eclipse che wants to abandon the support of usage settings.xml and I am not sure the Maven users are okay with it.

[tolusha]

@fbricon
Could you help us with this?
How m2e and jdt.ls works behind proxy? Which configuration do we need to set up?

[nickboldt]

Latest from Ming suggests that the complete workaround is to add a command/action/task that does:

 - name: copy maven settings.xml
    actions:
      - type: exec
        component: maven
        command: "ln -s ${CHE_PROJECTS_ROOT}/java-maven-helloworld/settings.xml /home/user/.m2/settings.xml"
        workdir: ${CHE_PROJECTS_ROOT}

Since a user might have a different settings.xml file in each project in his workspace, this might be an acceptable, safe solution.

Could also configure the mvn install command(s) use -s ${CHE_PROJECTS_ROOT}/java-maven-helloworld/settings.xml when building but that might not solve the maven project nature resolution as default settings.xml would still point to OUTSIDE the airgapped org rather than to the internal nexus instance.

Alternatively, we could set up some logic that says "if devfile only includes one project, and that project includes a settings.xml in the root of ${CHE_PROJECTS_ROOT}/checkout-folder then copy (or symlink) that file to /home/user/.m2/settings.xml as the default settings.xml

But that might accidentally overwrite/destroy something the user manually set up, so the above solution (and some Che 7 doc on how to use maven in airgap) might be the best approach.

[ghost]

I am wondering that the custom resource YAML proxy settings (following lines) can help?!

# protocol+hostname of a proxy server. Automatically added as JAVA_OPTS and https(s)_proxy
# to Che server and workspaces containers
proxyURL: ''
# port of a proxy server
proxyPort: ''
# username for a proxy server
proxyUser: ''
# password for a proxy user
proxyPassword: ''
# a list of non-proxy hosts. Use | as delimiter, eg localhost|my.host.com|123.42.12.32
nonProxyHosts: ''

[tsmaeder]

I don't think setting up settings.xml in the dev machine container's /home/user/.m2 is supported from the devfile right now. You can put a file in .m2 by hand, but you'll have to declare .m2 as a persistent volume in the dev container, or you'll lose it at restart. Of course, you can always set up your own container to use as the dev machine.
We're already handling the special case of adding repositories to settings.xml in the java tooling container, but I think a proper solution would have to be more general: maybe declaring an init container in the devfile that can copy files to the proper locations?

[amisevsk]

It could also potentially be done by mounting a configmap containing the proper settings.xml as a volume in the workspace pod, though I haven't tested that sort of flow directly.

[tsmaeder]

@amisevsk it is not obvious to me how mounting a config map would work for this. To meet user expectations, we need settings.xml to be a regular file that can be changed by the user.

[l0rd]

@tsmaeder in #14391 I mentioned the approach to:

• let users define maven settings as env variables of the java plugin in the devfile
  AND
• add a "maven-settings-creator" init container to the java plugin meta.yaml (as we do for theia) that will get the environment variables and create a settings.xml

This approach wasn't possible at the time you fixed #14391 but now you can define plugins init containers and platform team should address #14343 during this sprint.

Would that work?

@amisevsk it's not clear to me how configmaps would help, maybe in conjunction with an init container?

[amisevsk]

The use case I was imagining would be a user of Che configuring the server side to work with their internal repositories; if the settings.xml needs to be modifiable, then this doesn't work, but, if the use case is "paste the settings.xml you want to use here", then it could be created as a configmap and mounted as an overlay.

The issue with init containers is that it can end up being a long queue of containers that need to be pulled and started, all for a simple "copy this file to this location".

[nickboldt]

This remains a problem for our airgap customer with CRW 2.0.1 (on OCP 3.11 deployed via crwctl). His current workaround is a custom devfile which does this to override default maven-java devfile actions, w/ his settings.xml file in order to resolve proxied maven central clone inside airgap.

- name: maven build
  actions:
   - type: exec
    component: maven
    command: “mvn -Duser.home=${HOME} -s settings.xml clean install”
    workdir: ${CHE_PROJECTS_ROOT}/java-maven-helloworld
 - name: maven build and run
  actions:
   - type: exec
    component: maven
    command: “mvn -Duser.home=${HOME} -s settings.xml clean install && java -cp ./target/*.jar com.rbccm.app.Test”
    workdir: ${CHE_PROJECTS_ROOT}/java-maven-helloworld

without the correct setting for Maven, I will not able to see “Run|Debug” on top of my Java code
With “Run|Debug”, I do not need to set my own command

For Java with Maven, it does not work. I need my custom commands in order to work.
For Java standalone [devfile], it works.

So, overall for crw 2.0 is far better than [when] I setup 7.2 by hand.
I think that your team has done the great work there

[tsmaeder]

Here's what I don't understand: the ~/.m2 folder should be persistent anyway....why doesn't putting settings.xml there work?

[tsmaeder]

There is a script here (https://github.com/che-dockerfiles/che-sidecar-java/blob/11/etc/before-start.sh) that takes the env-variable $MAVEN_MIRROR_URL and puts it into settings.xml. Does that help?

[bryantson]

I am encountering the similar issue. Currently, I resolved it by creating a manual settings.xml with the mirrored Nexus info in .m2 folder through Che/CRW maven terminal window. Nick's suggestion seems to automate somewhat. But ideally, there should be a property through CheCluster that should be able to automate this somewhat or a command option built inside Che/CRW through setting.

[tsmaeder]

The trouble with init containers is that they can only write to shared volumes, so pre-populating ~/.m2 with mirrored artifacts would not work in that use case. On the other hand, we don't really want to have special base images for every technologies.

[amisevsk]

The trouble with init containers is that they can only write to shared volumes

@tsmaeder This is likely a hole in my understanding, but I'm not clear what you mean -- currently, I believe volumes in a pod are already shared by default (i.e. my java-maven workspace mounts the m2 volume to both the maven and vscode-java containers). Wouldn't an init container that mounts the volume and writes settings.xml work semi-naturally in this case? We currently do something similar to populate the /plugins folder, where the broker runs as an init container.

[tsmaeder]

@amisevsk I was referring to the case where a container wants to bring a pre-populated ~/.m2 folder. Share volumes in a workspace are always empty at workspace creation.

[amisevsk]

Ah right, that makes sense. I misunderstood the context.

[nickboldt]

FYI this is turning into a more pressing issue as more customers are asking for this feature.

[sunix]

Could be solved with #14680
We have to consider #14680 so that it would not just work for settings.xml but also any other services: aws (credential), kubernetes (kubeconfig), git https (personal github token), airgap npm, ssh (thought it's been done for git ... but hardcoded)

[tsmaeder]

@sunix no, having environment variables does not help here: we need to automatically create a config file to make this work out of the box.

[nickboldt]

Do we have an ETA for this to be solved? It would be cool if it could be moved out of backlog and into a specific milestone, such as 7.12 (ideal) or 7.13 (less ideal).

[ericwill]

Isn't this covered in the docs, or is this a different issue?

[l0rd]

@ericwill agree: my assumption is that the original problem has been addressed by #15503. The UX is still not optimal and we need to continue working on it so I have opened an epic #16832 that summarize the work done and what still needs to be implemented. I am closing this issue.