Apply Maven filters to configDirectory contents (server.xml, server.env, etc.)

[mikekuzak]

configDirectory copies over the files from the default dir /src/main/liberty/config.
We have environment variables which are set during build. Is it possible to use filters with this command ?

[cherylking]

@mikekuzak Are you trying to set the configDirectory value using an environment variable? Also not sure what filters you are referring to?

If you could give us a concrete example of what your pom.xml looks like, and what you are specifying as environment variables or on the command line that would help. And if you are requesting an enhancement to the plugin, please describe how it should work to meet your requirements. We are always looking to improve our plugin to make it more useful for customer scenarios.

[mikekuzak]

@cherylking Hi, sorry for not being explicit enough.

In the server.xml I have a JNDI entry
<jndiEntry jndiName="url/tidal-api" value="${tidal.url}" id="Tidal REST API" />

The POM file is using filers:

<build> <filters> <filter>${project.basedir}/src/main/filters/${environment}.properties</filter> </filters>

For example the maven-resources-plugin as an attribute
<filtering>true</filtering>

which would use the values from the ${environment}.properties and substitute them depending on the POM profile per environment. Maybe there is an other way to do this in OL i'm not familiar with.
Thanks

[scottkurz]

@mikekuzak just noticed this scrolling through some old issues.

If you're still interested...

One way you could do something like this is to use the support we added for translating Maven project properties to Liberty config (documented here) together with Maven profiles in order to differentiate across different environments.

So for example you could have:

	<profiles>
		<profile>
			<id>my-profile1</id>
   	                <properties>
                           <liberty.var.tidal.url>URL1</liberty.var.tidal.url>
                        </properties>
		</profile>

Now, that's probably fine if you weren't using this same filter to do other substitutions with these same properties, but if you were, you might not find this as useful.

Would be interested to hear your thoughts.

[scottkurz]

Added workaround idea here: https://github.com/scottkurz/lmp-issue-587/tree/main

[scottkurz]

Thinking about how to fix this...I can think of two design questions.

1. merging

We have a choice to make when handling the non-server.xml config files like jvm.options, server.env, since these are written from a mix of user source files and maven/system props (with some simplifying rules so not always in the same invocation though).

It seems like a starting point might be to assume every time we do the Ant copy we do the filter substitution, and don't otherwise..and see if we think that's a rational design from the liberty-maven-plugin external POV. The server.env is probably the most complicated so if we can make sense of it there we're probably good.

2. enablement

Would we need to subject the application of filtering to a switch? I think so. Otherwise you might unintentionally apply a substitution into server.xml from a project/system property with the same name as a Liberty server config substitution. So we couldn't just make this the new behavior by default.

[mikekuzak]

If I would want to pass this from a Helm Chart, what's the best way? As configMap, instead at build time?

[scottkurz]

@mikekuzak yes, you could parameterize Liberty config via Helm chart, using env vars directly, via ConfigMap or Secret, and/or other constructs like the conditional pieces in say this example.

It might be helpful to both parameterize at build time yet also build the image so that it can be parameterized as necessary during your deployment, (via Helm, yaml, etc.).

So note there are still a variety of ways to parameterize Liberty config at build time using Liberty server config variables, env variables, bootstrap properties, etc.. Since you can set these Liberty config values using Maven project properties and system properties, you can do things like control configuration with normal Maven profiles. You just can't use Maven filters themselves (which is probably only going to matter if you were using this same set of filters to provide variable values for use by other plugins, but not if you just needed a way to have a variable value supplied to Liberty config at build time but you didn't care as much how it was supplied).

This snippet shows a hint of what I mean #587 (comment) but if it's not clear let us know and we can explain more, here or in our Gitter chat room: https://gitter.im/OpenLiberty/developer-experience

[scottkurz]

UPDATE (2023-08-09):

• We did complete a design for a solution. This design was socialized and approved. This design was tracked along with issue: Support for filters (Maven variables) & WAR overlays - PART 1: WAR modules #1104. The reason that both were tracked together is that they both concerned the use of Maven filtering, at a high-level, though the design details needed to provide a full solution ended up very different.
  • As captured in 1104, the UFO design doc can be found at: https://ibm.box.com/s/k6zinwfzvwfzfp0fl4mv7fv9mbmuubfa
  • The relevant info for this issue of server config filtering can be found especially in slide 29 (as well as some of the overview, intro sections)
• This issue remains on the backlog. (Although other aspects of filtering via liberty-maven-plugin described in the 1104 design have been delivered).