[MRELEASE-1078] Support version policy to read and use SCM tags and commits

[nielsbasjes]

Following this checklist to help us incorporate your
contribution quickly and easily:

• Make sure there is a JIRA issue filed
  for the change (usually before you start working on it). Trivial changes like typos do not
  require a JIRA issue. Your pull request should address just this issue, without
  pulling in other changes.
• Each commit in the pull request should have a meaningful subject line and body.
• Format the pull request title like [MJAVADOC-XXX] - Fixes bug in ApproximateQuantiles,
  where you replace MJAVADOC-XXX with the appropriate JIRA issue. Best practice
  is to use the JIRA issue title in the pull request title and in the first line of the
  commit message.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Run mvn clean verify -Prun-its to make sure basic checks pass. A more thorough check will
  be performed on your pull request automatically.

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

• In any other case, please file an Apache Individual Contributor License Agreement.

This my implementation on creating the option to follow the ideas of Conventional Commits to automatically determine the next version.

This patch assumes the required changes for maven-scm are available:

• Jira issue SCM-977
• Pull request [SCM-977] Support for retrieving tags from the changelog maven-scm#135

Summary of the changes:

• Extended the VersionPolicyRequest to include the SCM information.
• New version policy that retrieves the ChangeLog from SCM (which includes the commit messages and the tags).
• An optional configuration string projectVersionPolicyConfig that contains the optional config for the version policy. NOTE: I have tried to make it a clean addition of the configuration for the maven-release-plugin but I have not been successful in adding the extra config schema to the plugin. So for now it is a String which (because it is XML) must be included as CDATA.
• I've added an integration test that shows it works and what the configuration looks like.

I'm really looking forward to your feedback on how I can improve this.

[michael-o]

I see a lot of changes which do not belong here, but are ultimately good to be applied. Can you move them into a separate PR? I'd be happy to review them.

[nielsbasjes]

I see a lot of changes which do not belong here, but are ultimately good to be applied. Can you move them into a separate PR? I'd be happy to review them.

https://issues.apache.org/jira/browse/MRELEASE-1096
#132

[nielsbasjes]

Note:

• I have rebased this on top of [MRELEASE-1096] Many cleanups and updates #132 to make sure it all keeps working. So the diff looks a lot bigger than it is.
• This requires a maven-scm that contains:
  • [SCM-977] Support for retrieving tags from the changelog.
  • i.e. the maven-scm master branch since 2022-05-27
  • So the CI build will fail.
• Also includes a few minor fixes around logging (only in IT tests) which will become a problem given the logging changes in the current maven-scm master branch.

[nielsbasjes]

I have a question for the reviewers (@michael-o):
This is essentially a plugin for a plugin for maven.
Is there a better way to do the configuration of this "plugin for a plugin for maven" ?

[michael-o]

I have a question for the reviewers (@michael-o): This is essentially a plugin for a plugin for maven. Is there a better way to do the configuration of this "plugin for a plugin for maven" ?

What do you mean? Maven Release Plugin is a not a plugin on top of a plugin. Maven Release Plugin => Maven Release => Maven SCM API => Maven SCM Providers

[nielsbasjes]

I have a question for the reviewers (@michael-o): This is essentially a plugin for a plugin for maven. Is there a better way to do the configuration of this "plugin for a plugin for maven" ?

What do you mean? Maven Release Plugin is a not a plugin on top of a plugin. Maven Release Plugin => Maven Release => Maven SCM API => Maven SCM Providers

The Conventional Commits part has specific configuration. That "next version calculator" is a plugin for the maven-release plugin.

[michael-o]

I have a question for the reviewers (@michael-o): This is essentially a plugin for a plugin for maven. Is there a better way to do the configuration of this "plugin for a plugin for maven" ?

What do you mean? Maven Release Plugin is a not a plugin on top of a plugin. Maven Release Plugin => Maven Release => Maven SCM API => Maven SCM Providers

The Conventional Commits part has specific configuration. That "next version calculator" is a plugin for the maven-release plugin.

I know see, it is not a plugin in Maven sense, since it is not a Maven plugin with mojos. You simply provide a new implementatino for an interface pluggable at runtime, no?

[nielsbasjes]

The Conventional Commits part has specific configuration. That "next version calculator" is a plugin for the maven-release plugin.

I know see, it is not a plugin in Maven sense, since it is not a Maven plugin with mojos.

Correct, this VersionPolicy is not a maven plugin, and as a consequence my editor (IntelliJ) does not support me in setting the right options.

You simply provide a new implementation for an interface pluggable at runtime, no?

Yes, and for this implementation there can be implementation specific configuration. I would like to offer settings configuration in the pom.xml of the project at hand.
Right now (and this is the best I was able to get running) you can see in maven-release-plugin/src/it/projects/prepare/ccsemver-policy/pom.xml that the config for this VersionPolicy is a single string in projectVersionPolicyConfig which contains an XML as CDATA... which I don't really like.

My question is simply: Is there a better way to do this?

<configuration>
  <projectVersionPolicyId>CCSemVerVersionPolicy</projectVersionPolicyId>

  <!-- projectVersionPolicyConfig is an XML structure:  -->
  <!-- versionTag: A regex with 1 capture group that MUST extract the project.version from the SCM tag. -->
  <!-- minorRules: A list regexes that will be matched against all lines in each commit message since   -->
  <!--             the last tag. If matched the next version is at least a MINOR update.                -->
  <!-- majorRules: A list regexes that will be matched against all lines in each commit message since   -->
  <!--             the last tag. If matched the next version is at least a MAJOR update.                -->
  <!-- If a match is found the commit will trigger either a minor or major version increase             -->
  <!-- instead of only a patch increase.  -->
  <projectVersionPolicyConfig>
    <![CDATA[
    <cCSemverConfig>
      <versionTag>^v([0-9]+(?:\.[0-9]+(?:\.[0-9]+)?)?)$</versionTag>
      <majorRules>
        <majorRule>^[a-zA-Z]+!(?:\([a-zA-Z0-9_-]+\))?: .*$</majorRule>
        <majorRule>^BREAKING CHANGE:.*$</majorRule>
      </majorRules>
      <minorRules>
        <minorRule>^feat(?:\([a-zA-Z0-9_-]+\))?: .*$</minorRule>
      </minorRules>
    </cCSemverConfig>
    ]]>
  </projectVersionPolicyConfig>

</configuration>

[michael-o]

There is an option to inject an Xpp3Dom object into the plugin config which in turn could be passed to your class.

[nielsbasjes]

There is an option to inject an Xpp3Dom object into the plugin config which in turn could be passed to your class.

Cool! Can you please point me towards this option on how to do this. Possibly even an example in another project? I would really like to have this in there.

[michael-o]

Good question. I don't know an example from the top of my head, but you can search maven-sources multirepo for Xpp3Dom. I am sure this will do. In Maven Settings we use this trick to inject config for a server to Maven Wagon. @rfscholte @hboutemy Do you know an example from the top of your head?

[michael-o]

Since this requires SCM 2, I'd recommend to try out the current snapshot of SCM 2 on top of this master. I know, that there will be failures...

[nielsbasjes]

I take the clean master branch and only change in the main pom.xml

-    <scmVersion>1.13.0</scmVersion>
+    <scmVersion>2.0.0-M2-SNAPSHOT</scmVersion>

Then I run mvn clean verify (without the -Prun-its !!)

Then the Maven Release Manager fails with a lot of errors (70 failures).
I get the same results with both java 8, 11 and 17

When looking through the errors I see these recurring ones:

• DefaultScmRepositoryConfiguratorTest.testGetConfiguredRepository:72 check provider expected:<s[vn]> but was:<s[tub-provider]>
• Expected text value 'release-label' but was 'original-tag'
• Expected text value 'scm:svn:file://localhost/tmp/scm-repo/tags/release-label' but was 'scm:svn:file://localhost/tmp/scm-repo/trunk/'
• ClassCast class org.apache.maven.scm.provider.ScmProviderRepositoryStub cannot be cast to class org.apache.maven.scm.provider.ScmProviderRepositoryWithHost

@michael-o Are these the failures you were referring to?
Are these caused by the scm change from 1.x to 2.x ?
I consider these errors something that should be fixed outside of this specific pull request. Do you agree?

[michael-o]

I take the clean master branch and only change in the main pom.xml

-    <scmVersion>1.13.0</scmVersion>
+    <scmVersion>2.0.0-M2-SNAPSHOT</scmVersion>

Then I run mvn clean verify (without the -Prun-its !!)

Then the Maven Release Manager fails with a lot of errors (70 failures). I get the same results with both java 8, 11 and 17

When looking through the errors I see these recurring ones:

* DefaultScmRepositoryConfiguratorTest.testGetConfiguredRepository:72 check provider expected:<s[vn]> but was:<s[tub-provider]>

* Expected text value 'release-label' but was 'original-tag'

* Expected text value 'scm:svn:file://localhost/tmp/scm-repo/tags/release-label' but was 'scm:svn:file://localhost/tmp/scm-repo/trunk/'

* ClassCast class org.apache.maven.scm.provider.ScmProviderRepositoryStub cannot be cast to class org.apache.maven.scm.provider.ScmProviderRepositoryWithHost

@michael-o Are these the failures you were referring to?

Correct.

Are these caused by the scm change from 1.x to 2.x ?

That is the question whether this is just a DI issue.

I consider these errors something that should be fixed outside of this specific pull request. Do you agree?

Yes, absolutely.

[nielsbasjes]

I checkout the master twice and did some debugging with both settings.
So far I have tracked the expected:<s[vn]> but was:<s[tub-provider]> down to the DefaultScmRepositoryConfigurator being constructed (@Inject) with two different implementations of the ScmManager: One gets DefaultScmManager (from Maven SCM) the other ScmManagerStub (test class of the maven release manager).

[nielsbasjes]

At this point my guess (I do not fully understand many of the details of the injection system used here) is that the root cause is related to some of the changes of #118 as I see changes in the ScmManagerStub parameters (like Priority) and also the PlexusJUnit4TestCase.java changes settings around the ScmManagerStub. @cstamas does this make sense?

[nielsbasjes]

Good question. I don't know an example from the top of my head, but you can search maven-sources multirepo for Xpp3Dom. I am sure this will do. In Maven Settings we use this trick to inject config for a server to Maven Wagon. @rfscholte @hboutemy Do you know an example from the top of your head?

I have tried to figure out how this could work and I have not succeeded. So if you guys can point me to an example then that would be awesome. My main problem is that I do not understand how things are loaded.

At this point I would put an extra method in the VersionPolicy interface to obtain a thing that understands the VersionPolicy specific config and somehow make that a part of the system that reads the main config. Then when a specific policy is actually used the config is available.

Any pointers towards making this possible is highly appreciated.

[michael-o]

@slawekjaranowski Can you help since you looked into XML-based mojo config for validation?

[nielsbasjes]

How shall we proceed? Commit the way it is now (the config is an xml within a cdata block) or wait till the config has been changed to the way I would like it (where I am not yet able to do that)?
@michael-o / @slawekjaranowski ?

[michael-o]

@nielsbasjes One thing I know for sure is Wagon config in settings.xml is Xpp3Dom. That is passed to Resolver down to Wagon. That could be a start: https://maven.apache.org/guides/mini/guide-http-settings.html

[nielsbasjes]

Just did a rebase upon the latest master branch.

[nielsbasjes]

@rfscholte @michael-o
I just had a long talk with @hboutemy and we also discussed this merge request.

As an option that I'm fine with; how about I split this merge request into 2 parts:

1. Extending the VersionPolicy api with the capabilities needed to make a new class of versionplicies possible (among which the conventional commits).
2. I publish the conventional commits version policy that uses these new api capabilities under my own name (i.e. outside Apache).

This way the level of code you guys need to maintain is much smaller and thus this pull request becomes easier(?) for you to review.

Just let me know what you prefer. I'm fine either way.

[michael-o]

@rfscholte @michael-o I just had a long talk with @hboutemy and we also discussed this merge request.

As an option that I'm fine with; how about I split this merge request into 2 parts:

1. Extending the VersionPolicy api with the capabilities needed to make a new class of versionplicies possible (among which the conventional commits).

2. I publish the conventional commits version policy that uses these new api capabilities under my own name (i.e. outside Apache).

This way the level of code you guys need to maintain is much smaller and thus this pull request becomes easier(?) for you to review.

Just let me know what you prefer. I'm fine either way.

This, I truly like. Extend and review API, then your implementation!

[nielsbasjes]

Ok, then I'll change this in the coming weeks.

[michael-o]

Ok, then I'll change this in the coming weeks.

Dank u!

[nielsbasjes]

Taking things out is much simpler than making things :)

I took out the version policy itself and the related documentation and comments.
I did leave some of the documentation in as it clarified what a VersionPolicy is.

This is the cleanest I could think of.

I have tested it by building my (out of project) VersionPolicy : https://github.com/nielsbasjes/conventional-commits-maven-release

[nielsbasjes]

I'll add a very small test to ensure these new connection points for the version policies.

[nielsbasjes]

Done.
I think this is good now.
@michael-o WDYT?

[nielsbasjes]

Note: This should really be squashed ...

[michael-o]

Since the actual implementation is now gone, the title of this PR as well as the JIRA issue should be renamed/rephrased. @nielsbasjes WDYT?

[nielsbasjes]

@michael-o Do you like this one?
Support for a VersionPolicy to use scm tags and commits.

When merging this should be put in the squashed commit message also.

[michael-o]

@michael-o Do you like this one? Support for a VersionPolicy to use scm tags and commits.

When merging this should be put in the squashed commit message also.

This looks good, I'd fine to it: Support version policy to read and use SCM tags and commits

WDYT?

[nielsbasjes]

Updated.

[nielsbasjes]

Thanks everyone.