Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gradle] Plugin release management #201

Merged

Conversation

jimschubert
Copy link
Member

@jimschubert jimschubert commented Jun 2, 2018

This applies steps necessary for publishing to Sonatype, including
sources, javadoc, jar and signing. Also includes full POM details, per
Sonatype requirements.

In my original implementation of the plugin, I hadn't realized maven-publish was an experimental update to maven. I had falsely assumed that it applied all necessary logic in the current diff of this PR, but a key requirement (POM signing) is not available in maven-publish.

PR checklist

  • Read the contribution guidelines.
  • Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh and ./bin/security/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\.
  • Filed the PR against the correct branch: Default: master.
  • Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

(details of the change, additional tests that have been done, reference to the issue for tracking, etc)

This applies steps necessary for publishing to Sonatype, including
sources, javadoc, jar and signing. Also includes full POM details, per
Sonatype requirements.
@jimschubert
Copy link
Member Author

I'm explicitly documenting these here, in case they're missed in the build.gradle comments.

Signing and deploying require the following properties:

  • signing.keyId
  • signing.password
  • signing.secretKeyRingFile
  • ossrhUsername
  • ossrhPassword

These can be defined as:

  • key-value pairs stored in ~/.gradle/gradle.properties
  • options passed during build (./gradlew -P signing.keyId=YOUR_ID)
  • environment variables (e.g. ORG_GRADLE_PROJECT_ossrhUsername=YOUR_USER)

Once defined, building and deploying the plugin is simply:

./gradlew assemble
./gradlew uploadArtifacts

* master:
  Fix javadoc in clioption (OpenAPITools#188)
  Build improvements (OpenAPITools#202)
  Prepare 3.0.1-SNAPSHOT (OpenAPITools#195)
@jimschubert
Copy link
Member Author

@wing328 @jmini if we want the gradle plugin to make it into 3.0.1, we'll want to merge this and work out any kinks in release. Response to my request for replacing org.openapitools:openapi-generator-gradle-plugin:3.0.0 is that artifacts synced to Central can't be deleted (see ticket).

As I understand Jeremie's recent changes, we'd be relying on current maven configuration to execute a deploy that copies artifacts/poms. If this doesn't work, my recommendation is to add ./gradle uploadArtifacts at the end of the CI script, as long as the configuration values listed above are accounted for.

@jmini
Copy link
Member

jmini commented Jun 5, 2018

Lets do a recap. Since the beginning I tell that to upload/deploy the generated jar to a distant maven repository we can use:

  • "1/" maven
  • "2/" gradle

I do not care. I can help with the maven way, I do not know enough of gradle in order to help.


Because having two artifacts with the same name can be really confusing, I have renamed the maven wrapper that calls gradle to openapi-generator-gradle-plugin-mvn-wrapper See #202
We now have:

  • openapi-generator-gradle-plugin built by gradle
  • openapi-generator-gradle-plugin-mvn-wrapper a helper pom that is not installed into the local repository and not deployed to any distant repository.

The role of the maven helper is to call gradle to build the gradle plugin as part of the maven build.


I also misused PR #202 to see if I was able to upload openapi-generator-gradle-plugin (after gradle build) as part of the openapi-generator-gradle-plugin-mvn-wrapper pom (my solution "1/").

This was not a clear success and I have reported my findings in #202.
Currently I am not working on it.


During the 3.0.0 release, my understanding is that @wing328 did a lot of steps manually. I found several issues during my tests, but I have discovered the gradle plugin problem too late (wrong pom and jar were picked up during the release process).

I am not sure to know what is requested for 3.0.1: I do not know if @wing328 wants to make the upload manually or not.
I also did not investigate how the signing is made by maven during a release.


I have no problem by doing the signing and the upload with gradle.
I have also no problem by calling a second build script at the end of the maven build.
If we have a solution that works it will be OK for me.

Requirements are:

  • having something that works for release and for snapshot.
  • having something that is documented and understood.
  • having the CI servers properly configured (for the moment this PR has 3 failing and 1 successful checks)
  • having something that requires as less as manual work as possible.

The long game is to move our entire build to gradle (see #200). So if we have an half-maven half-gradle solution for that moment, this is OK for me.

@jimschubert
Copy link
Member Author

@jmini

Let me respond to the points that you called out:

having something that works for release and for snapshot.

Changes in this PR work for release and for snapshot. With the configuration values I have listed above in place, you call ./gradlew uploadArtifacts after the build has occurred on master. I don't care if the build is through maven or separately through ./gradlew assemble. I have not added this to the CI scripts, again because the 5 required configs are not in place and this would fail if merged into master before those configs are added.

having something that is documented and understood.

Uploading to Sonatype repositories is well-documented on Sonatype's site. We can document ./gradlew assemble && ./gradlew uploadArtifacts separately if needed, but I don't see anywhere that we have documented the maven deployment process and I feel like your comment here is due to unfamiliarity with Gradle rather than a necessary requirement. To put it another way, I'm not sure what's meant by "understood" here; contributors don't need to understand how our artifacts make it to Sonatype anymore than is already documented by Sonatype, and maintainers would be relying on the CI scripts. Again, the 5 configuration values above (which are all "secrets") and two tasks are all that are needed to get our artifacts onto Sonatype. build.gradle handing release vs snapshots correctly is no more magic than maven doing the same thing. If this is an issue in this repository, we should move the gradle plugin to a separate repo as I originally suggested. I have no problem being the sole maintainer of an openapi-generator-gradle-plugin repository.

having the CI servers properly configured (for the moment this PR has 3 failing and 1 successful checks)

Build is failing because CI is not configured with the 5 required configurations as I have mentioned above. Again, I don't know how to configure these, and probably need assistance from @wing328 to get the values in place.

having something that requires as less as manual work as possible.

This is the whole point of this PR, and the 5 missing configurations mentioned above are required to get the PR completed.

@jmini
Copy link
Member

jmini commented Jun 6, 2018

@jimschubert: I was not implying that you are not meeting the requirements. I just listed them to have something explicit. Sorry if you understood that you should deliver something.

Nevertheless, with your additional explanations it becomes more clear (at least to me).

The documentation is not for contributors but for people doing a release (for the moment this task is made by @wing328, but I understood that his intention is to involve more people like you and me). I start with really low gradle knowledge and almost none online-CI knowledge (I am a Jenkins user - I never configured circleci, AppVeyor, Travis CI or Shippable)

From what I understood @wing328 gave you and me the necessary rights to move forward. I do not know if this include CI setup but I think so.

@wing328 wing328 added this to the 3.0.1 milestone Jun 7, 2018
@jmini
Copy link
Member

jmini commented Jun 7, 2018

@wing328: what about the CI jobs that are broken? My understanding is that we need to add some environment variables as described by @jimschubert somewhere in the CI config.

We do not want commit our private key and passwords to the repo do we?

For maven I guess that we have something similar in place (env variables or maven setting.xml file that is present during build not not inside the repository)

@jimschubert
Copy link
Member Author

I will be updating some scripts later (hopefully tonight). I'll comment when done.

* master:
  Add 'unblu inc.' to company list (OpenAPITools#246)
  put company list in alphabetical order (OpenAPITools#244)
  [jaxrs-spec] generate spec file (yaml) correctly (OpenAPITools#243)
  [C++] Adjust the names (script, sample folder, generator) to lang option (OpenAPITools#220)
  Add GMO Pepabo to company list (OpenAPITools#242)
  [Spring] Add apiFirst option (OpenAPITools#184)
  [cli] Write to stdout/stderr, allow redirection (OpenAPITools#207)
  [JAVA][Client] New object instead of null for empty POST request (OpenAPITools#98)
  Make yaml serialization deterministic (OpenAPITools#233)
  Add syntax highlighting to migration guide (OpenAPITools#237)
  Fix shippable badge (OpenAPITools#232)
  update company list (OpenAPITools#227)
  Fix OpenAPITools#210: [Ada] Update the code generator for required and optional parameters (OpenAPITools#211)
  Delete unused methods in DefaultCodegen (OpenAPITools#209)
  add note about maven plugins (OpenAPITools#216)
  add raiffeisen to company list (OpenAPITools#223)
  add a remark about homebrew installatio (OpenAPITools#217)
@jimschubert
Copy link
Member Author

I've updated the scripts. Release configuration can be done later, as it's also not in place for Maven. We were operating under a false assumption there. ;)

It's a simple add, but considering it's related to all artifacts, I'd like to get this merged and verify that the SNAPSHOT gets published before we tackle automating the release for all artifacts in maven and gradle.

@jmini
Copy link
Member

jmini commented Jun 8, 2018

Yes we need to move forward on this, especially if the 3.0.1 release is tomorrow...

If the signing config is not there, can we not just ignore the signArchives task?

For the moment:

continuous-integration/appveyor/pr — AppVeyor build failed

> Task :signArchives FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Could not evaluate onlyIf predicate for task ':signArchives'.
> Unable to retrieve secret key from key ring file 'C:\projects\openapi-generator\modules\openapi-generator-gradle-plugin\???' as it does not exist
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 50s

And:

continuous-integration/travis-ci/pr — The Travis CI build failed

> Task :signArchives FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Could not evaluate onlyIf predicate for task ':signArchives'.
> Unable to retrieve secret key from key ring file '/home/travis/build/OpenAPITools/openapi-generator/modules/openapi-generator-gradle-plugin/???' as it does not exist
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org

(excuse me if this is a gradle beginner question)

@jimschubert
Copy link
Member Author

The condition doesn't sign artifacts unless in a non-SNAPSHOT. These errors look more related to my using ??? as placeholders.

I'll try and set aside some time this afternoon to setup the signing. I discussed it with William last night as well, so I don't know if he looked at it yet. For 3.0.0, William did it manually, so those configs weren't in place for Maven either.

@jimschubert jimschubert force-pushed the gradle-plugin-release-management branch from f746932 to 7e0a515 Compare June 9, 2018 02:16
@jimschubert
Copy link
Member Author

This should be good now. Includes Sonatype publish for Maven and Gradle. Can't evaluate if it works until merged, which William and I discussed merging it Saturday in the morning.

@jimschubert
Copy link
Member Author

Merging this now to see if there are any hiccups that need ironing out before Monday.

@jimschubert jimschubert merged commit 5a332f3 into OpenAPITools:master Jun 9, 2018
@jimschubert jimschubert deleted the gradle-plugin-release-management branch June 9, 2018 18:21
jimschubert added a commit to jimschubert/openapi-generator that referenced this pull request Jun 12, 2018
* master: (26 commits)
  Prepare 3.0.1 release (OpenAPITools#280)
  [typescript-angular] strict type checking (OpenAPITools#218)
  [C++ server] Adjust the names (script, sample folder, generator) to lang option (OpenAPITools#250)
  Removed warnings for packages included in SDK for Net Core 2.0 (OpenAPITools#269)
  [cli] Completions command for suggestions (OpenAPITools#213)
  [Java][RestTemplate] Fix query parameter URL encoding (OpenAPITools#260)
  [cpp-qt5] Remove std::shared_ptr from Qt5 (OpenAPITools#267)
  Adds some links to the README (OpenAPITools#261)
  Update sec.gpg.enc to binary encoded secret
  Add gpg --check-trustdb, limit gpg to master
  Re-do encrypted gpg and reference in settings.xml
  Fix trailing semicolons in after_success Travis CI scripts
  Use ubuntu keyserver instead of mit (due to timeout)
  [gradle] Plugin release management (OpenAPITools#201)
  Updates small typo in qna.md (OpenAPITools#262)
  Fix ModelUtils.getUnusedSchema() (OpenAPITools#253)
  Add JaxRS to bin/ensure-up-to-date (OpenAPITools#248)
  feat(security): add cookie-auth support (OpenAPITools#240)
  Add 'unblu inc.' to company list (OpenAPITools#246)
  put company list in alphabetical order (OpenAPITools#244)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants