Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cloning from Bitbucket mirror #796

Merged
merged 3 commits into from
Feb 10, 2024

Conversation

andrey-fomin
Copy link
Contributor

@andrey-fomin andrey-fomin commented Jan 10, 2024

A mirrored Git repository can be configured for fetching references.

The mirror is not used in the following cases:

  • If the source branch in a pull request resides in a different repository, the source branch is fetched from the primary repository while the target branch is fetched from the mirror.
  • During initial pull request scanning, the mirror isn't used because PullRequestSCMHead doesn't contain the hash needed for the build and SCMRevision is null. This is the current design limitation and can be refactored later.

Cloning from the mirror can only be used with native web-hooks since plugin web-hooks don't provide a mirror identifier.

For branches and tags, the mirror sync event is used. Thus, at cloning time, the mirror is already synchronized. However, in the case of a pull request event, there is no such guarantee. The plugin optimistically assumes that the mirror is synced and the required commit hashes exist in the mirrored repository at cloning time. If the plugin can't find the required hashes, it falls back to the primary repository.

Fixes #592

Co-authors:

Your checklist for this pull request

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side) and not your master branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or in Jenkins JIRA
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Did you provide a test-case? That demonstrates feature works or fixes the issue.

LOGGER.log(Level.INFO, "Received hook from Bitbucket. Processing push event on {0}/{1}",
new Object[] { owner, repository });
scmSourceReIndex(owner, repository);
new Object[] { owner, repositoryName });

Check failure

Code scanning / CodeQL

Log Injection High

This log entry depends on a
user-provided value
.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This problem is not caused by this pull request. It's already existing. I believe validation of incoming events is another big task and out of scope of this pull request.

A mirrored Git repository can be configured for fetching references.

The mirror is not used in the following cases:
- If the source branch in a pull request resides in a different
  repository, the source branch is fetched from the primary
  repository while the target branch is fetched from the mirror.
- During initial pull request scanning, the mirror isn't used because
  PullRequestSCMHead doesn't contain the hash needed for the build and
  SCMRevision is null. This is the current design limitation and
  can be refactored later.

Cloning from the mirror can only be used with native web-hooks since
plugin web-hooks don't provide a mirror identifier.

For branches and tags, the mirror sync event is used. Thus, at cloning
time, the mirror is already synchronized. However, in the case of a
pull request event, there is no such guarantee. The plugin
optimistically assumes that the mirror is synced and the required
commit hashes exist in the mirrored repository at cloning time. If the
plugin can't find the required hashes, it falls back to the primary repository.

Fixes jenkinsci#592

Co-authors:
- Andrey Fomin https://github.com/andrey-fomin
- Andrei Kouznetchik https://github.com/akouznetchik
- Eugene Mercuriev https://github.com/zamonier
@lifeofguenter lifeofguenter added the java Pull requests that update Java code label Feb 10, 2024
@lifeofguenter lifeofguenter merged commit 28d74e8 into jenkinsci:master Feb 10, 2024
15 of 16 checks passed
public R getPull() {
public AbstractGitSCMSource.SCMRevisionImpl getPull() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using this API in a private plugin and now I get an error:

java.lang.NoSuchMethodError: 'jenkins.scm.api.SCMRevision com.cloudbees.jenkins.plugins.bitbucket.PullRequestSCMRevision.getPull()'

I suppose rebuilding my plugin might fix that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rolled back to 866.vdea_7dcd3008e for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The private plugin is using this kind of code to get the Git commit ID of the source branch of a pull request:

        SCMRevision revision = SCMRevisionAction.getRevision(scmSource, run);

        // revision might be one of:
        //
        // * in <https://github.com/jenkinsci/bitbucket-branch-source-plugin/>:
        //   - com.cloudbees.jenkins.plugins.bitbucket.BitbucketGitSCMRevision
        //   - com.cloudbees.jenkins.plugins.bitbucket.PullRequestSCMRevision
        // * in <https://github.com/jenkinsci/git-plugin/>:
        //   - jenkins.plugins.git.GitBranchSCMRevision
        //
        // Not all of those are derived from
        // jenkins.plugins.git.AbstractGitSCMSource.SCMRevisionImpl,
        // so we cannot unconditionally cast to that and call getHash().
        if (revision instanceof PullRequestSCMRevision) {
            final PullRequestSCMRevision pullRevision = (PullRequestSCMRevision)revision;
            revision = pullRevision.getPull();
        }

AFAIK, this information is not available from scm-api-plugin and must instead be retrieved via the API of bitbucket-branch-source-plugin. This API is not annotated with any restriction like @NoExternalUse.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#817 appears intended to fix this binary incompatibility.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified that Bitbucket Branch Source 877.vb_b_d5243f6794, which includes #817, restores binary compatibility with my private plugin.

andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 12, 2024
In Bitbucket Service "http" is used as http link name,
but in Bitbucket Cloud "https" is used.

Bug was introduced in jenkinsci#796. Fixes jenkinsci#804.
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 12, 2024
In Bitbucket Server "http" is used as http link name,
but in Bitbucket Cloud "https" is used.

Bug was introduced in jenkinsci#796. Fixes jenkinsci#804.
lifeofguenter pushed a commit that referenced this pull request Feb 12, 2024
In Bitbucket Server "http" is used as http link name,
but in Bitbucket Cloud "https" is used.

Bug was introduced in #796. Fixes #804.
@@ -66,11 +71,12 @@ public class BitbucketSCMSourceBuilder extends SCMSourceBuilder<BitbucketSCMSour
*/
public BitbucketSCMSourceBuilder(@CheckForNull String id, @NonNull String serverUrl,
@CheckForNull String credentialsId, @NonNull String repoOwner,
@NonNull String repoName) {
@NonNull String repoName, @CheckForNull String mirrorId) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an incompatible change and I think it broke Blue Ocean: jenkinsci/bom#2922 (comment)

Either switch the mirrorId to a (possibly “fluid”) setter, or retain a @Deprecated constructor with the original signature.

andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 13, 2024
Problem was introduced in jenkinsci#796.
Was mentioned in comment jenkinsci#796 (comment)
lifeofguenter pushed a commit that referenced this pull request Feb 13, 2024
Problem was introduced in #796.
Was mentioned in comment #796 (comment)
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 14, 2024
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 15, 2024
lifeofguenter pushed a commit that referenced this pull request Feb 15, 2024
Problem was introduced in #796. Fixes #808.
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 25, 2024
Makes RefSpecsSCMSourceTrait working again.

Fixes jenkinsci#814.
See also jenkinsci#812.
Problem was introduced in jenkinsci#796.
lifeofguenter pushed a commit that referenced this pull request Feb 25, 2024
…der (#815)

Makes RefSpecsSCMSourceTrait working again.

Fixes #814.
See also #812.
Problem was introduced in #796.
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 28, 2024
Method `getPull` is used in some private plugins.
jenkinsci#796 changed signature of this method.

Now signature is returned back.
See also jenkinsci#817.
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Feb 28, 2024
Method `getPull` is used in some private plugins.
jenkinsci#796 changed signature of this method.

Now signature is returned back.
See also jenkinsci#817.
andrey-fomin added a commit to andrey-fomin/bitbucket-branch-source-plugin that referenced this pull request Mar 10, 2024
Previously token was fetched only once and saved in BitbucketSCMSource.

Fixes jenkinsci#808.
See also jenkinsci#810 and jenkinsci#796.
lifeofguenter pushed a commit that referenced this pull request Mar 10, 2024
Previously token was fetched only once and saved in BitbucketSCMSource.

Fixes #808.
See also #810 and #796.
String scmSourceRepository = scmSource.getRepository();
String pullRequestRepoOwner = head.getRepoOwner();
String pullRequestRepository = head.getRepository();
boolean prFromTargetRepository = pullRequestRepoOwner.equals(scmSourceRepoOwner)
Copy link

@PayBas PayBas Apr 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrey-fomin I've run into an issue with this.

When configuring a job with a Bitbucket branch source, if you enter the "Owner" the input is case-insensitive.
The "Repository Name" field below "Owner" will be populated correctly, regardless of case.

But then when you actually build a same-origin PR, the prFromTargetRepository will return false, leading to very hard to diagnose issues.

In my case I was severely confused when for 1 specific pipeline my refs were generated as:

"+refs/pull-requests/" + pullId + "/from:refs/remotes/@{remote}/" + headName

instead of the expected:

"+refs/heads/" + branchName + ":refs/remotes/@{remote}/" + branchName

All because I had entered the "Owner" as lower-case (which the Jenkins job config happily accepted), whereas in Bitbucket it is upper-case.

I think this line should be:

boolean prFromTargetRepository = pullRequestRepoOwner.equalsIgnoreCase(scmSourceRepoOwner)
    && pullRequestRepository.equalsIgnoreCase(scmSourceRepository);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature java Pull requests that update Java code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bitbucket Branch Source Smart Mirror Support
5 participants