[JENKINS-75083] Build statuses do not appear in list of pull requests in Bitbucket Server #954

KalleOlaviNiemitalo · 2025-01-01T22:01:59Z

When posting a build status from a pull-request job to Bitbucket Server or Data Center, set the "ref" property to match the source branch, e.g. "refs/heads/myfeature". This fixes the problem that Bitbucket Server stopped showing build statuses in the list of pull requests when the plugin started using the newer build-status API in #930.

This PR does not change how the plugin posts a build status to Bitbucket Cloud. That case will have "ref": null for pull-request jobs, like before.

Fix JENKINS-75083.

Your checklist for this pull request

Make sure you are requesting to pull a topic/feature/bugfix branch (right side) and not your master branch!
Ensure that the pull request title represents the desired changelog entry
Please describe what you did
Link to relevant issues in GitHub or in Jenkins JIRA
Link to relevant pull requests, esp. upstream and downstream changes
Did you provide a test-case? That demonstrates feature works or fixes the issue.
Manually tested with Bitbucket Server; see [JENKINS-75083] Build statuses do not appear in list of pull requests in Bitbucket Server #954 (comment).

KalleOlaviNiemitalo · 2025-01-01T22:08:57Z

@nfalco79, does this need an issue in Jira?

KalleOlaviNiemitalo · 2025-01-01T22:22:05Z

The user interface of Bitbucket Server also supports these in build statuses:

Statistics on the number of passed/failed/skipped tests. These can be posted to the current build-status API. I'm planning to implement that in a separate PR for Bitbucket Branch Source.
Hyperlinks to artifacts and to build logs. The URLs for those cannot be set via the REST API (feature request BSERV-12946). Instead, Bitbucket Server seems to recognize the Bitbucket Server Integration plugin specifically and guess the URLs. The guess isn't even correct, as the artifacts link points to the Jenkins workspace and not to the artifacts.
Actions. This doesn't seem to be supported in the REST API, either.

nfalco79 · 2025-01-01T23:36:19Z

@nfalco79, does this need an issue in Jira?

Could be better to keep track of undocumented behaviour

nfalco79 · 2025-01-01T23:51:46Z

The user interface of Bitbucket Server also supports these in build statuses:

Statistics on the number of passed/failed/skipped tests. These can be posted to the current build-status API. I'm planning to implement that in a separate PR for Bitbucket Branch Source.

I know but this is a SCM plugin I did not want add dependency to the junt test (or competitor) just to provide something supported only by the server instance (for which I do not have long plan). When I added the refname attribute I had in mind a possible extension point (limited to extra fields only) to be implemented in other Bitbucket plugins.

KalleOlaviNiemitalo · 2025-01-02T00:05:42Z

I first tried installing the HPI built from this PR, but got an error

java.io.IOException: Failed to load: Bitbucket Branch Source Plugin (cloudbees-bitbucket-branch-source 1000.0.0-SNAPSHOT (private-de40f204-kalle))
 - Update required: Git plugin (git 5.6.0) to be updated to 5.6.1-rc5349.5d1753d390a_7 or higher

Not sure whether that is by design or whether I did something wrong with Maven.

I then rebased onto 933.1.0, got commit dbf3823, built HPI from that, and installed. The result seems to be working in a pull request from the same repository, and in a pull request from a fork; I did not test a pull request to a fork. This version incorrectly shows 933.1.0 as its version number though; I should have rebased onto 737df70 instead.

It'll probably take a few days before I continue this.

nfalco79 · 2025-01-02T00:08:58Z

Is this related to JENKINS-74970 ?

KalleOlaviNiemitalo · 2025-01-02T00:11:51Z

Not really. JENKINS-74970 is specific to pull requests from a fork, and the fix would be in error handling and logging. The issue that this PR fixes is with all pull requests, whether from the same repository or from a fork.

nfalco79 · 2025-01-02T00:13:34Z

Not sure whether that is by design or whether I did something wrong with Maven.

It's because in master there is a bug fix that requires a release of the git-plugin (some people are waiting for it).
Branch 933.x is where I'm releasing now. So if you want I get this merged than set 933.x as the target branch, After that I will cherry pick commit into master.

KalleOlaviNiemitalo · 2025-01-02T12:33:45Z

Filed JENKINS-75082 to request posting test results to Bitbucket Server.

KalleOlaviNiemitalo · 2025-01-02T12:56:00Z

Filed JENKINS-75083 for the problem that this PR is intended to fix.

KalleOlaviNiemitalo · 2025-01-02T12:59:39Z

I intend to test some more with other PR build strategies before I mark this ready to review.

KalleOlaviNiemitalo · 2025-01-03T15:34:15Z

I rebased this PR onto master again, because 933.2.0 was released from master and not from 933.x

KalleOlaviNiemitalo · 2025-01-04T10:29:32Z

Manually tested scenarios with commit c8c38ab and Bitbucket Server.

Jenkins configured to build "Both the current pull request revision and the pull request merged with the current target branch revision":
- Pull request from the same repository.
  - In the "Builds" column on the "Pull requests" page in Bitbucket, a progress symbol appears first, and is replaced with a checkmark when the builds complete. When I hover the mouse pointer on that, it shows a popup "2 builds passed". 🆗
  - In the "Overview" tab on the page of the pull request in Bitbucket, it shows "2 builds" and a checkmark. 🆗
  - In the "Builds" column in the "Commits" tab on the page of the pull request in Bitbucket, it shows a checkmark. When I hover the mouse pointer on that, it shows a popup "2 builds". 🆗
  - In the "Builds" tab on the page of the pull request in Bitbucket, it shows one build for "-head" and another for "-merge". 🆗
- Pull request from fork2 to fork1 in the same user account. The parent repository is not in a personal account. Jenkins is granted READ access to both forks and configured to search for pull requests in fork1. Webhooks are not configured in these forks but a multibranch pipeline scan is manually triggered in Jenkins.
  - In the "Builds" column on the "Pull requests" page of fork1 in Bitbucket, a progress symbol appears first, and is replaced with a checkmark when the builds complete. When I hover the mouse pointer on that, it shows a popup "2 builds passsed". 🆗
  - On the "Pull requests" page of fork2 in Bitbucket, there are no pull requests. 🆗
  - On the "Branches" page of fork2 in Bitbucket, the "jenkins-test-cross-fork-pr" branch shows "OPEN" in the "Pull requests" column and a checkmark in the "Builds" column. When I hover the mouse pointer on that, it shows a popup "2 builds passed". 🆗
  - On the "Commits" page of fork2 in Bitbucket, the newest commit of the "jenkins-test-cross-fork-pr" branch shows a checkmark in the "Builds" column. When I hover the mouse pointer on that, it shows a popup "2 builds passed". 🆗
  - In the "Overview" tab on the page of the pull request in fork1 in Bitbucket, it shows "2 builds" and a checkmark. 🆗
  - In the "Builds" column in the "Commits" tab on the page of the pull request in fork1 in Bitbucket, it shows a checkmark. When I hover the mouse pointer on that, it shows a popup "2 builds". 🆗
  - In the "Builds" tab on the page of the pull request in fork1 in Bitbucket, it shows one build for "-head" and another for "-merge". 🆗
  - Merged this pull request from fork2 to fork1. Then, in the "Builds" column on the "Commits" page of fork1 in Bitbucket, it does not show any build of the commit that was built in the PR job. I suppose that is because the build status was posted to fork2 and not to fork1. It might be better to show that build here, but not showing it is also tolerable. 🆗

jonesbusy · 2025-01-04T12:02:47Z

for which I do not have long plan

What does this mean ? What about Data Center users ?

nfalco79 · 2025-01-04T13:22:45Z

I express my concern about spend double effort to support datacenter instance when there is already other plugin developed and mantained by atlassian itself. So this year I would discuss with other plugin developers about this situation and decide what do.

jonesbusy · 2025-01-04T15:58:18Z

The question would be, why the atlassian-bitbucket-server-integration was developed years (arround 2019) way after the bitbucket-branch-source-plugin

I tested several times since it's inception the atlassian-bitbucket-server-integration and never my needs (for example poor JobDSL integration, missing traits, not possible to override the remote name JENKINS-67890, many dependencies instead of API plugin (https://github.com/jenkinsci/atlassian-bitbucket-server-integration-plugin/blob/master/pom.xml#L54-L79) etc... And some other issues that I don't remember now

KalleOlaviNiemitalo · 2025-01-04T16:26:05Z

I would like to keep this plugin compatible with on-premises Bitbucket for now, but I am not very experienced with Java and cannot make large changes quickly. @nfalco79, are you planning some changes to this plugin that would be easier if compatibility with Bitbucket Data Center and Server were removed?

KalleOlaviNiemitalo · 2025-01-04T19:33:08Z

If you are planning large changes to this plugin and don't want to spend time adapting the Bitbucket Server client to them, then I would like to know in advance so that I can try to get time for that.

If Bitbucket Server support is removed from this plugin, I think I'll try to maintain it in a private fork and port security fixes to that, while investigating whether it is feasible to port pipelines to the Bitbucket Server Integration plugin. But I don't entirely trust Atlassian's intentions with that plugin because of their stated goal to move customers to cloud services. So there's a risk that they'd make some intolerable change to it or shut it down.

jonesbusy · 2025-01-04T19:44:30Z

I think Bitbucket DataCenter support is crucial for this plugin and it's 70'000 installation. I doubt there is only Cloud user using it

While I highly appreciate many bugs and feature were resolved recently, I think due to this plugin adoption DataCenter support cannot be removed (As a reminder https://www.jenkins.io/project/governance/#compatibility-matters)

I also volunteer also to give help for this plugin

nfalco79 · 2025-01-04T19:45:22Z

I tested several times since it's inception the atlassian-bitbucket-server-integration and never my needs (for example poor JobDSL integration, missing traits, not possible to override the remote name JENKINS-67890, many dependencies instead of API plugin (https://github.com/jenkinsci/atlassian-bitbucket-server-integration-plugin/blob/master/pom.xml#L54-L79) etc... And some other issues that I don't remember now

Ok this is sad to hear, this means I have to support both scenarios for years until the support has been improved by atlassian team.

nfalco79 · 2025-01-04T20:03:33Z

I would like to keep this plugin compatible with on-premises Bitbucket for now, but I am not very experienced with Java and cannot make large changes quickly. @nfalco79, are you planning some changes to this plugin that would be easier if compatibility with Bitbucket Data Center and Server were removed?

I don't plan to remove any current features, but I won't implement any new features specific to the datacenter (as was done for the mirror feature), relying only on the documentation without even the possibility to test it myself. This means that the development of new features will be fast for the cloud version while it may not work properly, or at all, in the datacenter version.

What I can say is that since I will be replacing the deprecated Bitbucket server APIs with the new ones, this plugin will no longer support the server version declared in EOL. I mean... server version older than 7.x will no longer work. I don't want to complicate the logic of this plugin that not even Atlassian wants to provide support for.

Any help from datecenter users in bugfix (like this) and test like this feature is/will be welcome.

jonesbusy · 2025-01-04T20:14:41Z

I agreed for the EOL. I doesn't make sense to support a datacenter (or server) version that is end of life. If for some reason users cannot upgrade, they can still stay on an older version of the plugin

nfalco79 · 2025-01-04T20:18:03Z

OffTopic

I have a flexible plan, but what I would do in the next months is (the order does not mean priority):

upgrade client to apache http 5.x
resolve any issue in github (because I have to close it)
rework or remove the avatar, an aesthetic feature cannot compromise functionality
separate what is API and what is not
create some extension points so that users can provide custom implementations:
- migrate some implementation (like plugin webhook) outside this plugin
- allow user customize build notification (not totally) like messages, title or test result
- webhook processor
- ... other things, depends on the need

KalleOlaviNiemitalo · 2025-01-04T21:30:19Z

@nfalco79 do you have experience on Apache http vs. okhttp vs. java.net.http?

I recently ported some code from okhttp to java.net.http (to minimise dependencies) and the API was pretty similar; but java.net.http apparently supports HTTP Basic authentication only and it is not possible to provide an authenticator object that would handle other authentication schemes. Although this can be worked around by making another HttpRequest.

nfalco79 · 2025-01-05T00:50:42Z

I want migrate to http 5.x because of this

bitbucket-branch-source-plugin/src/main/java/com/cloudbees/jenkins/plugins/bitbucket/impl/client/ExponentialBackOffServiceUnavailableRetryStrategy.java

Line 43 in dcd189d

    
           private ThreadLocal<Integer> consecutiveFailedAttempts; // TODO call ThreadLocal#remove method is not possible using the lifecycle of apache client 4.x. Move to http client 5.x ASAP

Apache client has too many feature that jdk does not support and I have write from scratch by myself.

KalleOlaviNiemitalo · 2025-01-05T20:46:26Z

I think Bitbucket DataCenter support is crucial for this plugin and it's 70'000 installation. I doubt there is only Cloud user using it

Perhaps a telemetry experiment could be done. The payload would be based on the "Bitbucket Endpoints" list in the "System" configuration:

Are there any "Bitbucket Cloud" endpoints? (yes/no)
Are there any "Bitbucket Server" endpoints? (yes/no)

If the plugin was installed only to satisfy dependencies, then both would be "no".

However, the results of this telemetry could be misleading if the company has migrated all projects out of Bitbucket Server but has not unregistered the defunct endpoint from Jenkins.

… in Bitbucket Server (#954) Set "ref" in PR build status for Bitbucket Server

The commit 7ac886d switched to the newer repository commit build status api. This introduced a regression in the case the PR source repository is not the same as the target, for example a person fork. In this case the bitbucket api created from the PR head will point to the fork repo and may result in a 401 error if the credentials don't have the necessary permissions. ``` ERROR: Could not send notifications com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: HTTP/1.1 401 Response: {"errors":[{"context":null,"message":"You are not permitted to access this resource","exceptionName":"com.atlassian.bitbucket.AuthorisationException"}]} ``` This change will force the bitbucket api instance to be the target repo which is the appropriate target for the notification. The old api was commit hash centric, not project centric, so the api client owner/repo didn't matter. This change also reverts changes introduced by jenkinsci#954 to the refName parameter. Since notification is on the targget repo, the head ref branch may not exist on the target and result in no status update. No changes were made to the Bitbucket Cloud code path.

The commit 7ac886d switched to the newer repository commit build status api. This introduced a regression in the case the PR source repository is not the same as the target, for example a person fork. In this case the bitbucket api created from the PR head will point to the fork repo and may result in a 401 error if the credentials don't have the necessary permissions. ``` ERROR: Could not send notifications com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: HTTP/1.1 401 Response: {"errors":[{"context":null,"message":"You are not permitted to access this resource","exceptionName":"com.atlassian.bitbucket.AuthorisationException"}]} ``` This change will force the bitbucket api instance to be the target repo which is the appropriate target for the notification. The old api was commit hash centric, not project centric, so the api client owner/repo didn't matter. This change also reverts some changes introduced by jenkinsci#954 to the refName parameter. While local testing has shown that by changing the bitbucket client to point to the target repo, setting the refName to null works in all cases, this change has been restricted to only cases when the source repo is the same as the target. No changes were made to the Bitbucket Cloud code path.

KalleOlaviNiemitalo changed the title ~~Set refname in PR build status for Bitbucket Server~~ Set "ref" in PR build status for Bitbucket Server Jan 1, 2025

KalleOlaviNiemitalo force-pushed the bserv-build-status branch from dbf4b14 to 26c904d Compare January 1, 2025 22:06

KalleOlaviNiemitalo force-pushed the bserv-build-status branch 2 times, most recently from ad6e5d5 to de40f20 Compare January 1, 2025 22:34

This comment was marked as outdated.

Sign in to view

KalleOlaviNiemitalo force-pushed the bserv-build-status branch from de40f20 to 50464ae Compare January 2, 2025 09:07

KalleOlaviNiemitalo changed the base branch from master to 933.x January 2, 2025 09:07

KalleOlaviNiemitalo changed the title ~~Set "ref" in PR build status for Bitbucket Server~~ [JENKINS-75083] Build statuses do not appear in list of pull requests in Bitbucket Server Jan 2, 2025

nfalco79 added the bug label Jan 2, 2025

Set "ref" in PR build status for Bitbucket Server

c8c38ab

KalleOlaviNiemitalo force-pushed the bserv-build-status branch from 50464ae to c8c38ab Compare January 3, 2025 15:29

KalleOlaviNiemitalo changed the base branch from 933.x to master January 3, 2025 15:30

KalleOlaviNiemitalo marked this pull request as ready for review January 4, 2025 10:30

nfalco79 approved these changes Jan 4, 2025

View reviewed changes

nfalco79 merged commit d3c8e0d into jenkinsci:master Jan 4, 2025
15 checks passed

nfalco79 pushed a commit that referenced this pull request Jan 5, 2025

[JENKINS-75083] Build statuses do not appear in list of pull requests…

55ea389

… in Bitbucket Server (#954) Set "ref" in PR build status for Bitbucket Server

cronik mentioned this pull request Mar 1, 2025

fix: bitbucket service pull request build status from different source #998

Closed

6 tasks

[JENKINS-75083] Build statuses do not appear in list of pull requests in Bitbucket Server #954

[JENKINS-75083] Build statuses do not appear in list of pull requests in Bitbucket Server #954

Uh oh!

Conversation

KalleOlaviNiemitalo commented Jan 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Your checklist for this pull request

Uh oh!

KalleOlaviNiemitalo commented Jan 1, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 1, 2025

Uh oh!

nfalco79 commented Jan 1, 2025

Uh oh!

This comment was marked as outdated.

nfalco79 commented Jan 1, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 2, 2025

Uh oh!

nfalco79 commented Jan 2, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nfalco79 commented Jan 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KalleOlaviNiemitalo commented Jan 2, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 2, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 2, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 3, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

jonesbusy commented Jan 4, 2025

Uh oh!

nfalco79 commented Jan 4, 2025

Uh oh!

jonesbusy commented Jan 4, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 4, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonesbusy commented Jan 4, 2025

Uh oh!

nfalco79 commented Jan 4, 2025

Uh oh!

nfalco79 commented Jan 4, 2025

Uh oh!

jonesbusy commented Jan 4, 2025

Uh oh!

nfalco79 commented Jan 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KalleOlaviNiemitalo commented Jan 4, 2025

Uh oh!

nfalco79 commented Jan 5, 2025

Uh oh!

KalleOlaviNiemitalo commented Jan 5, 2025

Uh oh!

Uh oh!

KalleOlaviNiemitalo commented Jan 1, 2025 •

edited

Loading

KalleOlaviNiemitalo commented Jan 2, 2025 •

edited

Loading

nfalco79 commented Jan 2, 2025 •

edited

Loading

KalleOlaviNiemitalo commented Jan 4, 2025 •

edited

Loading

KalleOlaviNiemitalo commented Jan 4, 2025 •

edited

Loading

nfalco79 commented Jan 4, 2025 •

edited

Loading