Jenkins task in Blueprint created by script is not working vs blueprint created by using UI works correctly #8422

khalibartan · 2025-05-03T13:26:00Z

khalibartan
May 3, 2025

Hi team I struggling where I created a script to programatically configure blue prints for repos, but I am struggling where jenkins steps is not working correctly as it is not fetching the intended builds (few of the blueprints it worked). While it works correctly when i create from UI. I have checked my jenkins related tables there is no difference in data once i create from UI vs script. I am very much confused because blueprints are exactly same. Here I am sharing example blueprints to help deep dive.

Blueprint created via script

[
    [
        {
            "options": {
                "projectMappings": [
                    {
                        "projectName": "alfred",
                        "scopes": [
                            {
                                "rowId": "github:GithubRepo:1:686731171",
                                "table": "repos"
                            },
                            {
                                "rowId": "github:GithubRepo:1:686731171",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "github:GithubRepo:1:686731171",
                                "table": "boards"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-prod-replica",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-prod",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-qa",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-stage-replica",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-stage",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-uat",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-old-prod",
                                "table": "cicd_scopes"
                            }
                        ]
                    }
                ]
            },
            "plugin": "org",
            "subtasks": [
                "setProjectMapping"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionId": 1,
                "fullName": "dreamplug-tech/alfred",
                "githubId": 686731171,
                "name": "alfred"
            },
            "plugin": "github",
            "subtasks": [
                "Collect Comments",
                "Collect Events",
                "Collect Issues",
                "Collect Milestones",
                "Collect PR Review Comments",
                "Collect Pull Requests",
                "Collect Users",
                "Collect Workflow Runs",
                "Convert Releases",
                "Convert Repos",
                "Extract Comments",
                "Extract Events",
                "Extract Issues",
                "Extract Milestones",
                "Extract PR Review Comments",
                "Extract Pull Requests",
                "Extract Users",
                "Extract Workflow Runs",
                "Collect Job Runs",
                "Collect PR Commits",
                "Collect PR Reviews",
                "Convert Issue Assignees",
                "Convert Issue Comments",
                "Convert Issue Labels",
                "Convert Issues",
                "Convert Milestones",
                "Convert PR Labels",
                "Convert Pull Requests",
                "Convert Workflow Runs",
                "Enrich PR Issues",
                "Convert PR Issues",
                "Extract Jobs",
                "Extract PR Commits",
                "Extract PR Reviews",
                "Collect User Org",
                "Convert Jobs",
                "Convert PR Comments",
                "Convert PR Commits",
                "Convert PR Reviews",
                "Convert Reviews",
                "Extract User Org",
                "Convert Users"
            ]
        },
        {
            "options": {
                "connectionId": 1,
                "fullName": "dreamplug-tech/alfred",
                "name": "alfred",
                "pluginName": "gitextractor",
                "proxy": "",
                "repoId": "github:GithubRepo:1:686731171",
                "url": "ssh://git@github.com/dreamplug-tech/alfred.git"
            },
            "plugin": "gitextractor",
            "subtasks": null
        },
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-prod-replica"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-prod"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-qa"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-stage-replica"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-stage"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-uat"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-old-prod"
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "projectName": "alfred"
            },
            "plugin": "dora",
            "subtasks": [
                "generateDeployments",
                "generateDeploymentCommits",
                "enrichPrevSuccessDeploymentCommits"
            ]
        },
        {
            "options": {
                "projectName": "alfred",
                "scopeIds": null
            },
            "plugin": "issue_trace",
            "subtasks": [
                "ConvertIssueStatusHistory",
                "ConvertIssueAssigneeHistory"
            ]
        }
    ],
    [
        {
            "options": {
                "projectName": "alfred"
            },
            "plugin": "refdiff",
            "subtasks": [
                "calculateDeploymentCommitsDiff"
            ]
        }
    ],
    [
        {
            "options": {
                "projectName": "alfred"
            },
            "plugin": "dora",
            "subtasks": [
                "calculateChangeLeadTime",
                "ConvertIssuesToIncidents",
                "ConnectIncidentToDeployment"
            ]
        }
    ]
]

Blueprint created via UI

[
    [
        {
            "options": {
                "projectMappings": [
                    {
                        "projectName": "alfred-test",
                        "scopes": [
                            {
                                "rowId": "github:GithubRepo:1:686731171",
                                "table": "repos"
                            },
                            {
                                "rowId": "github:GithubRepo:1:686731171",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "github:GithubRepo:1:686731171",
                                "table": "boards"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-old-prod",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-prod",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-prod-replica",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-qa",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-stage",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-stage-replica",
                                "table": "cicd_scopes"
                            },
                            {
                                "rowId": "jenkins:JenkinsJob:1:alfred-uat",
                                "table": "cicd_scopes"
                            }
                        ]
                    }
                ]
            },
            "plugin": "org",
            "subtasks": [
                "setProjectMapping"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionId": 1,
                "fullName": "dreamplug-tech/alfred",
                "githubId": 686731171,
                "name": "dreamplug-tech/alfred"
            },
            "plugin": "github",
            "subtasks": [
                "Collect Comments",
                "Collect Events",
                "Collect Issues",
                "Collect Milestones",
                "Collect PR Review Comments",
                "Collect Pull Requests",
                "Collect Users",
                "Collect Workflow Runs",
                "Convert Releases",
                "Convert Repos",
                "Extract Comments",
                "Extract Events",
                "Extract Issues",
                "Extract Milestones",
                "Extract PR Review Comments",
                "Extract Pull Requests",
                "Extract Users",
                "Extract Workflow Runs",
                "Collect Job Runs",
                "Collect PR Commits",
                "Collect PR Reviews",
                "Convert Issue Assignees",
                "Convert Issue Comments",
                "Convert Issue Labels",
                "Convert Issues",
                "Convert Milestones",
                "Convert PR Labels",
                "Convert Pull Requests",
                "Convert Workflow Runs",
                "Enrich PR Issues",
                "Convert PR Issues",
                "Extract Jobs",
                "Extract PR Commits",
                "Extract PR Reviews",
                "Collect User Org",
                "Convert Jobs",
                "Convert PR Comments",
                "Convert PR Commits",
                "Convert PR Reviews",
                "Convert Reviews",
                "Extract User Org",
                "Convert Users"
            ]
        },
        {
            "options": {
                "connectionId": 1,
                "fullName": "dreamplug-tech/alfred",
                "name": "dreamplug-tech/alfred",
                "pluginName": "gitextractor",
                "proxy": "",
                "repoId": "github:GithubRepo:1:686731171",
                "url": "ssh://git@github.com/dreamplug-tech/alfred.git"
            },
            "plugin": "gitextractor",
            "subtasks": null
        },
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-old-prod",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-prod",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-prod-replica",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-qa",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-stage",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-stage-replica",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "connectionEndpoint": "",
                "connectionId": 1,
                "fullName": "alfred-uat",
                "scopeConfig": null
            },
            "plugin": "jenkins",
            "subtasks": [
                "collectApiJobs",
                "extractApiJobs",
                "convertJobs",
                "collectApiBuilds",
                "extractApiBuilds",
                "collectApiStages",
                "extractApiStages",
                "enrichApiBuildWithStages",
                "convertBuildsToCICD",
                "convertStages",
                "convertBuildRepos"
            ]
        }
    ],
    [
        {
            "options": {
                "projectName": "alfred-test"
            },
            "plugin": "dora",
            "subtasks": [
                "generateDeployments",
                "generateDeploymentCommits",
                "enrichPrevSuccessDeploymentCommits"
            ]
        },
        {
            "options": {
                "projectName": "alfred-test",
                "scopeIds": null
            },
            "plugin": "issue_trace",
            "subtasks": [
                "ConvertIssueStatusHistory",
                "ConvertIssueAssigneeHistory"
            ]
        }
    ],
    [
        {
            "options": {
                "projectName": "alfred-test"
            },
            "plugin": "refdiff",
            "subtasks": [
                "calculateDeploymentCommitsDiff"
            ]
        }
    ],
    [
        {
            "options": {
                "projectName": "alfred-test"
            },
            "plugin": "dora",
            "subtasks": [
                "calculateChangeLeadTime",
                "ConvertIssuesToIncidents",
                "ConnectIncidentToDeployment"
            ]
        }
    ]
]

@dosu · 2025-05-03T13:28:11Z

dosubot[bot]
bot May 3, 2025

The issue you're experiencing with the Jenkins tasks in the script-generated blueprint might be related to the connectionEndpoint being empty in the Jenkins plugin configuration. The connectionEndpoint is crucial for establishing a connection to the Jenkins server. If it's empty, the plugin won't be able to communicate with Jenkins, which could lead to the failure in fetching builds ^[1]^[2].

To resolve this, ensure that the connectionEndpoint is correctly set in your script-generated blueprint. It should match the endpoint used in the UI-generated blueprint that works correctly. This should help in establishing a proper connection and allow the Jenkins tasks to fetch the intended builds.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

0 replies

khalibartan · 2025-05-06T07:29:50Z

khalibartan
May 6, 2025
Author

@klesh can you help me on this?

0 replies

klesh · 2025-05-12T06:56:11Z

klesh
May 12, 2025
Collaborator

No errors?

0 replies

khalibartan · 2025-05-16T05:17:00Z

khalibartan
May 16, 2025
Author

@klesh no, not observing any errors in pipeline execution.

9 replies

khalibartan Jun 3, 2025
Author

yes scopes exists in the devlake, i verified via api and checked in db also.

khalibartan Jun 10, 2025
Author

@klesh how do i proceed here? Really stuck and unable to figure out a workaround here

khalibartan Jun 10, 2025
Author

Only difference is one has "scopeConfig": null in the template (blueprint created via UI) and blueprint created via api doesn't have any scopeConfig entry in template

marcoadasilvaa Jun 12, 2025

Similar problem here with gitlab repo and CI/CD

klesh Jun 16, 2025
Collaborator

@khalibartan Sorry, I have no idea what is wrong with the current information.

Jenkins task in Blueprint created by script is not working vs blueprint created by using UI works correctly #8422

Uh oh!

Uh oh!

khalibartan May 3, 2025

Replies: 4 comments · 9 replies

Uh oh!

dosubot[bot] bot May 3, 2025

Uh oh!

khalibartan May 6, 2025 Author

Uh oh!

klesh May 12, 2025 Collaborator

Uh oh!

Uh oh!

khalibartan May 16, 2025 Author

Uh oh!

khalibartan Jun 3, 2025 Author

Uh oh!

khalibartan Jun 10, 2025 Author

Uh oh!

khalibartan Jun 10, 2025 Author

Uh oh!

marcoadasilvaa Jun 12, 2025

Uh oh!

klesh Jun 16, 2025 Collaborator

khalibartan
May 3, 2025

Replies: 4 comments 9 replies

dosubot[bot]
bot May 3, 2025

khalibartan
May 6, 2025
Author

klesh
May 12, 2025
Collaborator

khalibartan
May 16, 2025
Author

khalibartan Jun 3, 2025
Author

khalibartan Jun 10, 2025
Author

khalibartan Jun 10, 2025
Author

klesh Jun 16, 2025
Collaborator