Script to reproduce the Conda Windows jobs locally #1279

j-rivero · 2025-03-12T19:39:11Z

The PR adds a local_build.bat command that can run the gz_foo-*-.bat scripts locally to reproduce the commands done in Jenkins. The README.md file contains detailed instructions about how to use the tool.

The script injects a temporary workspace to imitate the Jenkins directory and handles other required variables. The script uses as input a source code directory of one of the gz libraries and the corresponding script from the jenkins-scripts/*.bat files.

The tool can be tested (please follow the README.md documentation) but I would like to reduce the changes to the libraries used and split them in separate PRs before this is ready to merge.

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

Crola1702 · 2025-04-10T16:15:59Z

Testing locally (git bash):

python local_build.py gz_math-default-devel-windows-amd64.bat ../gz-math | tee local_build_gz-math.txt

local_build_gz-math.txt

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

j-rivero · 2025-04-14T15:09:42Z

Testing locally (git bash):

The script needs to be run from a cmd.exe environment. Added documentation and check to the script.

Crola1702 · 2025-04-16T19:12:06Z

Now testing with cmd: >python local_build.py gz_math-default-devel-windows-amd64.bat C:\Users\Crola1702\release-tools\gz-math

local_build_gz-math.txt

Error:

C:\Users\CROLA1~1\AppData\Local\Temp\58576\ws>echo "COLCON_EXTRA_ARGS: --packages-skip gz-math8"
"COLCON_EXTRA_ARGS: --packages-skip gz-math8"

C:\Users\CROLA1~1\AppData\Local\Temp\58576\ws>colcon build --build-base "build"  --install-base "install"  --parallel-workers 8  --packages-skip gz-math8  --cmake-args " -DCMAKE_BUILD_TYPE=Release"  "-DBUILD_TESTING=0" "-DCMAKE_CXX_FLAGS=-w"  --event-handler console_cohesion+   || goto :error
Starting >>> gz-cmake4
--- output: gz-cmake4
Traceback (most recent call last):
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_core\executor\__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_core\task\__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_ros\task\cmake\build.py", line 34, in build
    rc = await extension.build(
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_cmake\task\cmake\build.py", line 87, in build
    rc = await self._reconfigure(args, env)
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_cmake\task\cmake\build.py", line 153, in _reconfigure
    raise RuntimeError(
RuntimeError: VisualStudioVersion is not set, please run within a Visual Studio Command Prompt.
---
--- stderr: gz-cmake4
Traceback (most recent call last):
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_core\executor\__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_core\task\__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_ros\task\cmake\build.py", line 34, in build
    rc = await extension.build(
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_cmake\task\cmake\build.py", line 87, in build
    rc = await self._reconfigure(args, env)
  File "C:\Users\Crola1702\AppData\Local\Temp\pixi\project\.pixi\envs\default\lib\site-packages\colcon_cmake\task\cmake\build.py", line 153, in _reconfigure
    raise RuntimeError(
RuntimeError: VisualStudioVersion is not set, please run within a Visual Studio Command Prompt.
---
Failed   <<< gz-cmake4 [0.17s, exited with code 1]

Summary: 0 packages finished [0.66s]
  1 package failed: gz-cmake4
  1 package had stderr output: gz-cmake4
  1 package not processed
WNDPROC return value cannot be converted to LRESULT
TypeError: WPARAM is simple, so must be an int object (got NoneType)
VisualStudioVersion is not set, please run within a Visual Studio Command Prompt.

C:\Users\CROLA1~1\AppData\Local\Temp\58576\ws>echo Failed in windows_library with error #1.
Failed in windows_library with error #1.

C:\Users\CROLA1~1\AppData\Local\Temp\58576\ws>exit  1

 Local build finished

 FAILED

j-rivero · 2025-04-16T20:23:06Z

RuntimeError: VisualStudioVersion is not set, please run within a Visual Studio Command Prompt.

Do you have VS 2019 installed? If so, I would expect the conda package vs2019_win-64 to do the trick.

Crola1702 · 2025-04-17T01:40:26Z

Build succeeded.

Log output:

Summary: 706 tests, 0 errors, 0 failures, 0 skipped
# END SECTION
# END SECTION
# BEGIN SECTION: export testing results
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_AdditivelySeparableScalarField3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Angle_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_AxisAlignedBoxHelpers_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_AxisAlignedBox_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Box_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Capsule_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Color_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Cone_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Conversions_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_CoordinateVector3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Cylinder_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_DiffDriveOdometry_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Edge_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Ellipsoid_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Filter_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Frustum_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_GaussMarkovProcess_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_GraphAlgorithms_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_GraphDirected_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_GraphUndirected_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Graph_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Helpers_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Inertial_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_InterpolationPoint_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Interval_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Kmeans_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Line2_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Line3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_MassMatrix3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Material_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Matrix3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Matrix4_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Matrix6_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_MecanumDriveOdometry_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_MovingWindowFilter_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_OrientedBox_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_PID_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_PiecewiseScalarField3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Plane_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Polynomial3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Pose_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Quaternion_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Rand_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Region3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_RollingMean_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_RotationSpline_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_SemanticVersion_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_SignalStats_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_SpeedLimiter_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Sphere_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_SphericalCoordinates_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Spline_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Stopwatch_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Temperature_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_TimeVaryingVolumetricGridLookupField_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_TimeVaryingVolumetricGrid_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Triangle3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Triangle_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Util_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Vector2_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Vector3Stats_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Vector3_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Vector4_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_Vertex_TEST.xml
C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws\build\gz-math8\test_results\UNIT_VolumetricGridLookupField_TEST.xml
65 File(s) copied
# END SECTION

 Local build finished

 SUCCESS

  - Build root is C:\Users\CROLA1~1\AppData\Local\Temp\455037
  - Build workspace is C:\Users\CROLA1~1\AppData\Local\Temp\455037\ws

1;34mReproduce the call to the last build:
  - Only reusing pixi environment:
    - run 'C:\Users\Crola1702\release-tools\jenkins-scripts\gz_math-default-devel-windows-amd64.bat C:\Users\Crola1702\release-tools\gz-math --reuse-dependencies-environment'
  - Preparing pixi and colcon and go to the colcon workspace:
    - run 'call .debug_last_build.bat'

C:\Users\Crola1702\release-tools\jenkins-scripts>

Do you have VS 2019 installed?

:door_hide:

Anything else you need me to test? @j-rivero (or a specific package?)

Blast545 · 2025-08-19T14:46:50Z

@j-rivero Can we help you with this one? It would be great to merge this one

j-rivero · 2025-10-15T15:31:42Z

@j-rivero Can we help you with this one? It would be great to merge this one

Sorry for arrive so late. The script was pending an update so it can detect which Conda env needs to be used. That now is a wider PR that needs to be integrated into -pr- jobs, so I think we are good to merge this one.

j-rivero and others added 18 commits March 10, 2025 20:18

Temp file

722394a

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

WIP

395a34e

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

Relocate python dependant code

a16498d

Basic documentation for the script

111253f

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Use single var expansion

9a1d908

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Rework shell-hook

21a23b7

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Improve usability

c0f667f

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Improve documentation

6078b34

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Remove temp file

cd353a9

Signed-off-by: Jose Luis Rivero <jrivero@openrobotics.com>

Merge branch 'master' into jrivero/conda_local_build

a2eb690

Merge branch 'master' into jrivero/conda_local_build

d98eac9

Merge branch 'master' into jrivero/conda_local_build

7791b96

Merge branch 'master' into jrivero/conda_local_build

77b88f4

Fix bad merge of code

4cdde84

Remove comment

449795c

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

Replace build script with Python

dcb74c3

Need to be permissive

46f4629

Fix and color for the messages

569c110

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

j-rivero marked this pull request as ready for review March 14, 2025 18:54

j-rivero and others added 3 commits March 14, 2025 19:55

Update local_build.py

34c5975

Fix format in markdown

fe305e6

Merge branch 'master' into jrivero/conda_local_build

8cd55cb

j-rivero mentioned this pull request Apr 9, 2025

Conda Windows CI migration: execution plan #1247

Closed

24 tasks

local_build.py needs to be called from a cmd.exe

e253b72

Signed-off-by: Jose Luis Rivero <jrivero@honurobotics.com>

Merge branch 'master' into jrivero/conda_local_build

bb2a460

j-rivero requested a review from Crola1702 April 15, 2025 14:27

j-rivero added 3 commits May 12, 2025 16:29

Merge branch 'master' into jrivero/conda_local_build

bc0f2c4

-j is now supported by the script

a4d1e9a

Merge branch 'master' into jrivero/conda_local_build

f74a492

Merge branch 'master' into jrivero/conda_local_build

5d8b96e

j-rivero merged commit 0a68afa into master Oct 15, 2025
4 checks passed

j-rivero deleted the jrivero/conda_local_build branch October 15, 2025 15:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Script to reproduce the Conda Windows jobs locally #1279

Script to reproduce the Conda Windows jobs locally #1279

Uh oh!

j-rivero commented Mar 12, 2025 •

edited

Loading

Uh oh!

Crola1702 commented Apr 10, 2025

Uh oh!

j-rivero commented Apr 14, 2025

Uh oh!

Crola1702 commented Apr 16, 2025

Uh oh!

j-rivero commented Apr 16, 2025

Uh oh!

Crola1702 commented Apr 17, 2025

Uh oh!

Blast545 commented Aug 19, 2025

Uh oh!

j-rivero commented Oct 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Script to reproduce the Conda Windows jobs locally #1279

Script to reproduce the Conda Windows jobs locally #1279

Uh oh!

Conversation

j-rivero commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Crola1702 commented Apr 10, 2025

Uh oh!

j-rivero commented Apr 14, 2025

Uh oh!

Crola1702 commented Apr 16, 2025

Uh oh!

j-rivero commented Apr 16, 2025

Uh oh!

Crola1702 commented Apr 17, 2025

Uh oh!

Blast545 commented Aug 19, 2025

Uh oh!

j-rivero commented Oct 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

j-rivero commented Mar 12, 2025 •

edited

Loading