Random parallel build failures

[apteryks]

Hi,

The build of nx-libs appears to suffer from non-deterministic build failures when parallelized (make -jN). Here's an example:

making all in programs/Xserver...
make[6]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs'
make[6]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/include...
making all in programs/Xserver/dix...
making all in programs/Xserver/os...
making all in programs/Xserver/mi...
making all in programs/Xserver/Xext...
making all in programs/Xserver/Xi...
making all in programs/Xserver/xkb...
making all in programs/Xserver/GL...
making all in programs/Xserver/dbe...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/record...
make[7]: Nothing to be done for 'all'.
make[7]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver/include
'
making all in programs/Xserver/randr...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/render...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/xfixes...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/damageext...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
make[7]: *** No rule to make target 'all'.  Stop.
make[7]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver/os'
making all in programs/Xserver/miext/cw...
make[6]: *** [Makefile:682: os] Error 2
make[6]: *** Waiting for unfinished jobs....

It'd be nice if it worked normally. I also not that parallel build is disabled in the Debian rules file (--no-parallel).

[uli42]

You ouput is unclear to me. What subdir exactly fails?

Is it damageext?

[apteryks]

It's not clear to me either, but that's what I see! It does look like the error is triggered from the damageext directory.

[uli42]

Just stumbled over this in nx-X11/programs/Xserver/Imakefile:

#if HasParallelMake
MakeMutex($(NXAGENTDIRS) $(NXCOMPEXT) $(NXAGENTOBJS) $(NXAGENTLIBS) $(NXCOMPEXTSYSLIBS) $(NXAGENTSYSLIBS))
#endif

So you probably have to run the topdir makefile with IMAKE_DEFINES=-DHasParallelMake=1

[apteryks]

Hm, I tried that, but it didn't seem to make any difference:

make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/include/GL'
make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/include'
including in ./config...
make[4]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config'
including in config/cf...
make[5]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config/cf'
make[5]: Nothing to be done for 'includes'.
make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config/cf'
make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config'
including in ./programs...
make[4]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs'
including in programs/Xserver...
make[5]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
Makefile:539: *** missing separator.  Stop.
make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[4]: *** [Makefile:615: includes] Error 2
make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs'
make[3]: *** [xmakefile:674: includes] Error 2
make[3]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11'
make[2]: *** [Makefile:102: includes] Error 2
make[2]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11'
make[1]: *** [Makefile:148: build-env] Error 2
make[1]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source'
make: *** [Makefile:220: build] Error 2
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-j" "180" "PREFIX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "ETCDIR_NX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/etc" "LOCAL_LDFLAGS=-Wl,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib/nx/X11" "IMAKE_DEFINES=-DUseTIRPC=1 -DHasParallelMake=1 -DBourneShell=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh -DProjectRoot=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27 -DDefaultUserPath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/bin" "CONFIG_SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "CONFIGURE=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh ./configure --prefix=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "VERBOSE=1") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `build' failed after 5.9 seconds
command "make" "-j" "180" "PREFIX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "ETCDIR_NX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/etc" "LOCAL_LDFLAGS=-Wl,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib/nx/X11" "IMAKE_DEFINES=-DUseTIRPC=1 -DHasParallelMake=1 -DBourneShell=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh -DProjectRoot=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27 -DDefaultUserPath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/bin" "CONFIG_SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "CONFIGURE=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh ./configure --prefix=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "VERBOSE=1" failed with status 2
builder for `/gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv' failed with exit code 1
build of /gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv failed
View build log at '/var/log/guix/drvs/zi/k74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv.gz'.
guix build: error: build of `/gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv' failed

real    0m42.514s
user    0m6.178s
sys     0m0.594s

[uli42]

I see "./configure" in the output. That is used for _some_ components. Of course that is not synced with the imake build. So I think you have to modify the main makefile to introduce some "barriers" First compile the automake parts and then the imake parts (xserver). Be aware that nxcomp and nxcompshad are required by the other parts. And the server depends on both and libNX_x11. If you are wiling to help there's an automake branch that needs some polishing. It should be findable in the pull-requests. Let me know if you don't find it. Xorg uses meson nowadays so the optimum was to restructure everything to use just that. Uli Apteryks ***@***.***> schrieb am So., 19. Mai 2024, 03:11:

…

Hm, I tried that, but it didn't seem to make any difference: make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/include/GL' make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/include' including in ./config... make[4]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config' including in config/cf... make[5]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config/cf' make[5]: Nothing to be done for 'includes'. make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config/cf' make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config' including in ./programs... make[4]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs' including in programs/Xserver... make[5]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver' Makefile:539: *** missing separator. Stop. make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver' make[4]: *** [Makefile:615: includes] Error 2 make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs' make[3]: *** [xmakefile:674: includes] Error 2 make[3]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11' make[2]: *** [Makefile:102: includes] Error 2 make[2]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11' make[1]: *** [Makefile:148: build-env] Error 2 make[1]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source' make: *** [Makefile:220: build] Error 2 error: in phase 'build': uncaught exception: %exception #<&invoke-error program: "make" arguments: ("-j" "180" "PREFIX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "ETCDIR_NX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/etc" "LOCAL_LDFLAGS=-Wl,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib/nx/X11" "IMAKE_DEFINES=-DUseTIRPC=1 -DHasParallelMake=1 -DBourneShell=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh -DProjectRoot=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27 -DDefaultUserPath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/bin" "CONFIG_SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "CONFIGURE=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh ./configure --prefix=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "VERBOSE=1") exit-status: 2 term-signal: #f stop-signal: #f> phase `build' failed after 5.9 seconds command "make" "-j" "180" "PREFIX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "ETCDIR_NX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/etc" "LOCAL_LDFLAGS=-Wl,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib/nx/X11" "IMAKE_DEFINES=-DUseTIRPC=1 -DHasParallelMake=1 -DBourneShell=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh -DProjectRoot=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27 -DDefaultUserPath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/bin" "CONFIG_SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "CONFIGURE=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh ./configure --prefix=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "VERBOSE=1" failed with status 2 builder for `/gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv' failed with exit code 1 build of /gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv failed View build log at '/var/log/guix/drvs/zi/k74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv.gz'. guix build: error: build of `/gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv' failed real 0m42.514s user 0m6.178s sys 0m0.594s — Reply to this email directly, view it on GitHub <#1072 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQHBZBVX4SJEFQJGEEHA4DZC734PAVCNFSM6AAAAABELVQQ2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZGA2DSOJZGI> . You are receiving this because you commented.Message ID: ***@***.***>

[apteryks]

If you are wiling to help there's an automake branch that needs some polishing. It should be findable in the pull-requests. Let me know if you don't find it.

That's tempting but am I bit swamped with things to do at the moment. I'll try to remember revisiting this in the future!

[apteryks]

Xorg uses meson nowadays so the optimum was to restructure everything to use just that.

"Was" makes it sound like the migration of the build system to Meson already happened? Is that so?

[uli42]

No, "was" is subjunctive, like "If I were rich" ;-) Uli

…

On Wed, May 22, 2024 at 1:58 AM Apteryks ***@***.***> wrote: Xorg uses meson nowadays so the optimum was to restructure everything to use just that. "Was" makes it sound like the migration of the build system to Meson already happened? Is that so? — Reply to this email directly, view it on GitHub <#1072 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQHBZCFZI4T3U4SOUZODDTZDPNTTAVCNFSM6AAAAABELVQQ2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRTGYYTCOBSHA> . You are receiving this because you commented.Message ID: ***@***.***>

[apteryks]

Ah, I see :-) I guess I'll look at that Autotools branch at some point; it'd be a nice improvement over a build system whose default assumption is that make doesn't support parallel builds ^^'.

[uli42]

;-) Here are the respective PRs: #947 #946

…

On Wed, May 22, 2024 at 2:33 PM Apteryks ***@***.***> wrote: Ah, I see :-) I guess I'll look at that Autotools branch at some point; it'd be a nice improvement over a build system whose default assumption is that make doesn't support parallel builds ^^'. — Reply to this email directly, view it on GitHub <#1072 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQHBZA65DCW5IQUFD7P6QLZDSGAJAVCNFSM6AAAAABELVQQ2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRUGY3TSNRRGM> . You are receiving this because you commented.Message ID: ***@***.***>