Builds failing on OSX and Linux with Github Actions CI

[smk762]

Recently our CI builds at https://github.com/KomodoPlatform/atomicDEX-Desktop/actions have been failing to build libwally.

I suspect it may have something to do with a recent libtools update https://savannah.gnu.org/forum/forum.php?forum_id=10139

related mac errors in log:

2022-03-21T15:15:44.8170920Z *** Warning: Linking the shared library libwallycore.la against the
2022-03-21T15:15:44.8262000Z *** static library secp256k1/.libs/libsecp256k1.a is not portable!
2022-03-21T15:15:44.8762380Z copying selected object files to avoid basename conflicts...
2022-03-21T15:15:44.9182980Z /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: libwallycore_la-elements.o has no symbols
2022-03-21T15:15:44.9184050Z /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: libwallycore_la-blech32.o has no symbols
2022-03-21T15:15:44.9576510Z /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: .libs/libwallycore.a(libwallycore_la-elements.o) has no symbols
2022-03-21T15:15:44.9577510Z /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: .libs/libwallycore.a(libwallycore_la-blech32.o) has no symbols

2022-03-22T10:49:20.9051540Z ./libtool: line 151: -o: command not found
2022-03-22T10:49:29.0185510Z   CCLD     libsecp256k1.la
2022-03-22T10:49:29.0256440Z ./libtool: line 151: -o: command not found
2022-03-22T10:49:30.3257350Z error: /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: no output file specified (specify with -o output)
2022-03-22T10:49:30.3258410Z Usage: /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -static [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-sacLT] [-no_warning_for_no_symbols]
2022-03-22T10:49:30.3259840Z Usage: /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -dynamic [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-o output] [-install_name name] [-compatibility_version #] [-current_version #] [-seg1addr 0x#] [-segs_read_only_addr 0x#] [-segs_read_write_addr 0x#] [-seg_addr_table <filename>] [-seg_addr_table_filename <file_system_path>] [-all_load] [-noall_load]
2022-03-22T10:49:30.3281370Z make[2]: *** [libsecp256k1.la] Error 1
2022-03-22T10:49:30.3292420Z make[1]: *** [install-recursive] Error 1
2022-03-22T10:49:30.3301950Z make: *** [install-recursive] Error 1

Related Linux errors in log:

2022-03-22T15:40:02.5243502Z === configuring in src/secp256k1 (/home/runner/work/atomicDEX-Desktop/atomicDEX-Desktop/cmake-3.19.0-rc3-Linux-x86_64/libwally-core/src/secp256k1)
2022-03-22T15:40:02.5293435Z configure: WARNING: no configuration information is in src/secp256k1
2022-03-22T15:40:02.5804049Z Making install in src
2022-03-22T15:40:02.5865562Z make[1]: Entering directory '/home/runner/work/atomicDEX-Desktop/atomicDEX-Desktop/cmake-3.19.0-rc3-Linux-x86_64/libwally-core/src'
2022-03-22T15:40:02.5918363Z Making install in secp256k1
2022-03-22T15:40:02.5941948Z make[2]: *** No rule to make target 'install'.  Stop.
2022-03-22T15:40:02.5942747Z make[2]: Entering directory '/home/runner/work/atomicDEX-Desktop/atomicDEX-Desktop/cmake-3.19.0-rc3-Linux-x86_64/libwally-core/src/secp256k1'
2022-03-22T15:40:02.5945184Z make[2]: Leaving directory '/home/runner/work/atomicDEX-Desktop/atomicDEX-Desktop/cmake-3.19.0-rc3-Linux-x86_64/libwally-core/src/secp256k1'
2022-03-22T15:40:02.5950953Z Makefile:1666: recipe for target 'install-recursive' failed
2022-03-22T15:40:02.5952592Z make[1]: Leaving directory '/home/runner/work/atomicDEX-Desktop/atomicDEX-Desktop/cmake-3.19.0-rc3-Linux-x86_64/libwally-core/src'
2022-03-22T15:40:02.5953409Z make[1]: *** [install-recursive] Error 1
2022-03-22T15:40:02.5977357Z make: *** [install-recursive] Error 1
2022-03-22T15:40:02.5978115Z Makefile:434: recipe for target 'install-recursive' failed

I've attempted a few things to resolve this without success in our CI_libwally_test branch. For example:

• pinning older version of libtools
• using macos-10.15 instead of mac latest in https://github.com/KomodoPlatform/atomicDEX-Desktop/blob/CI_libwally_test/.github/workflows/atomicdex-desktop-ci.yml

Our windows builds are ok, because we used a cached prebuild.

Any hints you might have or insights regarding what might be missing / misconfigured in our workflow file or https://github.com/actions/virtual-environments would be appreciated

[smk762]

I figured it was worth trying to run your Github actions on a fork, seems similar issue - https://github.com/smk762/libwally-core/runs/5672778765?check_suite_focus=true#step:5:549

      Making all in secp256k1
      gcc -DHAVE_CONFIG_H -I/usr/local/opt/valgrind/include  -I. -I./src -O2  -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wconditional-uninitialized -fvisibility=hidden   -c src/gen_context.c -o gen_context.o
      gcc -O2  -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wconditional-uninitialized -fvisibility=hidden   -O2  gen_context.o -o gen_context
      ./gen_context
        CC       src/libsecp256k1_la-secp256k1.lo
      ./libtool: line 151: -o: command not found
        CCLD     libsecp256k1.la
      ./libtool: line 151: -o: command not found
      error: /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: no output file specified (specify with -o output)
      Usage: /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -static [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-sacLT] [-no_warning_for_no_symbols]
      Usage: /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -dynamic [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-o output] [-install_name name] [-compatibility_version #] [-current_version #] [-seg1addr 0x#] [-segs_read_only_addr 0x#] [-segs_read_write_addr 0x#] [-seg_addr_table <filename>] [-seg_addr_table_filename <file_system_path>] [-all_load] [-noall_load]
      make[3]: *** [libsecp256k1.la] Error 1
      make[2]: *** [all-recursive] Error 1
      make[1]: *** [all] Error 2
      make: *** [all-recursive] Error 1
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-yr9k171x/setup.py", line 23, in <module>
          call('make -j{}'.format(multiprocessing.cpu_count()))
        File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-yr9k171x/setup.py", line 18, in call
          subprocess.check_call(cmd.split(' '), cwd=abs_path)
        File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 311, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['make', '-j3']' returned non-zero exit status 2.
      ----------------------------------------
  WARNING: Discarding file:///Users/runner/work/libwally-core/libwally-core. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
                                                             ✕ 62.29s
Error: Command ['python', '-m', 'pip', 'wheel', PosixPath('/Users/runner/work/libwally-core/libwally-core'), '--wheel-dir=/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheellcpz52lq/built_wheel', '--no-deps'] failed with code 1. None```

[jgriffiths]

Hi @smk762

Please check ./libtool: line 151: -o: command not found

Line 151 presumably references an environment variable that is expected to be set to a command (such as CC or LD). If it was empty I would expect this to be the error you get.

If that is the case, you can try setting that variable to the correct value before invoking ./configure.

[smk762]

Hi @smk762

Please check ./libtool: line 151: -o: command not found

Line 151 presumably references an environment variable that is expected to be set to a command (such as CC or LD). If it was empty I would expect this to be the error you get.

If that is the case, you can try setting that variable to the correct value before invoking ./configure.

The ./libtool: line 151: -o: command not found is also occurring when running the Github actions workflow via https://github.com/ElementsProject/libwally-core/blob/master/.github/workflows/wheels.yml on a fork of this repo.

[cdecker]

Found a possible solution to this issue. Recently libtool was updated from 2.4.6_4 to 2.4.7 (formula history) which broke libtool support for secp256k-zkp in Intel and M1 macs.

Checking with the upstream version secp256k1 I noticed that this error does not occur and asked whether there were any build changes that were not backported into secp256k1-zkp.
As a response @jonasnick opened PR BlockstreamResearch/secp256k1-zkp#174 which backports a couple of missing fixes, apparently including the required changes to get mac support working again.

Now secp256k1-zkp compiles fine in isolation, however when compiling libwally-core we get other errors relating to libtool:

./configure
[...]
checking the archiver (/usr/bin/libtool) interface... unknown
configure: error: could not determine /usr/bin/libtool interface
configure: error: ./configure failed for src/secp256k1

Initially I was under the impression it might be libtool as used by libwally-core but it seems it is somehow instrumenting the ./configure call in src/secp256k1 which then fails. Is there anything special about the way ./configure is invoked from libwally-cores ./configure?

[real-or-random]

Now secp256k1-zkp compiles fine in isolation, however when compiling libwally-core we get other errors relating to libtool:

./configure
[...]
checking the archiver (/usr/bin/libtool) interface... unknown
configure: error: could not determine /usr/bin/libtool interface
configure: error: ./configure failed for src/secp256k1

Initially I was under the impression it might be libtool as used by libwally-core but it seems it is somehow instrumenting the ./configure call in src/secp256k1 which then fails. Is there anything special about the way ./configure is invoked from libwally-cores ./configure?

Can you share a config.log for this error?

[jurvis]

+1 on this issue. currently trying to build libwally-core too on M1 Mac with libtool@2.4.7

here's my output when running ./build-libwally.sh -dsc:

/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in secp256k1
gcc -DHAVE_CONFIG_H -I. -I./src -O2  -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wconditional-uninitialized -fvisibility=hidden -g -O2 -c src/gen_context.c -o gen_context.o
gcc -O2  -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wconditional-uninitialized -fvisibility=hidden -g -O2  gen_context.o -o gen_context
./gen_context
  CC       src/libsecp256k1_la-secp256k1.lo
./libtool: line 151: -o: command not found
  CCLD     libsecp256k1.la
./libtool: line 151: -o: command not found
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: no output file specified (specify with -o output)

[jgriffiths]

@smk762 @cdecker @jurvis I've created a branch to fix these build issues, please see #321

If you could test this and ack for your use case(s) that would be great. I will test this with gdk and merge once I have confirmation everything is working as expected. Thanks!

[jurvis]

hey @jgriffiths, thanks for looking into this! I just noticed I mis-typed earlier where my logs were from; I was trying to build libwally-swift with their build script, not libwally-core.

Anyways, I was still able to produce that output with current HEAD of master with the following commands:

./configure --disable-shared --host=aarch64-apple-darwin --enable-static --disable-elements`
make clean
make

I checked out test_build_fix, and re-ran the commands above. Now, I am getting this error output instead:

*** Warning: Linking the shared library libwallycore.la against the
*** static library secp256k1/.libs/libsecp256k1.a is not portable!
copying selected object files to avoid basename conflicts...
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive member: .libs/libwallycore.a(libsecp256k1.a) fat file for cputype (16777223) cpusubtype (3) is not an object file (bad magic number)
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar: internal ranlib command failed
make[3]: *** [libwallycore.la] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

hope this is helpful. let me know if there's more info I can help you with collecting

[jgriffiths]

@jurvis Thanks, I think this error is because its attempting to build a universal binary instead of a singular arch, and that doesn't seem to be supported for adding archive into archives.

Can you try limiting the arch to arm64 via CFLAGS="-march=arm64" or similar?

[jgriffiths]

@jurvis You can also try this commit 0dca9bf without -march which may fix fat linking for the shared lib.

[jurvis]

@jgriffiths yep, it looks like that was it. I was able to build libwally-core a-okay by just setting CFLAGS="-O3 -arch arm64 (trying to get as close as possible to command provided in libwally-swift's use-case).

However, when I try and build libwally-swift, I get this error:

*** static library secp256k1/.libs/libsecp256k1.a is not portable!
copying selected object files to avoid basename conflicts...
  CCLD     test_bech32
ld: warning: ignoring file ./.libs/libwallycore.a, building for iOS Simulator-arm64 but attempting to link with file built for iOS Simulator-arm64
Undefined symbols for architecture arm64:
  "_wally_addr_segwit_to_bytes", referenced from:
      _main in test_bech32-test_bech32.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [test_bech32] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

here's what the script is doing (abbreviated):

export CFLAGS="-O3 -arch arm64 -fembed-bitcode-marker -mios-simulator-version-min=10.0 -isysroot `xcrun -sdk iphonesimulator --show-sdk-path`"
export CXXFLAGS="-O3 -arch arm64 -fembed-bitcode-marker -mios-simulator-version-min=10.0 -isysroot `xcrun -sdk iphonesimulator --show-sdk-path`"
./configure --disable-shared --host=$arch_target --enable-static --disable-elements
make clean
make

I apologize I don't really have knowledge about building universal binaries and how they're being managed to run on iOS. It looks like everything may be okay on libwally-core's end. If you agree, I think we can consider this issue fixed for me, and I can dig a little further on my end to see how I can fix the build script for libwally-swift

[jgriffiths]

@jurvis You may also need to set LDFLAGS=-arch arm64 so the linker knows what kind of ABI it is meant to be producing. In that case you would want to make 2 builds, one for x86_64 and one for arm64, rather than a single universal binary build.

Aside from allowing M1 builds for wally, my goal is to also have the CI produce wheels that we can release via PyPI so that building from source isn't needed on that platform. That is a question of CI hacking which I am experimenting on in #322. So while its good that you can build wally there, I'll probably consider this closed once we can produce working binaries on the CI, at least for arm64 specifically if not universal2.