[install] gumbo.c compilation failure

[joyboy67]

I'm trying to install Nokogiri on OpenBSD 7.0 (amd_64) I got a compilation failure even with CFLAGS=-std=c99

Thank you for your help :)

Have you read and followed the installation tutorial at http://www.nokogiri.org/tutorials/installing_nokogiri.html?

• Yes!

What is the complete output of gem install ?

current directory: /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ext/nokogiri
/usr/local/bin/ruby -I /usr/local/lib/ruby/3.1.0 -r ./siteconf20220222-48874-xtpw65.rb extconf.rb
checking for whether -I/usr/local/include is accepted as CPPFLAGS... yes
checking for whether -std=c99 is accepted as CFLAGS... yes
checking for whether -Wno-declaration-after-statement is accepted as CFLAGS... yes
checking for whether -g is accepted as CFLAGS... yes
checking for whether -Winline is accepted as CFLAGS... yes
checking for whether -Wmissing-noreturn is accepted as CFLAGS... yes
Building nokogiri using packaged libraries.
Static linking is enabled.
Cross build is disabled.
Using mini_portile version 2.8.0
checking for iconv... yes
---------- IMPORTANT NOTICE ----------
Building Nokogiri with a packaged version of libxml2-2.9.13.
Configuration options: --host\=x86_64-unknown-openbsd7.0 --enable-static --disable-shared --libdir\=/usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ports/x86_64-openbsd7.0/libxml2/2.9.13/lib --with-iconv\=yes --disable-dependency-tracking --without-python --without-readline --with-c14n --with-debug --with-threads --disable-shared --enable-static CFLAGS\=-O2\ -U_FORTIFY_SOURCE\ -g\ -fPIC
The following patches are being applied:
  - 0001-Remove-script-macro-support.patch
  - 0002-Update-entities-to-remove-handling-of-ssi.patch
  - 0003-libxml2.la-is-in-top_builddir.patch
  - 0004-use-glibc-strlen.patch
  - 0005-avoid-isnan-isinf.patch
  - 0006-update-automake-files-for-arm64.patch
  - 0008-htmlParseComment-handle-abruptly-closed-comments.patch
  - 0009-allow-wildcard-namespaces.patch
  - 0010-Revert-Different-approach-to-fix-quadratic-behavior.patch

The Nokogiri maintainers intend to provide timely security updates, but if
this is a concern for you and want to use your OS/distro system library
instead, then abort this installation process and install nokogiri as
instructed at:

  https://nokogiri.org/tutorials/installing_nokogiri.html#installing-using-standard-system-libraries

Note, however, that nokogiri cannot guarantee compatibility with every
version of libxml2 that may be provided by OS/package vendors.

[{:url=>
   "https://mirror.csclub.uwaterloo.ca/gnome/sources/libxml2/2.9/libxml2-2.9.13.tar.xz",
  :sha256=>"276130602d12fe484ecc03447ee5e759d0465558fbc9d6bd144e3745306ebf0e"}]
Extracting libxml2-2.9.13.tar.xz into tmp/x86_64-unknown-openbsd7.0/ports/libxml2/2.9.13... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0001-Remove-script-macro-support.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0002-Update-entities-to-remove-handling-of-ssi.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0003-libxml2.la-is-in-top_builddir.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0004-use-glibc-strlen.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0005-avoid-isnan-isinf.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0006-update-automake-files-for-arm64.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0008-htmlParseComment-handle-abruptly-closed-comments.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0009-allow-wildcard-namespaces.patch... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxml2/0010-Revert-Different-approach-to-fix-quadratic-behavior.patch... OK
Running 'configure' for libxml2 2.9.13... OK
Running 'compile' for libxml2 2.9.13... OK
Running 'install' for libxml2 2.9.13... OK
Activating libxml2 2.9.13 (from /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ports/x86_64-openbsd7.0/libxml2/2.9.13)...
Using mini_portile version 2.8.0
---------- IMPORTANT NOTICE ----------
Building Nokogiri with a packaged version of libxslt-1.1.35.
Configuration options: --host\=x86_64-unknown-openbsd7.0 --enable-static --disable-shared --libdir\=/usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ports/x86_64-openbsd7.0/libxslt/1.1.35/lib --disable-dependency-tracking --without-python --without-crypto --with-debug --with-libxml-prefix\=/usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ports/x86_64-openbsd7.0/libxml2/2.9.13 --disable-shared --enable-static CFLAGS\=-O2\ -U_FORTIFY_SOURCE\ -g\ -fPIC
The following patches are being applied:
  - 0001-update-automake-files-for-arm64.patch

The Nokogiri maintainers intend to provide timely security updates, but if
this is a concern for you and want to use your OS/distro system library
instead, then abort this installation process and install nokogiri as
instructed at:

  https://nokogiri.org/tutorials/installing_nokogiri.html#installing-using-standard-system-libraries

[{:url=>
   "https://mirror.csclub.uwaterloo.ca/gnome/sources/libxslt/1.1/libxslt-1.1.35.tar.xz",
  :sha256=>"8247f33e9a872c6ac859aa45018bc4c4d00b97e2feac9eebc10c93ce1f34dd79"}]
Extracting libxslt-1.1.35.tar.xz into tmp/x86_64-unknown-openbsd7.0/ports/libxslt/1.1.35... OK
Running git apply with /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/patches/libxslt/0001-update-automake-files-for-arm64.patch... OK
Running 'configure' for libxslt 1.1.35... OK
Running 'compile' for libxslt 1.1.35... OK
Running 'install' for libxslt 1.1.35... OK
Activating libxslt 1.1.35 (from /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ports/x86_64-openbsd7.0/libxslt/1.1.35)...
checking for whether -DNOKOGIRI_PACKAGED_LIBRARIES is accepted as CPPFLAGS... yes
checking for whether -DNOKOGIRI_LIBXML2_PATCHES="\"0001-Remove-script-macro-support.patch 0002-Update-entities-to-remove-handling-of-ssi.patch 0003-libxml2.la-is-in-top_builddir.patch 0004-use-glibc-strlen.patch 0005-avoid-isnan-isinf.patch 0006-update-automake-files-for-arm64.patch 0008-htmlParseComment-handle-abruptly-closed-comments.patch 0009-allow-wildcard-namespaces.patch 0010-Revert-Different-approach-to-fix-quadratic-behavior.patch\"" is accepted as CPPFLAGS... yes
checking for -llzma... yes
checking for whether -DNOKOGIRI_LIBXSLT_PATCHES="\"0001-update-automake-files-for-arm64.patch\"" is accepted as CPPFLAGS... yes
checking for xmlParseDoc() in libxml/parser.h... yes
checking for xsltParseStylesheetDoc() in libxslt/xslt.h... yes
checking for exsltFuncRegister() in libexslt/exslt.h... yes
Using mini_portile version 2.8.0
---------- IMPORTANT NOTICE ----------
Building Nokogiri with a packaged version of libgumbo-1.0.0-nokogiri.
Configuration options: --disable-shared --enable-static CFLAGS\=-fPIC
[]
Copying gumbo-parser files into tmp/x86_64-unknown-openbsd7.0/ports/libgumbo/1.0.0-nokogiri/gumbo-parser...
Running 'compile' for libgumbo 1.0.0-nokogiri... OK
Activating libgumbo 1.0.0-nokogiri (from ports/x86_64-openbsd7.0/libgumbo/1.0.0-nokogiri)...
checking for whether -I/usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ext/nokogiri/ports/x86_64-openbsd7.0/libgumbo/1.0.0-nokogiri/include is accepted as CPPFLAGS... yes
checking for gumbo_parse_with_options() in gumbo.h... yes
checking for xmlHasFeature()... yes
checking for xmlFirstElementChild()... yes
checking for xmlRelaxNGSetParserStructuredErrors()... yes
checking for xmlRelaxNGSetValidStructuredErrors()... yes
checking for xmlSchemaSetValidStructuredErrors()... yes
checking for xmlSchemaSetParserStructuredErrors()... yes
checking for vasprintf()... yes
checking for whether -DNOKOGIRI_OTHER_LIBRARY_VERSIONS="\"libgumbo:1.0.0-nokogiri\"" is accepted as CPPFLAGS... yes
creating Makefile

current directory: /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ext/nokogiri
make DESTDIR\= clean

current directory: /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ext/nokogiri
make DESTDIR\=
compiling gumbo.c
gumbo.c:100:31: warning: implicit declaration of function 'gumbo_status_to_string' is invalid in C99 [-Wimplicit-function-declaration]
  const char *status_string = gumbo_status_to_string(output->status);
                              ^
gumbo.c:100:62: error: no member named 'status' in 'struct GumboInternalOutput'
  const char *status_string = gumbo_status_to_string(output->status);
                                                     ~~~~~~  ^
gumbo.c:101:19: error: no member named 'status' in 'struct GumboInternalOutput'
  switch (output->status) {
          ~~~~~~  ^
gumbo.c:106:34: error: too few arguments to function call, expected 2, have 1
      gumbo_destroy_output(output);
      ~~~~~~~~~~~~~~~~~~~~       ^
/usr/local/include/gumbo.h:665:6: note: 'gumbo_destroy_output' declared here
void gumbo_destroy_output(const GumboOptions* options, GumboOutput* output);
     ^
gumbo.c:109:34: error: too few arguments to function call, expected 2, have 1
      gumbo_destroy_output(output);
      ~~~~~~~~~~~~~~~~~~~~       ^
/usr/local/include/gumbo.h:665:6: note: 'gumbo_destroy_output' declared here
void gumbo_destroy_output(const GumboOptions* options, GumboOutput* output);
     ^
gumbo.c:205:86: error: no member named 'name' in 'GumboElement'
        xml_child = xmlNewDocNode(doc, NULL, (const xmlChar *)gumbo_child->v.element.name, NULL);
                                                              ~~~~~~~~~~~~~~~~~~~~~~ ^
gumbo.c:271:7: error: use of undeclared identifier 'GumboError'
      GumboError *err = errors->data[i];
      ^
gumbo.c:271:19: error: use of undeclared identifier 'err'
      GumboError *err = errors->data[i];
                  ^
gumbo.c:272:38: warning: implicit declaration of function 'gumbo_error_position' is invalid in C99 [-Wimplicit-function-declaration]
      GumboSourcePosition position = gumbo_error_position(err);
                                     ^
gumbo.c:272:59: error: use of undeclared identifier 'err'; did you mean 'erf'?
      GumboSourcePosition position = gumbo_error_position(err);
                                                          ^~~
                                                          erf
/usr/include/math.h:219:8: note: 'erf' declared here
double erf(double);
       ^
gumbo.c:272:27: error: initializing 'GumboSourcePosition' with an expression of incompatible type 'int'
      GumboSourcePosition position = gumbo_error_position(err);
                          ^          ~~~~~~~~~~~~~~~~~~~~~~~~~
gumbo.c:274:21: warning: implicit declaration of function 'gumbo_caret_diagnostic_to_string' is invalid in C99 [-Wimplicit-function-declaration]
      size_t size = gumbo_caret_diagnostic_to_string(err, input_str, input_len, &msg);
                    ^
gumbo.c:274:54: error: use of undeclared identifier 'err'; did you mean 'erf'?
      size_t size = gumbo_caret_diagnostic_to_string(err, input_str, input_len, &msg);
                                                     ^~~
                                                     erf
/usr/include/math.h:219:8: note: 'erf' declared here
double erf(double);
       ^
gumbo.c:278:32: warning: implicit declaration of function 'gumbo_error_code' is invalid in C99 [-Wimplicit-function-declaration]
      const char *error_code = gumbo_error_code(err);
                               ^
gumbo.c:278:49: error: use of undeclared identifier 'err'; did you mean 'erf'?
      const char *error_code = gumbo_error_code(err);
                                                ^~~
                                                erf
/usr/include/math.h:219:8: note: 'erf' declared here
double erf(double);
       ^
gumbo.c:278:19: warning: incompatible integer to pointer conversion initializing 'const char *' with an expression of type 'int' [-Wint-conversion]
      const char *error_code = gumbo_error_code(err);
                  ^            ~~~~~~~~~~~~~~~~~~~~~
gumbo.c:307:36: error: too few arguments to function call, expected 2, have 1
  gumbo_destroy_output(args->output);
  ~~~~~~~~~~~~~~~~~~~~             ^
/usr/local/include/gumbo.h:665:6: note: 'gumbo_destroy_output' declared here
void gumbo_destroy_output(const GumboOptions* options, GumboOutput* output);
     ^
gumbo.c:327:11: error: no member named 'max_attributes' in 'struct GumboInternalOptions'
  options.max_attributes = NUM2INT(max_attributes);
  ~~~~~~~ ^
gumbo.c:329:11: error: no member named 'max_tree_depth' in 'struct GumboInternalOptions'
  options.max_tree_depth = NUM2INT(max_depth);
  ~~~~~~~ ^
gumbo.c:521:19: warning: implicit declaration of function 'gumbo_compute_quirks_mode' is invalid in C99 [-Wimplicit-function-declaration]
    quirks_mode = gumbo_compute_quirks_mode(
                  ^
gumbo.c:531:11: error: no member named 'max_attributes' in 'struct GumboInternalOptions'
  options.max_attributes = NUM2INT(max_attributes);
  ~~~~~~~ ^
gumbo.c:534:11: error: no member named 'max_tree_depth' in 'struct GumboInternalOptions'
  options.max_tree_depth = depth < 0 ? -1 : (depth + 1);
  ~~~~~~~ ^
gumbo.c:535:28: warning: incompatible pointer to integer conversion assigning to 'GumboTag' from 'const char *' [-Wint-conversion]
  options.fragment_context = ctx_tag;
                           ^ ~~~~~~~
gumbo.c:537:11: error: no member named 'fragment_encoding' in 'struct GumboInternalOptions'
  options.fragment_encoding = encoding;
  ~~~~~~~ ^
gumbo.c:538:11: error: no member named 'quirks_mode' in 'struct GumboInternalOptions'
  options.quirks_mode = quirks_mode;
  ~~~~~~~ ^
gumbo.c:539:11: error: no member named 'fragment_context_has_form_ancestor' in 'struct GumboInternalOptions'
  options.fragment_context_has_form_ancestor = form;
  ~~~~~~~ ^
gumbo.c:504:30: warning: implicit conversion loses integer precision: 'VALUE' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
                             rb_utf8_str_new_static("encoding", 8));
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
fatal error: too many errors emitted, stopping now [-ferror-limit=]
8 warnings and 20 errors generated.
*** Error 1 in /usr/local/lib/ruby/gems/3.1.0/gems/nokogiri-1.13.3/ext/nokogiri (Makefile:473 'gumbo.o': @ clang -I. -I/usr/local/include/ru...)

make failed, exit code 2

Tell us about your system!

What is the output from ruby -v?

ruby 3.1.p18 [x68_64-openbsd7.0]

What is the output from gem -v?

3.3.7

What is the output from gem env?

RubyGems Environment:
  - RUBYGEMS VERSION: 3.3.7
  - RUBY VERSION: 3.1.1 (2022-02-18 patchlevel 18) [x86_64-openbsd7.0]
  - INSTALLATION DIRECTORY: /usr/local/lib/ruby/gems/3.1.0
  - USER INSTALLATION DIRECTORY: /home/cloud/.local/share/gem/ruby/3.1.0
  - RUBY EXECUTABLE: /usr/local/bin/ruby
  - GIT EXECUTABLE: /usr/local/bin/git
  - EXECUTABLE DIRECTORY: /usr/local/bin
  - SPEC CACHE DIRECTORY: /home/cloud/.local/share/gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /usr/local/etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-openbsd-7.0
  - GEM PATHS:
     - /usr/local/lib/ruby/gems/3.1.0
     - /home/cloud/.local/share/gem/ruby/3.1.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /home/cloud/bin
     - /home/cloud/.local/bin
     - /bin
     - /sbin
     - /usr/bin
     - /usr/sbin
     - /usr/X11R6/bin
     - /usr/local/bin
     - /usr/local/sbin
     - /usr/games
     - /opt/bin
     - .
     - /usr/local/jdk-1.8.0/jre/bin/
     - /usr/local/share/isotop/bin
     - /home/cloud/.cabal/bin
     - /home/cloud/.local/bin
     - /root/.local/share/gem/ruby/3.1.0/bin
     - /home/cloud/.gem/ruby/3.1/bin

If Linux or a BSD variant, please note:

• the output of uname -a

OpenBSD 7.0 GENERIC.PM#5 amd64

• the contents of /etc/lsb-release.

No such file or directory

[flavorjones]

@joyboy67 Thanks for opening this issue, and I'm sorry you're having trouble. I'll try to help.

Do you know if this was a problem with Nokogiri v1.13.1? I'm going to try to reproduce what you're seeing.

[flavorjones]

For whatever it's worth, we run tests on FreeBSD and they're green: https://github.com/sparklemotion/nokogiri/runs/5283373352?check_suite_focus=true

[stevecheckoway]

I think the problem is that the #include <gumbo.h> is picking up /usr/local/include/gumbo.h.

/usr/local/include/gumbo.h:665:6: note: 'gumbo_destroy_output' declared here
void gumbo_destroy_output(const GumboOptions* options, GumboOutput* output);
     ^
gumbo.c:109:34: error: too few arguments to function call, expected 2, have 1
      gumbo_destroy_output(output);
      ~~~~~~~~~~~~~~~~~~~~       ^

We could rename the gumbo headers to make sure they don't conflict with any system headers.

[flavorjones]

Oh! Thanks for spotting that, @stevecheckoway.

I wonder if we could also work around this by specifying -I. in the gumbo-parser Makefile? It seems odd that this behavior (the order of flags on the gcc command line?) is different on OpenBSD.

@joyboy67 You could help us by providing a script to set up a fresh openbsd7 box with the same set of packages that you're using. I'm unable to reproduce this from within a VM based on https://app.vagrantup.com/generic/boxes/openbsd7 primarily because I'm not familar with the package management tools.

[flavorjones]

@joyboy67 OK, I've reproduced this problem after installing the following packages:

pkg_add ruby30 libxslt p5-HTML-Gumbo

What's interesting is that even if I use gcc (via pkg_add gcc-11.2.0p0 and CC=egcc) I see the same behavior on OpenBSD.

However, if I install libgumbo on my linux machine system directories the correct (local) header file is used. Does anyone have any insight into why this might be (the system directory is preferred to the local directory for header files)?

[joyboy67]

@joyboy67 Thanks for opening this issue, and I'm sorry you're having trouble. I'll try to help.

Do you know if this was a problem with Nokogiri v1.13.1? I'm going to try to reproduce what you're seeing.

I runned doas gem install nokogiri

From the output error, it's seems to be the Nokogiri 1.13.3

[stevecheckoway]

#2465 might fix it, but unfortunately, I don't have time to test on OpenBSD.

[flavorjones]

@stevecheckoway I have a VM handy, I'll kick the tires.

[flavorjones]

Will be fixed in Nokogiri v1.14.0