Unable to complete Make process on Ubuntu 22.04 due to Resynthesis Plugin

[UsuallyReadsTheManual]

OS - Ubuntu 22.04

Downloaded the tar source file for hal 4.4.1.

Expected result of make -$(nproc) should yield completed build.

Actual results: make: *** [Makefile:156: all] Error 2
Due to:

Command ran for cmake: cmake .. -DPL_RESYNTHESIS=OFF
Have also tried -CMAKE_DPL_RESYNTHESIS=OFF and variants of =0
cmake seems to be ignoring the directive to ignore resynthesis plugin during make/cmake process.

Actual error from build process:

/hal-4.4.1/plugins/netlist_preprocessing/src/netlist_preprocessing.cpp:16:10: fatal error: resynthesis/resynthesis.h: No such file or directory
   16 | #include "resynthesis/resynthesis.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~

Here is the output of the cmake process:

-- Selected build type: Release
-- ccache found!
-- System is:
-- Linux distro is: Ubuntu
-- Linux Distro version name is: jammy
-- System Arch is: x86_64
-- Extracting Version Information in PATH: /opt/hal-4.4.1
-- HAL_VERSION_RETURN:
-- HAL_VERSION: v4.4.1
-- Checking for module 'bitwuzla'
--   Package 'bitwuzla', required by 'virtual:world', not found
-- Bitwuzla not found, but this is optional...
-- openMP found
-- RapidJSON found. Headers: /usr/include
-- Found rapidjson /usr/include
-- Set rapidjson path successully: /usr/include
-- Found pybind11: /usr/include (found version "2.9.1")
-- using spdlog from deps
-- using subprocess from deps
-- using nlohmann_json from deps
-- Python3_INCLUDE_DIRS: /usr/include/python3.10
-- Python3_LIBRARIES: /usr/lib/x86_64-linux-gnu/libpython3.10.so
-- PYTHON_MODULE_PREFIX:
-- PYTHON_MODULE_EXTENSION: .cpython-310-x86_64-linux-gnu.so
-- Checking for module 'libcgraph'
--   Package 'libcgraph', required by 'virtual:world', not found
-- Checking for module 'libcdt'
--   Package 'libcdt', required by 'virtual:world', not found
-- Checking for module 'libgvc'
--   Package 'libgvc', required by 'virtual:world', not found
-- Checking for module 'libpathplan'
--   Package 'libpathplan', required by 'virtual:world', not found
-- Could NOT find Graphviz (missing: GRAPHVIZ_LIBRARIES GRAPHVIZ_INCLUDE_DIR)
-- ABC not found
-- Will build abc ourselves, check README.md to see how to speed up the process...
-- Found z3
--     Z3_LIBRARIES: /usr/lib/x86_64-linux-gnu/libz3.so
--     Z3_INCLUDE_DIRS: /usr/include
-- Using igraph from /opt/hal-4.4.1/deps/igraph-0.10.x
-- Checking supported compiler flags...
-- Performing Test supports_-Wall
-- Performing Test supports_-Wall - Success
-- Performing Test supports_-Wextra
-- Performing Test supports_-Wextra - Success
-- Performing Test supports_-pedantic
-- Performing Test supports_-pedantic - Success
-- Performing Test supports_-Wshadow
-- Performing Test supports_-Wshadow - Success
-- Performing Test supports_-g
-- Performing Test supports_-g - Success
-- Performing Test supports_-Wno-undef
-- Performing Test supports_-Wno-undef - Success
-- Performing Test supports_-Werror=return-type
-- Performing Test supports_-Werror=return-type - Success
-- Performing Test supports_-fcolor-diagnostics
-- Performing Test supports_-fcolor-diagnostics - Failed
-- Performing Test supports_-fdiagnostics-color=always
-- Performing Test supports_-fdiagnostics-color=always - Success
-- Performing Test supports_-fsized-deallocation
-- Performing Test supports_-fsized-deallocation - Success
-- Performing Test supports_-O0
-- Performing Test supports_-O0 - Success
-- Performing Test supports_-g
-- Performing Test supports_-g - Success
-- Performing Test supports_-O3
-- Performing Test supports_-O3 - Success
-- Performing Test supports_-DNDEBUG
-- Performing Test supports_-DNDEBUG - Success
-- Performing Test supports_-Os
-- Performing Test supports_-Os - Success
-- Performing Test supports_-DNDEBUG
-- Performing Test supports_-DNDEBUG - Success
-- Performing Test supports_-O2
-- Performing Test supports_-O2 - Success
-- Performing Test supports_-g
-- Performing Test supports_-g - Success
-- Performing Test supports_-O1
-- Performing Test supports_-O1 - Success
-- Performing Test supports_-g
-- Performing Test supports_-g - Success
-- Performing Test supports_-fno-inline-functions
-- Performing Test supports_-fno-inline-functions - Success
-- Performing Test supports_-fno-inline-functions-called-once
-- Performing Test supports_-fno-inline-functions-called-once - Success
-- Performing Test supports_-fno-optimize-sibling-calls
-- Performing Test supports_-fno-optimize-sibling-calls - Success
-- Performing Test supports_-fno-omit-frame-pointer
-- Performing Test supports_-fno-omit-frame-pointer - Success
-- HAL_STUDY mode turned OFF!
-- GATE_LIBS: /opt/hal-4.4.1/plugins/gate_libraries/definitions/.gitignore;/opt/hal-4.4.1/plugins/gate_libraries/definitions/NanGate_15nm_OCL.hgl;/opt/hal-4.4.1/plugins/gate_libraries/defi>
-- HELPER_GATE_LIBS: /opt/hal-4.4.1/plugins/gate_libraries/definitions/helper_libs/aoixm_hal_i4.hgl;/opt/hal-4.4.1/plugins/gate_libraries/definitions/helper_libs/im_hal_i4.hgl
-- Using Qt version 5
-- Using quazip from /opt/hal-4.4.1/deps/quazip-1.3
-- CPACK_GENERATOR:
-- PROJECT_ARCH: x86_64
-- Building for focal
-- version-abc: 4.4.1
-- version: 4.4.1
-- Debian version: 4.4.1-ppa1~focal1
-- CPACK_SOURCE_IGNORE_FILES: /mxe/;/\\.vscode/;/\\.codelite/;/\\.idea/;/cmake-.*/;\\.project$;\\.workspace$;\\.iml$;\\.mk$;\\.astylerc$;\\.editorconfig$;/Makefile$
-- Upload PPA is OFF
-- SIMULATE_DPUT_STR is -s
-- Configuring done
-- Generating done
-- Build files have been written to: /opt/hal-4.4.1/build

is cmake blatantly ignoring option inputs??
Any assistance is greatly appreciated, thank you!

[UsuallyReadsTheManual]

Edit: tried with cmake ..-DBUILD_ALL_PLUGINS=ON
This worked.

Issue seems akin to issue 588 ?

[UsuallyReadsTheManual]

Unfortunately some additional issues now that the initial build process worked. I'd like to build the .deb package if possible but I cannot:

dpkg-checkbuilddeps: error: Unmet build dependencies: libigraph0-dev (>= 0.7.1) qt5-default (>= 5.7)

make[2]: *** [packaging/CMakeFiles/debuild_focal.dir/build.make:78: Debian/focal/hal-reverse_4.4.1-ppa1~focal1_source.changes] Error 29
make[1]: *** [CMakeFiles/Makefile2:2334: packaging/CMakeFiles/debuild_focal.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

qt5-default was dropped prior to ubuntu 22.04 . base, and dev tools for qt5 are installed but this doesn't seem to matter to the build process. Also for libigraph, shouldn't that get built as part of the make process in the first place? The source gets pulled in? The version in the ubuntu repo is lower than the version expected, and even making and then doing a make install on igraph doesn't seem to satisfy it either?

[joern274]

Sorry, must admit that the .deb package has not been maintained for quite some time. It is on the TODO list.

I am glad you found a workaround for the Resysnthesis problem originally reported. I will look into the plugin dependency list. Obviously some dependencies are not reflected by the build process.

[UsuallyReadsTheManual]

Also I should note, even with having manually downloaded the expected tar for the igraph source and placing it in the deps folder, I had to chain together a cmake, followed by make, copying over the cmakelist and extracted contents from the tar to the build/igraph/download and hal/deps/igraph.../ Folder, and then a make again, before that would complete a build properly.

I also had to feed a url and hash to one of the files (not in front of me sorry) that called for downloading the package if not present as the build output mentioned finding it but not having a matching hash.

For offline building it would be great to have a way to allow direct turn off/on of downloading and such. I tried using the use vendor package option but it had no effect.

[joern274]

The reported Resysnthesis <-> Netlist preprocessing dependecy problem in version 4.4.1 (tarball) has been fixed (actually by me, I just forgot) on October 15th last year. We should issue a new release soon so that people who prefer installing from tarball rather than by git clone can also benefit from that fix.

I am not sure I understand the IGRAPH problems you experienced. Once I unpacked the tarball on an Out-of-the-box-Ubuntu system the IGRAPH library was downloaded, built and installed automatically during the HAL make. You cannot use the IGRAPH package from Ubuntu since Ubuntu ships a rather old version and there was a major API break in IGRAPH. Or did I misunderstand and you were downloading a specific IGRAPH version manually on purpose? That could of course interfere with the build script.

The third topic - not building the .deb package - was already mentioned in other issues and on our TODO list. I would therefore like to close this issue unless I get more detailed information how to reproduce the IGRAPH problem.

[UsuallyReadsTheManual]

For the build, in this case, I have to do this on an offline system. Though I have full REPO access (copied locally) for some dependency installs.

Per reading the make process and wiki, etc. I snagged the igraph 10.x tar release from the igraph project and placed it into the folder referenced during the build process (which was the hal/deps/igraph-10.x folder I believe). However that still only sometimes manages to find it but then complains about not having a hash, which I then added in manually. Still I have had to chain my make/build process to make, build, copy back over a cmake target list for igraph to it's folder, untar the contents into the build/igraph/download directory, and re-run make again, before it will actually build.

With everything built and installed I am still running into various issues I am trying to suss out, for instance, when now running hal -i and feeding in a test netlist file, I get errors such as:

...hal_plugins/boolean_influence.so error:z3_utils.s could not load object... did you properly link this (I will try and print out the actual full errors) but it does this for several of the plugins. The referenced so files are literally in the same directory as the rest of the hal_plugins, so I am not sure why the issue exists. The errors also list the directory from which I built the hal package for (e.g. /opt/hal/) starting out with (for example) /opt/hal/src... and referencing various .h or .cpp files from the source. With the installation, shouldn't everything have linked to the install directory (e.g. /usr/local/hal) and not be referencing the source build directory the make was conducted in?

Edit: Went back to grab one of the errors:
[core] [error] [/opt/hal/src/plugin_system | runtime_library.cpp:47] CAnnot load library '/usr/local/lib/hal_plugins/module_identification.so' (Error boolean_influence.so:cannot open shared object file: No such file or directory) - Did you properly link the plugin?

Seems to be reminiscent of this issue #592
Running ldconfig as either root/sudo or as a general user account does not resolve the linkage issue either, either immediately post installation or otherwise.

[joern274]

You are right, the issue #592 (make install not working correctly) was never fixed. Most people were starting HAL from the build directory skipping the installation, thus other issues were more urgent. But I am working on the install issue right now.

My preliminary findings are that there is a cmake statement missing setting the RPATH/RUNPATH when installing. You can verify it when you run ldd /usr/local/lib/hal_plugins/module_identification.so or readelf -d /usr/local/lib/hal_plugins/module_identification.so.

Not sure why ldconfig doesn't resolve the issue. I was setting LD_LIBRARY_PATH as workaround and with that all dependencies were resolved. Does that work in your place ?

[joern274]

RPATH/RUNPATH should get fixed on make install. You might want to test if you pull the latest version. Will produce a new tarball soon, but there are some PR's which needs to be reviewed first.

[UsuallyReadsTheManual]

Running ldd against the path and module does return expected info:
linux-vdso.so.1 (0x00007fffae7e2000) libpython3.10.so.1.0 => /lib/x86_64-linux-gnu/libpython3.10.so.1.0 (0x00007fb5e1b34000) boolean_influence.so => /usr/local/lib/hal_plugins/boolean_influence.so (0x00007fb5e1a93000) libhal_plugin_system.so => /usr/local/lib/libhal_plugin_system.so (0x00007fb5e1a25000) libhal_netlist.so => /usr/local/lib/libhal_netlist.so (0x00007fb5e172e000) libhal_utilities.so => /usr/local/lib/libhal_utilities.so (0x00007fb5e167e000) libz3.so.4 => /lib/x86_64-linux-gnu/libz3.so.4 (0x00007fb5e0747000) libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fb5e051b000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb5e04fb000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb5e02d2000) /lib64/ld-linux-x86-64.so.2 (0x00007fb5e2322000) libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fb5e02a1000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fb5e0285000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb5e019c000) z3_utils.so => /usr/local/lib/hal_plugins/z3_utils.so (0x00007fb5e009a000) libabc-pic.so => /usr/local/lib/libabc-pic.so (0x00007fb5ded03000)

I added LD_LIBRARY_PATH to my .bashrc as /usr/local/lib/hal_plugins/ , sourced it, and the cmake and make processes (make and then make install).

I also had to manually build igraph, and run the initial cmake with: cmake .. -DBUILD_ALL_PLUGINS=ON
This properly linked up the missing igraph.so file(s).

Once that was done, a ldconfig let the igraph dependent plugins work, and ldd found what was missing before.

I do still get an error about creating projects (here is my test bed output for the line in question):

[core] [error] [/builds/hal/app/main.cpp:291] Cannot create project <test_netlist>
edit: fixed the project error by adding .v to the end of test_netlist file. I suppose .txt would work as well, but looks like that was a file/folder name conflict.

edit2:
So while the command line seemingly works, the gui does not:

When I go to create a new project in my home directory, I receive the following error:

Cannot create project </home/testuser/hal_project_2>, cannot load gate library </usr/local/share/hal/gate_libraries/example_library.hgl>

I've already performed the necessary chmod (775 in this case) recursively on /usr/local/share/hal/ and can otherwise access it as testuser (non root account).

The file(s) definitely exist, I can cat them, etc.

edit 3: sorry this is getting into the weeds now a bit. It seems in order to create any project using any of the libraries, after launching the gui, first you must go to Utilities -> Gate Library Manager -> Open, and then select any of the libraries... I was then able to create a new project. Seems I have to do that each time I launch the gui before creating a project.

Basically looks like the workaround from there would be to go first into Plugin Manager and basically load everything up (at least loading the parsers and writers seem to allow creation of new projects and such).

[joern274]

I am not sure I understand the GUI issues, at least I cannot reproduce them. I tested

• install HAL (after yesterdays RPATH fix) to /usr/local (thus gate libraries get installed in (/usr/local/share/hal/gate_libraries)
• copy the sample netlist examples/fsm/fsm.v to a location of choice
• run hal -g from that directory
• import netlist -> select fsm.v

Creating fsm project and loading netlist into GUI worked as expected, both with explicitly parsing with example_library.hgl and with autodetect gate library.

Since you tested that the gate library is in the right place and accessible and since there are no other error messages giving a hint what went wrong my best guess is that parsing of the gate library failed. Maybe by some strange reason the HGL-Parser Plugin did not get loaded. Can you verify whether it works better if you enter the Plugin-Manager page and load the HGL-Parser manually?

So you found a new bug. Please open a new issue and try to describe all steps which led to the bug (build, install, import netlist) as exact as possible.