Introduce infrastructure for running Lua API tests and add bitop tests #121

ligurio · 2025-03-19T12:21:25Z

It is needed to discuss the following questions on review:

how to integrate Lua API tests to tarantool? (see PR cmake: integrate Lua API tests tarantool/tarantool#11301)
(postponed) how to generate random compiler options (-O1, -O2, -O3 etc.) ¹ and apply them?
is it possible to unroll CMake function build_luzer()?
Building is failed on OSS Fuzz, see Introduce infrastructure for running Lua API tests and add bitop tests #121 (comment)

Blocked by:

https://luajit.org/running.html ↩

ligurio · 2025-03-19T12:51:34Z

 -- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:30 (find_package):
  Could not find a package configuration file provided by "LLVM" with any of
  the following names:

    LLVMConfig.cmake
    llvm-config.cmake

  Add the installation prefix of "LLVM" to CMAKE_PREFIX_PATH or set
  "LLVM_DIR" to a directory containing one of the above files.  If "LLVM"
  provides a separate development package or SDK, be sure it has been
  installed.

https://github.com/ligurio/lua-c-api-tests/actions/runs/13946406205/job/39034554410?pr=121

Build Tarantool: ``` CFLAGS="-fsanitize=fuzzer-no-link" LDFLAGS="-fsanitize=fuzzer-no-link" CC=clang CXX=clang++ cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DENABLE_FUZZER=ON -DLUAJIT_NUMMODE=2 ``` Build luzer: ``` git clone https://github.com/ligurio/luzer CC=clang-21 CXX=clang++-21 cmake -S . -B build cmake --build build/ --parallel ``` Run test: ``` LUA_CPATH='./luzer/build/luzer/?.so;;' LUA_PATH='./luzer/?/init.lua;;' ./src/tarantool ./test/fuzz/lua-tests/src/tests/lapi/bitop_bxor_test.lua LSAN_OPTIONS=suppressions=$(pwd)/asan/lsan.supp ASAN_OPTIONS=heap_profile=0:unmap_shadow_on_exit=1:print_suppressions=0 tarantool tarantool_csv.lua ``` Follows up commit 8ad443d ("cmake: introduce a module to build fuzzing tests"). TODO: - integrate with LSAN_OPTIONS and ASAN_OPTIONS 1. ligurio/lua-c-api-tests#121 NO_CHANGELOG=internal NO_DOC=internal NO_TEST=internal

The commit 8ad443d ("cmake: introduce a module to build fuzzing tests") has added tests for LuaJIT. These tests mostly written in C and tests LuaJIT via Lua C API. The patch integrates tests that written in Lua and tests LuaJIT via Lua API, see [1]. Depends on: 1. ligurio/lua-c-api-tests#121 2. tarantool#11056 NO_CHANGELOG=testing NO_DOC=testing

cmake/BuildLuzer.cmake

cmake/MakeLuaPath.cmake

Buristan

Hi, Sergey!
Thanks for the patch!

I believe that the most part of problems are from the bit library API misuse. It expects 32-bit integers for inputs, see details here.

Also we may provide more checks like the following.

tests/lapi/CMakeLists.txt

tests/lapi/bitop_arshift_test.lua

tests/lapi/bitop_band_test.lua

tests/lapi/bitop_arshift_test.lua

tests/lapi/lib.lua

tests/lapi/bitop_arshift_test.lua

tests/lapi/lib.lua

The commit 8ad443d ("cmake: introduce a module to build fuzzing tests") has added tests for LuaJIT. These tests mostly written in C and tests LuaJIT via Lua C API. The patch integrates tests that written in Lua and tests LuaJIT via Lua API, see [1]. Depends on: 1. ligurio/lua-c-api-tests#121 2. tarantool#11056 NO_CHANGELOG=testing NO_DOC=testing

Buristan

Thanks for the fixes!
LGTM.

The patch add a CMake module that builds a luzer [1], a coverage-guided, native Lua fuzzing engine. Needed for the following commit. 1. https://github.com/ligurio/luzer

While extending tests it is often required to append additional path where Lua or Lua C auxiliary modules are located to LUA_PATH or LUA_CPATH environment variables. Due to insane semicolon interpolation in CMake strings (that converts such string to a list as a result), we need to escape semicolon in LUA_PATH/LUA_CPATH strings while building the resulting value. The patch introduce MakeLuaPath.cmake module to make LUA_PATH and LUA_CPATH definition convenient with <lapi_tests_make_lua_path> helper. This function takes all paths given as a variable list argument, joins them in a reverse order by a semicolon and yields the resulting string to a specified CMake variable. Needed for the following commit.

The patch adds initial infrastructure for Lua API tests and changes existed infrastructure for running these tests and adds a fuzzing tests for bitwise operations in PUC Rio (since 5.2) [1] Lua and bitwise functions in LuaJIT [2][3]. Lua API tests can be enabled by CMake option ENABLE_LAPI_TESTS. The option is disabled by default, because necessary dependencies are not installed in OSS Fuzz and thus workflow is failed. PUC Rio Lua provided auto-coercion of string arguments to numbers by default, but it has been removed from the core language in 5.4. LuaJIT provides auto-coercion of string arguments to numbers by default, but it doesn not tested by proposed tests. The patch requires commit "cmake: allow to set a Lua library outside" [4] in the `luzer` project. The proposed bitop tests are capable to reproduce a LuaJIT issue with bit op coercion for shifts in DUALNUM builds [5]. Rules from boolean algebra [6] has been used as invariants for bitwise expressions. 1. https://www.lua.org/manual/5.2/manual.html#6.7 2. https://bitop.luajit.org/semantics.html 3. https://bitop.luajit.org/api.html 4. ligurio/luzer@4ce52a6 5. LuaJIT/LuaJIT@69bbf3c1 6. https://en.wikipedia.org/wiki/Bitwise_operation#Boolean_algebra

The commit "tests/lapi: add bitop tests" [1]. 1. ligurio/lua-c-api-tests@d4b91f6 Follows up ligurio/lua-c-api-tests#121 Related to tarantool/tarantool#11401 Related to tarantool/tarantool#11301

The commit 8ad443d ("cmake: introduce a module to build fuzzing tests") has added tests for LuaJIT. These tests mostly written in C and tests LuaJIT via Lua C API. The patch integrates tests that written in Lua and tests LuaJIT via Lua API, see [1]. Depends on: 1. ligurio/lua-c-api-tests#121 2. tarantool#11056 NO_CHANGELOG=testing NO_DOC=testing

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch from 6593cea to 9461590 Compare March 19, 2025 12:22

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch from 9461590 to b89866c Compare March 19, 2025 14:54

ligurio mentioned this pull request Mar 19, 2025

Tarantool generates much bigger number of traces than LuaJIT #122

Closed

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch 4 times, most recently from 88cdf48 to 2a1da10 Compare March 25, 2025 10:11

ligurio mentioned this pull request Mar 26, 2025

cmake: integrate Lua API tests tarantool/tarantool#11301

Open

This was referenced Mar 26, 2025

tests/lapi: add math tests #123

Merged

tests/lapi: add string tests #124

Merged

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch 3 times, most recently from 37d8d5e to 6a381f9 Compare March 27, 2025 12:18

Buristan reviewed Apr 7, 2025

View reviewed changes

cmake/BuildLuzer.cmake Outdated Show resolved Hide resolved

Buristan approved these changes Apr 7, 2025

View reviewed changes

cmake/MakeLuaPath.cmake Outdated Show resolved Hide resolved

cmake/MakeLuaPath.cmake Outdated Show resolved Hide resolved

Buristan suggested changes Apr 8, 2025

View reviewed changes

tests/lapi/bitop_arshift_test.lua Outdated Show resolved Hide resolved

tests/lapi/lib.lua Outdated Show resolved Hide resolved

tests/lapi/lib.lua Outdated Show resolved Hide resolved

tests/lapi/lib.lua Outdated Show resolved Hide resolved

tests/lapi/lib.lua Outdated Show resolved Hide resolved

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch 6 times, most recently from 4a0af62 to 84f2aa3 Compare April 14, 2025 18:26

ligurio requested a review from Buristan April 15, 2025 04:57

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch from 84f2aa3 to f234fcb Compare April 15, 2025 11:16

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch 6 times, most recently from 26c535a to 7fb04d3 Compare April 15, 2025 17:29

ligurio mentioned this pull request Apr 18, 2025

Integrate testing for bitop, math and string Lua libraries tarantool/tarantool#11401

Open

4 tasks

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch 2 times, most recently from 3770f24 to b7b0525 Compare April 22, 2025 14:39

cmake: print testsuite name only once

5124304

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch from b7b0525 to c3c3f05 Compare May 12, 2025 09:39

Buristan approved these changes May 13, 2025

View reviewed changes

ligurio added 4 commits May 13, 2025 17:41

cmake: initial version of luzer module

3d801e6

The patch add a CMake module that builds a luzer [1], a coverage-guided, native Lua fuzzing engine. Needed for the following commit. 1. https://github.com/ligurio/luzer

cmake: fix using USE_LUAJIT

54a92f6

ligurio force-pushed the ligurio/gh-xxxx-lapi-bitop branch from c3c3f05 to 5715005 Compare May 13, 2025 14:41

ligurio merged commit d4b91f6 into master May 14, 2025
12 checks passed

ligurio deleted the ligurio/gh-xxxx-lapi-bitop branch May 14, 2025 11:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Introduce infrastructure for running Lua API tests and add bitop tests #121

Introduce infrastructure for running Lua API tests and add bitop tests #121

Uh oh!

ligurio commented Mar 19, 2025 •

edited

Loading

Uh oh!

ligurio commented Mar 19, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Buristan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Buristan left a comment

Uh oh!

Uh oh!

Uh oh!

Introduce infrastructure for running Lua API tests and add bitop tests #121

Introduce infrastructure for running Lua API tests and add bitop tests #121

Uh oh!

Conversation

ligurio commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Footnotes

Uh oh!

ligurio commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Buristan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Buristan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ligurio commented Mar 19, 2025 •

edited

Loading

ligurio commented Mar 19, 2025 •

edited

Loading