cmake build, configurable from env #2115

KarelVesely84 · 2025-04-10T12:05:03Z

make sure the defaults in cmake/cmake_extension.py variable extra_cmake_args can be overriden by cmake_args from ${SHERPA_ONNX_CMAKE_ARGS} env variable
fix a bug in sherpa-onnx/csrc/parse-options.cc which appears when using -DSHERPA_ONNX_ENABLE_CHECK=ON

KarelVesely84 · 2025-04-10T12:12:23Z

Next, could someone look into the setuptools and cmake code ?

I tried to build with custom config:

SHERPA_ONNX_CMAKE_ARGS=(
-DCMAKE_BUILD_TYPE=Debug
-DBUILD_SHARED_LIBS=ON
-DSHERPA_ONNX_ENABLE_GPU=OFF
-DSHERPA_ONNX_ENABLE_CHECK=ON
-DSHERPA_ONNX_ENABLE_PYTHON=ON
-DSHERPA_ONNX_ENABLE_PORTAUDIO=OFF
-DSHERPA_ONNX_ENABLE_WEBSOCKET=OFF
-DSHERPA_ONNX_ENABLE_TTS=OFF
-DSHERPA_ONNX_ENABLE_SPEAKER_DIARIZATION=OFF
)

I am building with:

SHERPA_ONNX_CMAKE_ARGS="${SHERPA_ONNX_CMAKE_ARGS[@]}" python3 setup.py bdist_wheel

so cmake build logs are visible on the screen.

The build fails as the cmake/cmake_extension.py code assumes the binaries do exist.
But here they are disabled by : -DSHERPA_ONNX_ENABLE_BINARY=OFF.

Similar problems arise also from other non-default options:

SHERPA_ONNX_ENABLE_PORTAUDIO=OFF
DSHERPA_ONNX_ENABLE_SPEAKER_DIARIZATION=OFF
...

csukuangfj · 2025-04-11T01:20:24Z

please add after

sherpa-onnx/cmake/cmake_extension.py

Lines 216 to 217 in 95ba6b4

    
           if not src_file.is_file(): 
        
               src_file = install_dir / ".." / (f + suffix)

if not src_file.is_file():
  continue

KarelVesely84 · 2025-04-11T11:34:27Z

Okay, added those two lines, now getting:

[100%] Built target _sherpa_onnx
Installing the project stripped...
-- Install configuration: "Debug"
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/lib/libonnxruntime.so
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/./sherpa-onnx.pc
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/../_sherpa_onnx.cpython-39-x86_64-linux-gnu.so
-- Set non-toolchain portion of runtime path of "/mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/../_sherpa_onnx.cpython-39-x86_64-linux-gnu.so" to 
"$ORIGIN"
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/lib/libsherpa-onnx-c-api.so
-- Set non-toolchain portion of runtime path of "/mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/lib/libsherpa-onnx-c-api.so" to "$ORIGIN"
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/lib/libsherpa-onnx-cxx-api.so
-- Set non-toolchain portion of runtime path of "/mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/lib/libsherpa-onnx-cxx-api.so" to "$ORIGIN"
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/include/sherpa-onnx/c-api/c-api.h
-- Installing: /mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/include/sherpa-onnx/c-api/cxx-api.h
**error: [Errno 2] No such file or directory: '/mnt/matylda5/iveselyk/EU-ASR_TENDER/SHERPA_ONNX/src/sherpa-onnx/build/lib.linux-x86_64-cpython-39/sherpa_onnx/bin'**

KarelVesely84 · 2025-04-11T11:46:08Z

it seems like something in sherpa-onnx/csrc/CMakeLists.txt, but all the install() commands with
bin are "guarded" by the SHERPA_ONNX_ENABLE_BINARY variable... so, on the 1st read it seems okay

but it can be also somewhere else, there is also one install() in the main CMakelists.txt
and also some in cmake/*.cmake files

csukuangfj · 2025-04-11T13:58:24Z

I see the problem.

You need to change two places.

Please change

sherpa-onnx/setup.py

Line 47 in 95ba6b4

def get_binaries_to_install():

You can add something like:

def get_binaries_to_install():
  cmake_args = os.environ.get("SHERPA_ONNX_CMAKE_ARGS", "")
  if '-DSHERPA_ONNX_ENABLE_BINARY=OFF' in cmake_args:
    return None

Change

sherpa-onnx/setup.py

Line 72 in 95ba6b4

data_files=[("bin", get_binaries_to_install())],

to

 data_files=[("bin", get_binaries_to_install())] if get_binaries_to_install() else [],

or

 data_files=[("bin", get_binaries_to_install())] if get_binaries_to_install() else None,

KarelVesely84 · 2025-04-11T14:36:48Z

Thank you for the suggestions!
Now, the build with -DSHERPA_ONNX_ENABLE_BINARY=OFF works for me locally.
(some extra checks for existing folders were also added into cmake/cmake_extension.py:227-232)

PR is updated.

csukuangfj · 2025-04-11T16:19:49Z

Is it ready to merge?

csukuangfj · 2025-04-11T16:20:40Z

sherpa-onnx/csrc/parse-options.cc

@@ -262,7 +262,7 @@ static bool MustBeQuoted(const std::string &str, ShellType st) {
 // will get passed to the program in the same way.
 static std::string QuoteAndEscape(const std::string &str, ShellType /*st*/) {
  // Only Bash is supported (for the moment).
-  SHERPA_ONNX_CHECK_EQ(st, kBash) << "Invalid shell type.";
+  // SHERPA_ONNX_CHECK_EQ(st, kBash) << "Invalid shell type.";


Can you call exit() after printing a log?

returned to it. i don't understand the context now... there seems to be no logprint...

here the compilation was crashing when -DSHERPA_ONNX_ENABLE_CHECK=ON was set.
the variable st was already commented in the header of the function QuoteAndEscape,
and it was used in the macro SHERPA_ONNX_CHECK_EQ, being an undefined variable.

so I commented out the line 265 fix it... there is no other use of variable st in that function...
should there be done something more ?

Then please just remove it instead of commenting it out. Removing it leads to less code.

ok, good, done !

KarelVesely84 · 2025-04-16T09:49:57Z

Is it ready to merge?

Left a comment above on the call exit() remark.
Otherwise, to me it seems ready for merging.

- make sure the defaults in `cmake/cmake_extension.py` variable `extra_cmake_args` can be overriden by `cmake_args` from `SHERPA_ONNX_CMAKE_ARGS` env variable - fix a bug in `sherpa-onnx/csrc/parse-options.cc` which appears when using `-DSHERPA_ONNX_ENABLE_CHECK=ON` - avoid copying binaries when these are disabled

KarelVesely84 · 2025-04-16T12:09:31Z

Ok, the commented line is removed. The PR was squashed to be a single commit.
It is ready for mering.

csukuangfj

Thank you for your contribution!

csukuangfj reviewed Apr 11, 2025

View reviewed changes

KarelVesely84 force-pushed the cmake_build_env branch from 2cd87f7 to b61e0e7 Compare April 16, 2025 12:07

csukuangfj approved these changes Apr 16, 2025

View reviewed changes

csukuangfj merged commit f3d23aa into k2-fsa:master Apr 16, 2025
180 of 222 checks passed

KarelVesely84 deleted the cmake_build_env branch May 22, 2025 12:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

cmake build, configurable from env #2115

cmake build, configurable from env #2115

Uh oh!

KarelVesely84 commented Apr 10, 2025 •

edited

Loading

Uh oh!

KarelVesely84 commented Apr 10, 2025 •

edited

Loading

Uh oh!

csukuangfj commented Apr 11, 2025

Uh oh!

KarelVesely84 commented Apr 11, 2025

Uh oh!

KarelVesely84 commented Apr 11, 2025 •

edited

Loading

Uh oh!

csukuangfj commented Apr 11, 2025

Uh oh!

KarelVesely84 commented Apr 11, 2025

Uh oh!

csukuangfj commented Apr 11, 2025

Uh oh!

csukuangfj Apr 11, 2025

Uh oh!

KarelVesely84 Apr 16, 2025

Uh oh!

csukuangfj Apr 16, 2025

Uh oh!

KarelVesely84 Apr 16, 2025

Uh oh!

KarelVesely84 commented Apr 16, 2025

Uh oh!

KarelVesely84 commented Apr 16, 2025

Uh oh!

csukuangfj left a comment

Uh oh!

Uh oh!

Uh oh!

cmake build, configurable from env #2115

cmake build, configurable from env #2115

Uh oh!

Conversation

KarelVesely84 commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KarelVesely84 commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

csukuangfj commented Apr 11, 2025

Uh oh!

KarelVesely84 commented Apr 11, 2025

Uh oh!

KarelVesely84 commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

csukuangfj commented Apr 11, 2025

Uh oh!

KarelVesely84 commented Apr 11, 2025

Uh oh!

csukuangfj commented Apr 11, 2025

Uh oh!

csukuangfj Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

KarelVesely84 Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

csukuangfj Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

KarelVesely84 Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

KarelVesely84 commented Apr 16, 2025

Uh oh!

KarelVesely84 commented Apr 16, 2025

Uh oh!

csukuangfj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

KarelVesely84 commented Apr 10, 2025 •

edited

Loading

KarelVesely84 commented Apr 10, 2025 •

edited

Loading

KarelVesely84 commented Apr 11, 2025 •

edited

Loading