FINN build flow fails with ONNXRuntimeError because of invalid graph

[LinusJungemann]

The FINN build process fails with the following error when building the dummy transformer:

Traceback (most recent call last):
  File "/dev/shm/temporary_finn_dir/finn/src/finn/builder/build_dataflow.py", line 158, in build_dataflow_cfg
    model = transform_step(model, cfg)
  File "/scratch/hpc-prf-ekiapp/linusjun/test/finn-on-n2/attention/build_steps.py", line 731, in step_apply_folding_config
    verify_step(model, cfg, "folded_hls_cppsim", need_parent=True)
  File "/dev/shm/temporary_finn_dir/finn/src/finn/builder/build_dataflow_steps.py", line 166, in verify_step
    out_dict = execute_parent(parent_model_fn, child_model_fn, in_npy, return_full_ctx=True)
  File "/dev/shm/temporary_finn_dir/finn/src/finn/util/test.py", line 171, in execute_parent
    ret = execute_onnx(parent_model, {iname: input_tensor_npy}, True)
  File "/dev/shm/temporary_finn_dir/finn/src/finn/core/onnx_exec.py", line 54, in execute_onnx
    return execute_onnx_base(model, input_dict, return_full_exec_context, start_node, end_node)
  File "/dev/shm/temporary_finn_dir/finn/deps/qonnx/src/qonnx/core/onnx_exec.py", line 179, in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
  File "/dev/shm/temporary_finn_dir/finn/deps/qonnx/src/qonnx/core/onnx_exec.py", line 87, in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 454, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. In Node, ("Squeeze_0", Squeeze, "", -1) : ("global_in": tensor(float),"Squeeze_0_param0": tensor(int64),) -> ("Squeeze_0_out0": tensor(float),) , Error Unrecognized attribute: axes for operator Squeeze
> /usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py(454)_create_inference_session()
-> sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

Workaround:

1. Use step_convert_attention_to_hw from https://github.com/iksnagreb/radioml-transformer/blob/conformer/build_steps.py#L282
2. Remove step_tidy_up_post_attention from build.py

[LinusJungemann]

Files to reproduce: code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua.zip
TODO: Workaround for

****** Vitis HLS - High-Level Synthesis from C, C++ and OpenCL v2022.1 (64-bit)
  **** SW Build 3526262 on Mon Apr 18 15:47:01 MDT 2022
  **** IP Build 3524634 on Mon Apr 18 20:55:01 MDT 2022
    ** Copyright 1986-2022 Xilinx, Inc. All Rights Reserved.

source /opt/software/FPGA/Xilinx/Vitis_HLS/2022.1/scripts/vitis_hls/hls.tcl -notrace
INFO: [HLS 200-10] Running '/opt/software/FPGA/Xilinx/Vitis_HLS/2022.1/bin/unwrapped/lnx64.o/vitis_hls'
/opt/software/FPGA/Xilinx/Vitis_HLS/2022.1/tps/tcl/tcl8.5/tzdata/Europe/Dublin can't be opened.
INFO: [HLS 200-10] For user 'linusjun' on host 'n2fpga34' (Linux_x86_64 version 4.18.0-477.51.1.el8_8.x86_64) on Fri Dec 20 10:06:33 CET 2024
INFO: [HLS 200-10] On os Ubuntu 22.04.1 LTS
INFO: [HLS 200-10] In directory '/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua'
Sourcing Tcl script '/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/hls_syn_ElementwiseAdd_hls_0.tcl'
INFO: [HLS 200-1510] Running: source /dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/hls_syn_ElementwiseAdd_hls_0.tcl
HLS project: project_ElementwiseAdd_hls_0
HW source dir: /dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua
finn-hlslib dir: /dev/shm/temporary_finn_dir/finn/deps/finn-hlslib
attention-hlslib dir: /dev/shm/temporary_finn_dir/finn/deps/attention-hlslib
custom HLS dir: /dev/shm/temporary_finn_dir/finn/custom_hls
INFO: [HLS 200-1510] Running: open_project project_ElementwiseAdd_hls_0 
INFO: [HLS 200-10] Creating and opening project '/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/project_ElementwiseAdd_hls_0'.
INFO: [HLS 200-1510] Running: add_files /dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp -cflags -std=c++14 -I/dev/shm/temporary_finn_dir/finn/deps/finn-hlslib -I/dev/shm/temporary_finn_dir/finn/custom_hls -I/dev/shm/temporary_finn_dir/finn/deps/attention-hlslib 
INFO: [HLS 200-10] Adding design file '/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp' to the project
INFO: [HLS 200-1510] Running: set_top ElementwiseAdd_hls_0 
INFO: [HLS 200-1510] Running: open_solution sol1 
INFO: [HLS 200-10] Creating and opening solution '/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/project_ElementwiseAdd_hls_0/sol1'.
INFO: [HLS 200-1505] Using default flow_target 'vivado'
Resolution: For help on HLS 200-1505 see www.xilinx.com/cgi-bin/docs/rdoc?v=2022.1;t=hls+guidance;d=200-1505.html
INFO: [HLS 200-435] Setting 'open_solution -flow_target vivado' configuration: config_interface -m_axi_latency=0
INFO: [HLS 200-1510] Running: set_part xcu55c-fsvh2892-2L-e 
INFO: [HLS 200-1611] Setting target device to 'xcu55c-fsvh2892-2L-e'
INFO: [HLS 200-1510] Running: config_compile -disable_unroll_code_size_check -pipeline_style flp 
WARNING: [XFORM 203-506] Disable code size check when do loop unroll.
WARNING: [HLS 200-643] The 'config_compile -disable_unroll_code_size_check' hidden command is deprecated and will be removed in a future release.
INFO: [HLS 200-1510] Running: config_interface -m_axi_addr64 
INFO: [HLS 200-1510] Running: config_rtl -module_auto_prefix 
INFO: [HLS 200-1510] Running: config_rtl -deadlock_detection none 
INFO: [HLS 200-1510] Running: create_clock -period 10.0 -name default 
INFO: [SYN 201-201] Setting up clock 'default' with a period of 10ns.
INFO: [HLS 200-1510] Running: csynth_design 
INFO: [HLS 200-111] Finished File checks and directory preparation: CPU user time: 0.01 seconds. CPU system time: 0 seconds. Elapsed time: 0 seconds; current allocated memory: 448.070 MB.
INFO: [HLS 200-10] Analyzing design file '/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp' ... 
INFO: [HLS 200-111] Finished Source Code Analysis and Preprocessing: CPU user time: 90.79 seconds. CPU system time: 3.77 seconds. Elapsed time: 97.39 seconds; current allocated memory: 456.859 MB.
INFO: [HLS 200-777] Using interface defaults for 'Vivado' flow target.
INFO: [HLS 214-131] Inlining function 'Slice<ap_int<2>, 2u>::Container<ap_uint<2> >::Container(ap_uint<2> const&)' into 'Slice<ap_int<2>, 2u>::Container<ap_uint<2> > Slice<ap_int<2>, 2u>::operator()<ap_uint<2> >(ap_uint<2> const&) const' (/dev/shm/temporary_finn_dir/finn/deps/finn-hlslib/interpret.hpp:247:9)
INFO: [HLS 214-131] Inlining function 'ap_uint<(1ul) * (ap_int<3>::width)> flatten<1ul, ap_int<3> >(ap_int<3> const*)' into 'ElementwiseAdd_hls_0(hls::stream<ap_uint<2>, 0>&, hls::stream<ap_uint<3>, 0>&)' (/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp:71:13)
INFO: [HLS 214-131] Inlining function 'Slice<ap_int<2>, 2u>::Container<ap_uint<2> >::operator()(unsigned int, unsigned int) const' into 'ElementwiseAdd_hls_0(hls::stream<ap_uint<2>, 0>&, hls::stream<ap_uint<3>, 0>&)' (/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp:59:18)
INFO: [HLS 214-131] Inlining function 'Slice<ap_int<2>, 2u>::Container<ap_uint<2> > Slice<ap_int<2>, 2u>::operator()<ap_uint<2> >(ap_uint<2> const&) const' into 'ElementwiseAdd_hls_0(hls::stream<ap_uint<2>, 0>&, hls::stream<ap_uint<3>, 0>&)' (/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp:54:25)
ERROR: [HLS 214-171] in function 'ElementwiseAdd_hls_0(hls::stream<ap_uint<2>, 0>&, hls::stream<ap_uint<3>, 0>&)': Cannot apply disaggregate pragma/directive on object 'rhs' due to the existence of one or more type conversion operators (/dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/top_ElementwiseAdd_hls_0.cpp:67:27)
ERROR: [HLS 200-1715] Encountered problem during source synthesis
INFO: [HLS 200-111] Finished Command csynth_design CPU user time: 93.96 seconds. CPU system time: 4.06 seconds. Elapsed time: 100.87 seconds; current allocated memory: 8.789 MB.
Pre-synthesis failed.
    while executing
"source /dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/hls_syn_ElementwiseAdd_hls_0.tcl"
    invoked from within
"hls::main /dev/shm/temporary_finn_dir/FINN_TMP/code_gen_ipgen_ElementwiseAdd_hls_0_bxrgqsua/hls_syn_ElementwiseAdd_hls_0.tcl"
    ("uplevel" body line 1)
    invoked from within
"uplevel 1 hls::main {*}$newargs"
    (procedure "hls_proc" line 16)
    invoked from within
"hls_proc [info nameofexecutable] $argv"
INFO: [HLS 200-112] Total CPU user time: 97.43 seconds. Total CPU system time: 7.17 seconds. Total elapsed time: 123.81 seconds; peak allocated memory: 456.859 MB.
INFO: [Common 17-206] Exiting vitis_hls at Fri Dec 20 10:08:35 2024...

[bwintermann]

We just found out that this particular HLS error does not seem to happen in version vitis_hls 23.01 anymore.