Builtins Implementation Status compared to GCC

[realqhc]

Here hosts the generated result of trying to compile gcc's builtins test suite on LLVM clang

Script

import os
import re
import subprocess

LLVM_PATH = "~/llvm-project/llvm/cmake-build-debug/bin"
GNU_TEST_PATH = "~/corev-gcc/gcc/testsuite/gcc.target/riscv"

def test_output(dg_final_line, output):
  # Extract the regular expression and expected count from the dg-final line
  match = re.search(r'scan-assembler-times "([^"]*)" (\d+)', dg_final_line)

  if match:
      pattern = match.group(1).replace("\\\\.", re.escape(".")).replace("\\(", "(").replace("\\)", ")").replace("\\[", "[") \
          .replace("\\]", "]").replace("\\?", "?").replace("\\:", ":").replace("\\|", "|").replace(",", ", ") \
          .replace('(?:.L[0-9])', '(?:.L[A-z0-9_]*)').replace('(?:.L[0-9]+)', '(?:.L[A-z0-9_]*)')
      expected_count = int(match.group(2))

      # Count the occurrences of the pattern in the output
      occurrences = len(re.findall(pattern, output))

      # Check if the occurrences match the expected count
      if occurrences == expected_count:
          return True, pattern, occurrences
      else:
          return False, pattern, occurrences
  else:
      return False, None, None


# walk through gnu tests
for root, dirs, files in os.walk(GNU_TEST_PATH):
  for file in files:
      if "cv-" in file and "fail" not in file:
          # get the builtin name
          with open(os.path.join(root, file)) as f:
              builtins = []
              tests = []
              for line in f.readlines():
                  if "__builtin_" in line:
                      builtin = re.match(r".*__builtin_(\w+).*", line).group(1)
                      if builtin not in builtins:
                          builtins.append(builtin)
                  if "dg-final" in line:
                      tests.append(line)

              # run the test
              if "cv-interrupt-all-f-" in file:
                  process = (f"{LLVM_PATH}/clang -target riscv32-unknown-linux-gnu -S -march=rv32if -mabi=ilp32f {os.path.join(root, file)} -o - ")
              else:
                  process = (f"{LLVM_PATH}/clang -target riscv32-unknown-linux-gnu "
                             f"-S -march=rv32if_xcvelw1p0_xcvmac1p0_xcvbitmanip1p0_xcvsimd1p0_xcvalu1p0 -mabi=ilp32f "
                             f"{os.path.join(root, file)} -o - ")
              result = 0
              try:
                  output = subprocess.check_output(process, shell=True, stderr=subprocess.STDOUT)
              except subprocess.CalledProcessError as e:
                  output = e.output
                  result = e.returncode
              stdout = output.decode("utf-8")
              # print("### " + file)
              if result != 0:
                  failed_builtins = []
                  for line in stdout.split("\n"):
                      if "__builtin_" in line:
                          builtin = re.match(r".*__builtin_(\w+).*", line).group(1)
                          if builtin not in failed_builtins:
                              failed_builtins.append(builtin)
                  print(f"- [ ] {file}")
                  for failed_builtin in failed_builtins:
                      print(f"  - {failed_builtin}")
              else:
                  failed_builtins = []
                  for test in tests:
                      passed, pattern, occurrences = test_output(line, stdout)
                      if not passed:
                          failed_builtins.append(pattern)
                  if len(failed_builtins) == 0:
                      print(f"- [x] {file}")
                  else:
                      print(f"- [ ] {file}")
                      for failed_builtin in failed_builtins:
                          print(f"  - {failed_builtin}")

• cv-interrupt-1.c
• cv-interrupt-2.c
• cv-interrupt-5.c
• cv-interrupt-all-f-1.c
• cv-interrupt-all-f-2.c
• cv-interrupt-all-x-1.c
• cv-interrupt-all-x-2.c
• cv-interrupt-all-x-3.c
• cv-interrupt-all-x-4.c
• cv-interrupt-conflict-type.c
• cv-interrupt-debug.c
• cv-interrupt-mmode.c
• cv-interrupt-smode.c
• cv-interrupt-umode.c
• cv-alu-compile.c
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
  • cv.suburnr\t
• cv-mac-compile.c
• cv-mac-test-autogeneration.c
  • cv.msu
  • cv.msu
• cv-bi-beqimm-compile-1.c
  • cv.beqimm\t(?:t[0-6]|a[0-7]|s[1-11]), 10, (?:.L[A-z0-9_]*)
• cv-bi-beqimm-compile-2.c
  • beq\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
  • beq\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
  • beq\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
  • beq\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
• cv-interrupt-3.c
• cv-interrupt-4.c
• cv-hwlp-shiftsub.c
• cv-bitmanip-compile-bclr.c
• cv-bitmanip-compile-bclrr.c
• cv-bitmanip-compile-bitrev.c
• cv-bitmanip-compile-bset.c
• cv-bitmanip-compile-bsetr.c
• cv-bitmanip-compile-clb.c
• cv-bitmanip-compile-cnt.c
• cv-bitmanip-compile-extract.c
• cv-bitmanip-compile-extractr.c
• cv-bitmanip-compile-extractu.c
• cv-bitmanip-compile-extractur.c
• cv-bitmanip-compile-ff1.c
• cv-bitmanip-compile-fl1.c
• cv-bitmanip-compile-insert.c
• cv-bitmanip-compile-insertr.c
• cv-bitmanip-compile-ror.c
• cv-mem-operand-compile-1.c
• cv-mem-operand-compile-2.c
• cv-mem-operand-compile-3.c
• cv-mem-operand-compile-4.c
• cv-mem-operand-compile-5.c
• cv-mem-operand-compile-6.c
• cv-mem-operand-compile-7.c
• cv-mem-operand-compile-8.c
• cv-simd-march-compile-1.c
  • riscv_cv_simd_extract_h
  • riscv_cv_simd_extract_b
  • riscv_cv_simd_extractu_h
  • riscv_cv_simd_extractu_b
  • riscv_cv_simd_insert_h
  • riscv_cv_simd_insert_b
• cv-bi-bneimm-compile-1.c
  • cv.bneimm\t(?:t[0-6]|a[0-7]|s[1-11]), 10, (?:.L[A-z0-9_]*)
• cv-bi-bneimm-compile-2.c
  • bne\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
  • bne\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
  • bne\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
  • bne\t(?:t[0-6]|a[0-7]|s[1-11]), (?:t[0-6]|a[0-7]|s[1-11]), (?:.L[A-z0-9_]*)
• cv-elw-elw-compile-1.c
• cv-mem-lb-compile-1.c
  • cv.lb\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 1
• cv-mem-lb-compile-2.c
  • cv.lb\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-lb-compile-3.c
• cv-mem-lbu-compile-1.c
  • cv.lbu\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 1
• cv-mem-lbu-compile-2.c
  • cv.lbu\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-lbu-compile-3.c
• cv-mem-lh-compile-1.c
  • cv.lh\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 2
• cv-mem-lh-compile-2.c
  • cv.lh\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-lh-compile-3.c
• cv-mem-lhu-compile-1.c
  • cv.lhu\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 2
• cv-mem-lhu-compile-2.c
  • cv.lhu\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-lhu-compile-3.c
• cv-mem-lw-compile-1.c
  • cv.lw\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 4
• cv-mem-lw-compile-2.c
  • cv.lw\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-lw-compile-3.c
• cv-mem-sb-compile-1.c
  • cv.sb\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 1
• cv-mem-sb-compile-2.c
  • cv.sb\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-sb-compile-3.c
• cv-mem-sh-compile-1.c
  • cv.sh\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 2
• cv-mem-sh-compile-2.c
  • cv.sh\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-sh-compile-3.c
• cv-mem-sw-compile-1.c
  • cv.sw\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), 4
• cv-mem-sw-compile-2.c
  • cv.sw\t(?:t[0-6]|a[0-7]|s[1-11]), ((?:t[0-6]|a[0-7]|s[1-11])), (?:t[0-6]|a[0-7]|s[1-11])
• cv-mem-sw-compile-3.c
• cv-simd-abs-b-compile-1.c
• cv-simd-abs-h-compile-1.c
• cv-simd-add-b-compile-1.c
• cv-simd-add-div2-compile-1.c
• cv-simd-add-div4-compile-1.c
• cv-simd-add-div8-compile-1.c
• cv-simd-add-h-compile-1.c
• cv-simd-add-sc-b-compile-1.c
• cv-simd-add-sc-h-compile-1.c
• cv-simd-and-b-compile-1.c
• cv-simd-and-h-compile-1.c
• cv-simd-and-sc-b-compile-1.c
• cv-simd-and-sc-h-compile-1.c
• cv-simd-avg-b-compile-1.c
• cv-simd-avg-h-compile-1.c
• cv-simd-avg-sc-b-compile-1.c
• cv-simd-avg-sc-h-compile-1.c
• cv-simd-avgu-b-compile-1.c
• cv-simd-avgu-h-compile-1.c
• cv-simd-avgu-sc-b-compile-1.c
• cv-simd-avgu-sc-h-compile-1.c
• cv-simd-cmpeq-b-compile-1.c
• cv-simd-cmpeq-h-compile-1.c
• cv-simd-cmpeq-sc-b-compile-1.c
• cv-simd-cmpeq-sc-h-compile-1.c
• cv-simd-cmpge-b-compile-1.c
• cv-simd-cmpge-h-compile-1.c
• cv-simd-cmpge-sc-b-compile-1.c
• cv-simd-cmpge-sc-h-compile-1.c
• cv-simd-cmpgeu-b-compile-1.c
• cv-simd-cmpgeu-h-compile-1.c
• cv-simd-cmpgeu-sc-b-compile-1.c
• cv-simd-cmpgeu-sc-h-compile-1.c
• cv-simd-cmpgt-b-compile-1.c
• cv-simd-cmpgt-h-compile-1.c
• cv-simd-cmpgt-sc-b-compile-1.c
• cv-simd-cmpgt-sc-h-compile-1.c
• cv-simd-cmpgtu-b-compile-1.c
• cv-simd-cmpgtu-h-compile-1.c
• cv-simd-cmpgtu-sc-b-compile-1.c
• cv-simd-cmpgtu-sc-h-compile-1.c
• cv-simd-cmple-b-compile-1.c
• cv-simd-cmple-h-compile-1.c
• cv-simd-cmple-sc-b-compile-1.c
• cv-simd-cmple-sc-h-compile-1.c
• cv-simd-cmpleu-b-compile-1.c
• cv-simd-cmpleu-h-compile-1.c
• cv-simd-cmpleu-sc-b-compile-1.c
• cv-simd-cmpleu-sc-h-compile-1.c
• cv-simd-cmplt-b-compile-1.c
• cv-simd-cmplt-h-compile-1.c
• cv-simd-cmplt-sc-b-compile-1.c
• cv-simd-cmplt-sc-h-compile-1.c
• cv-simd-cmpltu-b-compile-1.c
• cv-simd-cmpltu-h-compile-1.c
• cv-simd-cmpltu-sc-b-compile-1.c
• cv-simd-cmpltu-sc-h-compile-1.c
• cv-simd-cmpne-b-compile-1.c
• cv-simd-cmpne-h-compile-1.c
• cv-simd-cmpne-sc-b-compile-1.c
• cv-simd-cmpne-sc-h-compile-1.c
• cv-simd-cplxconj-compile-1.c
• cv-simd-cplxmul-i-compile-1.c
• cv-simd-cplxmul-i-div2-compile-1.c
• cv-simd-cplxmul-i-div4-compile-1.c
• cv-simd-cplxmul-i-div8-compile-1.c
• cv-simd-cplxmul-r-compile-1.c
• cv-simd-cplxmul-r-div2-compile-1.c
• cv-simd-cplxmul-r-div4-compile-1.c
• cv-simd-cplxmul-r-div8-compile-1.c
• cv-simd-dotsp-b-compile-1.c
• cv-simd-dotsp-h-compile-1.c
• cv-simd-dotsp-sc-b-compile-1.c
• cv-simd-dotsp-sc-h-compile-1.c
• cv-simd-dotup-b-compile-1.c
• cv-simd-dotup-h-compile-1.c
• cv-simd-dotup-sc-b-compile-1.c
• cv-simd-dotup-sc-h-compile-1.c
• cv-simd-dotusp-b-compile-1.c
• cv-simd-dotusp-h-compile-1.c
• cv-simd-dotusp-sc-b-compile-1.c
• cv-simd-dotusp-sc-h-compile-1.c
• cv-simd-extract-b-compile-1.c
  • riscv_cv_simd_extract_b
• cv-simd-extract-h-compile-1.c
  • riscv_cv_simd_extract_h
• cv-simd-extractu-b-compile-1.c
  • riscv_cv_simd_extractu_b
• cv-simd-extractu-h-compile-1.c
  • riscv_cv_simd_extractu_h
• cv-simd-insert-b-compile-1.c
  • riscv_cv_simd_insert_b
• cv-simd-insert-h-compile-1.c
  • riscv_cv_simd_insert_h
• cv-simd-max-b-compile-1.c
• cv-simd-max-h-compile-1.c
• cv-simd-max-sc-b-compile-1.c
• cv-simd-max-sc-h-compile-1.c
• cv-simd-maxu-b-compile-1.c
• cv-simd-maxu-h-compile-1.c
• cv-simd-maxu-sc-b-compile-1.c
• cv-simd-maxu-sc-h-compile-1.c
• cv-simd-min-b-compile-1.c
• cv-simd-min-h-compile-1.c
• cv-simd-min-sc-b-compile-1.c
• cv-simd-min-sc-h-compile-1.c
• cv-simd-minu-b-compile-1.c
• cv-simd-minu-h-compile-1.c
• cv-simd-minu-sc-b-compile-1.c
• cv-simd-minu-sc-h-compile-1.c
• cv-simd-neg-b-compile-1.c
• cv-simd-neg-h-compile-1.c
• cv-simd-or-b-compile-1.c
• cv-simd-or-h-compile-1.c
• cv-simd-or-sc-b-compile-1.c
• cv-simd-or-sc-h-compile-1.c
• cv-simd-pack-compile-1.c
• cv-simd-pack-h-compile-1.c
• cv-simd-packhi-b-compile-1.c
• cv-simd-packlo-b-compile-1.c
• cv-simd-sdotsp-b-compile-1.c
• cv-simd-sdotsp-h-compile-1.c
• cv-simd-sdotsp-sc-b-compile-1.c
• cv-simd-sdotsp-sc-h-compile-1.c
• cv-simd-sdotup-b-compile-1.c
• cv-simd-sdotup-h-compile-1.c
• cv-simd-sdotup-sc-b-compile-1.c
• cv-simd-sdotup-sc-h-compile-1.c
• cv-simd-sdotusp-b-compile-1.c
• cv-simd-sdotusp-h-compile-1.c
• cv-simd-sdotusp-sc-b-compile-1.c
• cv-simd-sdotusp-sc-h-compile-1.c
• cv-simd-shuffle-sci-h-compile-1.c
• cv-simd-shuffle2-b-compile-1.c
• cv-simd-shuffle2-h-compile-1.c
• cv-simd-shufflei0-sci-b-compile-1.c
• cv-simd-shufflei1-sci-b-compile-1.c
• cv-simd-shufflei2-sci-b-compile-1.c
• cv-simd-shufflei3-sci-b-compile-1.c
• cv-simd-sll-b-compile-1.c
• cv-simd-sll-h-compile-1.c
• cv-simd-sll-sc-b-compile-1.c
• cv-simd-sll-sc-h-compile-1.c
• cv-simd-sra-b-compile-1.c
• cv-simd-sra-h-compile-1.c
• cv-simd-sra-sc-b-compile-1.c
• cv-simd-sra-sc-h-compile-1.c
• cv-simd-srl-b-compile-1.c
• cv-simd-srl-h-compile-1.c
• cv-simd-srl-sc-b-compile-1.c
• cv-simd-srl-sc-h-compile-1.c
• cv-simd-sub-b-compile-1.c
• cv-simd-sub-div2-compile-1.c
• cv-simd-sub-div4-compile-1.c
• cv-simd-sub-div8-compile-1.c
• cv-simd-sub-h-compile-1.c
• cv-simd-sub-sc-b-compile-1.c
• cv-simd-sub-sc-h-compile-1.c
• cv-simd-subrotmj-compile-1.c
• cv-simd-subrotmj-div2-compile-1.c
• cv-simd-subrotmj-div4-compile-1.c
• cv-simd-subrotmj-div8-compile-1.c
• cv-simd-xor-b-compile-1.c
• cv-simd-xor-h-compile-1.c
• cv-simd-xor-sc-b-compile-1.c
• cv-simd-xor-sc-h-compile-1.c
• vadd-rv32gcv-nofm.c
• vadd-rv64gcv-nofm.c
• vdiv-rv32gcv-nofm.c
• vdiv-rv64gcv-nofm.c
• vmul-rv32gcv-nofm.c
• vmul-rv64gcv-nofm.c
• vsub-rv32gcv-nofm.c
• vsub-rv64gcv-nofm.c

Oct. 17 2023: Updated a run with commit bd44b26 and openhwgroup/corev-gcc@276b330
Aug. 25 2023: Updated a run with #75 and #77
Aug. 28 2023: Updated a run on openhwgroup/corev-gcc@613923e
Dec. 28 2023: Updated a run on e93f95c

[realqhc]

To reproduce the result, run clang -target riscv32-unknown-linux-gnu -S -march=rv32if_xcvelw1p0_xcvmac1p0_xcvbitmanip1p0_xcvsimd1p0_xcvalu1p0 -mabi=ilp32f {gcc test file} -o -

[melonedo]

The cv-interrupt-all-f-* tests need hard-float support: clang -target riscv32-unknown-linux-gnu -S -march=rv32if -mabi=ilp32f {gcc test file}, while other cv-interrupt tests concern "corev-fast" interrupt mode (openhwgroup/corev-gcc@d4528cf).
The cv-simd-shufflei* tests are not correctly implemented by GCC (#61).
The cv-simd-{extract/extractu/insert/sll/sra/srl}-* tests require a signed immediate operand, which is inconsistent with the spec for builtins.
The cv-simd-avgu-sc-* test failures are due to a change to the signedness (openhwgroup/corev-gcc@eeedfe3).

[melonedo]

To reproduce the result, run clang -cc1 -triple riscv32 -S -target-feature +xcvbi -target-feature +xcvelw -target-feature +xcvmac -target-feature +xcvbitmanip -target-feature +xcvsimd -target-feature +xcvalu {gcc test file} -o -

-target-feature +xcvmem is also needed to get cv-mem-* tests passed. Also, two tests did not pass because test-dg-not is not implemented.

[sun-jacobi]

@realqhc Hi, do you know which march I should use if I want to use the xcvmem ?

[realqhc]

@realqhc Hi, do you know which march I should use if I want to use the xcvmem ?

-march=rv32if_xcvelw1p0_xcvmac1p0_xcvbitmanip1p0_xcvsimd1p0_xcvalu1p0_xcvmem1p0 after #89 is merged

[sun-jacobi]

cv-mac-test-autogeneration.c will be passed if using rv32im. Without m, the multiplication would be expanded to libcall, the pattern match could not work on this case.

Is it possible that there exists MAC but no normal multiplication ?

[sun-jacobi]

cv-alu-compile.c is due to gcc using lower case for builtin like cv.suburnr