Rebase refactored TableGen backends onto LLVM 18.

The MCInstDesc table changed. Bsides this only minor changes were done
and some additional code is emitted now for LLVM.

This commit is the combination of all previous Auto-Sync commits.
The list of commit messages follows:

-----------

Combination of all commits of the refactored tablegen backends.

These are the changes made for LLVM 16.

Refactor Capstone relevant TableGen Emitter backends.

This commit extracts the code which emits generated tables into two printer classes.
The Printer is called whenever actual code is written to a file.
There is the PrinterLLVM which emits tht code as before and
PrinterCapstone which is tailored to or needs (emitting C and generates
more info).

Additionally missing memory access properties were added to ARMs td
files.

Emit a single header for all files.

Captialize Target name for enums.

Add lay metric to emit enum value for Banked and system regs.

Malloc substr

Sort instructions in ascending order.

Free substr after use

Add vanished constrainsts

Fix `regInfoEmitEnums()` and indent

Fix `GenDisassemblerTables.inc#checkDecoderPredicate()`

Fix `TriCoreGenRegisterInfo.inc` | `PrinterCapstone::regInfoEmitRegClasses`

revert changes to NEON instructions

Add instructions with duplicate operands as Matchables.

Add memory load and store info

Correct memory access and out operand info

Set register lists again as read ops due to llvm/llvm-project#62455

Make printAliasInstr and getMnemonic static.

Generate CS instruction enums from actual mnemonic. Not via the flawed AsmMatcher.

Fix typo in InstrInfoEmitter.cpp

Add deprecated QPX feature

Replace + and - with p and m

Add AssemblerPredicates to PPC

Generate RegEncodingTable

Define functions which are called by the Mapper as static.

Necessary because these functions are present in each arch'

Remove set_mem_access().

The cases where this is used to mark access to actual memory operands are
either very rare, or those are neon lane indicies.

Generate correct op type for absolute addresses.

Check for RegisterPointer operands first to prevent mis-categorization.

Add missing Operand types

Generate Instruction formats for PPC.

Add Paired Single instructions.

Partly revert 94e41ce (introduces accidentially removed code.)

Set correct operand types for PS operands

Add memory read/write attributes

Add missing operand types

Add mayLoad and mayStore information.

Add documentation.

Handle special AArch64 operand

Replace C++ with C code.

Check for duplicate enum instr. names

Check for duplicate defintions of system registers.

Add note about missing target names.

Resolve templates in a single static method and add docs about it.

Revert printing target name in upper case.

Revert partially C++ syntax fixes in .td files.

They break the TemplateCOllector since it searches for exactly those references but can't find any'

Add all SubtargetFeatures to feature enum.

Not just the one used by CGIs.

Pass Decoder

Enable to check specific table fields to determine if reg enum must be emitted.

Allow to add namespace to type name/

Formatting

Rework emitting of tables.

The system operands are now emitted in reg, imm and aliass groups.
Also a bug was fixed which emitted incorrect code..

Check for rename IMPLICIT_IMM operand types

Pass DecodeComplete as pointer not as reference

Print undef when it needs to be printed.

Add namespace ids to all types and functions.

Rework C translation.

Pass MCOp as pointer not as ref

Add missing SysImm type

Fix syntax mistakes

Generate additonal sys immediates and op groups.

Handle edge case for printSVERegOp

Handle default arguments of template functions.

Add two missing op groups

Generate a static RecEncodingTable

Set enum values to encodings of the sys ops

Generate a single Enum value file for system operands.

Replace System operand groups with their operand types

Fix missing braces warning

Emit MCOperand validator.

Emit lookupByName functions for sys operands

Add namespaces for ARM.

Check for Target if default arguments of template functions are resolved.

auto-sync opcode & operand encoding info generation (#14)

* Added operand and opcode info generation

* Wrapped deprecated macro under an intellisense check

Basically intellisense fails, causing multiple errors in other files,

so when intellisense parses the code it will use the different version of the macro

* Fixed a small bug

Used double braces to prevent an old bug

Removed extra new line and fixed a bug regarding move semantics

Author: Rot127

DeprecatedFeatures.md

@@ -0,0 +1,41 @@
+# Deprecated Features
+
+Capstone needs to support features which were removed by LLVM in the past.
+Here we explain how to reintroduce them.
+
+## Reintroduction
+
+To get the old features back we copy them from the old `.td` files and include them in the new ones.
+
+To include removed features from previous LLVM versions do the following:
+
+1. Checkout the last LLVM version the feature was present.
+2. Copy all feature related definitions into a `<ARCH>Deprecated.td` file.
+3. Checkout the newest LLVM version again.
+4. Wrap the different definition types in include guards. For example the `InstrInfo` definitions could be included in:
+
+    ```
+    #ifndef INCLUDED_CAPSTONE_DEPR_INSTR
+    #ifdef CAPSTONE_DEPR_INSTR
+    #define INCLUDED_CAPSTONE_DEPR_INSTR // Ensures it is only included once
+    
+    [Instruction definitions of removed feature]
+    
+    #endif // INCLUDED_CAPSTONE_DEPR_INSTR
+    #endif // CAPSTONE_DEPR_INSTR
+    ```
+
+    _Note that the order of `#ifndef` and `#ifdef` matters (otherwise you'll get an error from `tblgen`)._
+
+5. Include the definitions in the current definition files with:
+
+    ```
+    #define CAPSTONE_DEPR_INSTR
+    include "<ARCH>Deprecated.md"
+    ```
+
+## Notes
+- It is possible that you have to change some definitions slightly.
+Because certain classes no longer exist or were replaced (e.g.: `GCCBuiltin` -> `ClangBuiltin`).
+- Some new processors might need to have the feature flag (`Has<DeprecatedFeature>`) added
+to their `UnsupportedFeatures` list.

README.md

@@ -1,44 +1,93 @@
-# The LLVM Compiler Infrastructure
+# Capstone's LLVM with refactored TableGen backends
 
-[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/llvm/llvm-project/badge)](https://securityscorecards.dev/viewer/?uri=github.com/llvm/llvm-project)
-[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8273/badge)](https://www.bestpractices.dev/projects/8273)
-[![libc++](https://github.com/llvm/llvm-project/actions/workflows/libcxx-build-and-test.yaml/badge.svg?branch=main&event=schedule)](https://github.com/llvm/llvm-project/actions/workflows/libcxx-build-and-test.yaml?query=event%3Aschedule)
+This LLVM version has the purpose to generate code for the
+[Capstone disassembler](https://github.com/capstone-engine/capstone).
 
-Welcome to the LLVM project!
+It refactors the TableGen emitter backends, so they can emit C code
+in addition to the C++ code they normally emit.
 
-This repository contains the source code for LLVM, a toolkit for the
-construction of highly optimized compilers, optimizers, and run-time
-environments.
+Please note that within LLVM we speak of a `Target` if we refer to an architecture.
 
-The LLVM project has multiple components. The core of the project is
-itself called "LLVM". This contains all of the tools, libraries, and header
-files needed to process intermediate representations and convert them into
-object files. Tools include an assembler, disassembler, bitcode analyzer, and
-bitcode optimizer.
+## Code generation
 
-C-like languages use the [Clang](http://clang.llvm.org/) frontend. This
-component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode
--- and from there into object files, using LLVM.
+### Relevant files
 
-Other components include:
-the [libc++ C++ standard library](https://libcxx.llvm.org),
-the [LLD linker](https://lld.llvm.org), and more.
+The TableGen emitter backends are located in `llvm/utils/TableGen/`.
 
-## Getting the Source Code and Building LLVM
+The target definition files (`.td`), which define the
+instructions, operands, features etc., can be
+found in `llvm/lib/Target/<ARCH>/`.
 
-Consult the
-[Getting Started with LLVM](https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm)
-page for information on building and running LLVM.
+### Code generation overview
 
-For information on how to contribute to the LLVM project, please take a look at
-the [Contributing to LLVM](https://llvm.org/docs/Contributing.html) guide.
+Generating code for a target has 6 steps:
 
-## Getting in touch
+```
+                                                                         5                 6
+                                                                    ┌──────────┐      ┌──────────┐
+                                                                    │Printer   │      │CS .inc   │
+    1               2                 3                4        ┌──►│Capstone  ├─────►│files     │
+┌───────┐     ┌───────────┐     ┌───────────┐     ┌──────────┐  │   └──────────┘      └──────────┘
+│ .td   │     │           │     │           │     │ Code-    │  │
+│ files ├────►│ TableGen  ├────►│  CodeGen  ├────►│ Emitter  │◄─┤
+└───────┘     └──────┬────┘     └───────────┘     └──────────┘  │
+                     │                                 ▲        │   ┌──────────┐      ┌──────────┐
+                     └─────────────────────────────────┘        └──►│Printer   ├─────►│LLVM .inc │
+                                                                    │LLVM      │      │files     │
+                                                                    └──────────┘      └──────────┘
+```
 
-Join the [LLVM Discourse forums](https://discourse.llvm.org/), [Discord
-chat](https://discord.gg/xS7Z362),
-[LLVM Office Hours](https://llvm.org/docs/GettingInvolved.html#office-hours) or
-[Regular sync-ups](https://llvm.org/docs/GettingInvolved.html#online-sync-ups).
+1. LLVM targets are defined in `.td` files. They describe instructions, operands,
+features and other properties.
 
-The LLVM project has adopted a [code of conduct](https://llvm.org/docs/CodeOfConduct.html) for
-participants to all modes of communication within the project.
+2. [LLVM TableGen](https://llvm.org/docs/TableGen/index.html) parses these files
+and converts them to an internal representation of [Classes, Records, DAGs](https://llvm.org/docs/TableGen/ProgRef.html)
+ and other types.
+
+3. In the second step a TableGen component called [CodeGen](https://llvm.org/docs/CodeGenerator.html)
+abstracts this even further.
+The result is a representation which is _not_ specific to any target
+(e.g. the `CodeGenInstruction` class can represent a machine instruction of any target).
+
+4. Different code emitter backends use the result of the former two components to
+generated code.
+
+5. Whenever the emitter emits code it calls a `Printer`. Either the `PrinterCapstone` to emit C or `PrinterLLVM` to emit C++.
+Which one is controlled by the `--printerLang=[CCS,C++]` option passed to `llvm-tblgen`.
+
+6. After the emitter backend is done, the `Printer` writes the `output_stream` content into the `.inc` files.
+
+### Emitter backends and their use cases
+
+We use the following emitter backends
+
+| Name | Generated Code | Note |
+|------|----------------|------|
+| AsmMatcherEmitter | Mapping tables for Capstone | |
+| AsmWriterEmitter | State machine to decode the asm-string for a `MCInst` | |
+| DecoderEmitter | State machine which decodes bytes to a `MCInst`. | |
+| InstrInfoEmitter | Tables with instruction information (instruction enum, instr. operand information...) | |
+| RegisterInfoEmitter | Tables with register information (register enum, register type info...) | |
+| SubtargetEmitter | Table about the target features. | |
+| SearchableTablesEmitter | Usually used to generate tables and decoding functions for system registers. | **1.** Not all targets use this. |
+| | | **2.** Backend can't access the target name. Wherever the target name is needed `__ARCH__` or `##ARCH##` is printed and later replaced. |
+
+## Developer notes
+
+- If you find C++ code within the generated files you need to extend `PrinterCapstone::translateToC()`.
+If this still doesn't fix the problem, the code snipped wasn't passed through `translateToC()` before emitting.
+So you need to figure out where this specific code snipped is printed and add `translateToC()`.
+
+- If the mapping files miss operand types or access information, then the `.td` files are incomplete (happens surprisingly often).
+You need to search for the instruction or operands with missing or incorrect values and fix them.
+  ```
+    Wrong access attributes for:
+      - Registers, Immediates: The instructions defines "out" and "in" operands incorrectly.
+      - Memory: The "mayLoad" or "mayStore" variable is not set for the instruction.
+
+    Operand type is invalid:
+      - The "OperandType" variable is unset for this operand type.
+  ```
+
+- If certain target features (e.g. architecture extensions) were removed from LLVM or you want to add your own,
+checkout [DeprecatedFeatures.md](DeprecatedFeatures.md).

llvm/include/llvm/Support/Compiler.h

@@ -151,7 +151,7 @@
 #define LLVM_ATTRIBUTE_USED
 #endif
 
-#if defined(__clang__)
+#if defined(__clang__) && !defined(__INTELLISENSE__)
 #define LLVM_DEPRECATED(MSG, FIX) __attribute__((deprecated(MSG, FIX)))
 #else
 #define LLVM_DEPRECATED(MSG, FIX) [[deprecated(MSG)]]

llvm/include/llvm/TableGen/StringMatcher.h

@@ -13,6 +13,7 @@
 #ifndef LLVM_TABLEGEN_STRINGMATCHER_H
 #define LLVM_TABLEGEN_STRINGMATCHER_H
 
+#include "PrinterTypes.h"
 #include "llvm/ADT/StringRef.h"
 #include <string>
 #include <utility>
@@ -35,18 +36,26 @@ class StringMatcher {
   StringRef StrVariableName;
   const std::vector<StringPair> &Matches;
   raw_ostream &OS;
+  PrinterLanguage PL;
 
 public:
   StringMatcher(StringRef strVariableName,
                 const std::vector<StringPair> &matches, raw_ostream &os)
-    : StrVariableName(strVariableName), Matches(matches), OS(os) {}
+    : StrVariableName(strVariableName), Matches(matches), OS(os), PL(PRINTER_LANG_CPP) {}
+  StringMatcher(StringRef strVariableName,
+                const std::vector<StringPair> &matches, raw_ostream &os, PrinterLanguage PL)
+    : StrVariableName(strVariableName), Matches(matches), OS(os), PL(PL) {}
 
   void Emit(unsigned Indent = 0, bool IgnoreDuplicates = false) const;
+  void EmitCPP(unsigned Indent = 0, bool IgnoreDuplicates = false) const;
 
 private:
   bool EmitStringMatcherForChar(const std::vector<const StringPair *> &Matches,
                                 unsigned CharNo, unsigned IndentCount,
                                 bool IgnoreDuplicates) const;
+  bool EmitStringMatcherForCharCPP(const std::vector<const StringPair *> &Matches,
+                                unsigned CharNo, unsigned IndentCount,
+                                bool IgnoreDuplicates) const;
 };
 
 } // end namespace llvm

llvm/include/llvm/TableGen/StringToOffsetTable.h

@@ -9,10 +9,12 @@
 #ifndef LLVM_TABLEGEN_STRINGTOOFFSETTABLE_H
 #define LLVM_TABLEGEN_STRINGTOOFFSETTABLE_H
 
+#include "PrinterTypes.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/Support/raw_ostream.h"
+#include "llvm/TableGen/Error.h"
 #include <cctype>
 
 namespace llvm {
@@ -22,10 +24,14 @@ namespace llvm {
 /// It can then output this string blob and use indexes into the string to
 /// reference each piece.
 class StringToOffsetTable {
+  PrinterLanguage PL;
   StringMap<unsigned> StringOffset;
   std::string AggregateString;
 
 public:
+  StringToOffsetTable() : PL(PRINTER_LANG_CPP) {};
+  StringToOffsetTable(PrinterLanguage PL) : PL(PL) {};
+
   bool Empty() const { return StringOffset.empty(); }
 
   unsigned GetOrAddStringOffset(StringRef Str, bool appendZero = true) {
@@ -42,6 +48,16 @@ class StringToOffsetTable {
   }
 
   void EmitString(raw_ostream &O) {
+    switch(PL) {
+    default:
+      PrintFatalNote("No StringToOffsetTable method defined to emit the selected language.\n");
+    case PRINTER_LANG_CPP:
+      EmitStringCPP(O);
+      break;
+    }
+  }
+
+  void EmitStringCPP(raw_ostream &O) {
     // Escape the string.
     SmallString<256> Str;
     raw_svector_ostream(Str).write_escaped(AggregateString);

llvm/lib/Support/BLAKE3/CMakeLists.txt

@@ -10,6 +10,7 @@ if (LLVM_DISABLE_ASSEMBLY_FILES)
 else()
   set(CAN_USE_ASSEMBLER TRUE)
 endif()
+set(CAN_USE_ASSEMBLER FALSE)
 
 macro(disable_blake3_x86_simd)
   add_compile_definitions(BLAKE3_NO_AVX512 BLAKE3_NO_AVX2 BLAKE3_NO_SSE41 BLAKE3_NO_SSE2)

llvm/lib/TableGen/CMakeLists.txt

@@ -6,7 +6,6 @@ add_llvm_component_library(LLVMTableGen
   Parser.cpp
   Record.cpp
   SetTheory.cpp
-  StringMatcher.cpp
   TableGenBackend.cpp
   TableGenBackendSkeleton.cpp
   TGLexer.cpp

llvm/lib/Target/ARM/ARMInstrFormats.td

@@ -669,6 +669,7 @@ class AIldr_ex_or_acq<bits<2> opcod, bits<2> opcod2, dag oops, dag iops, InstrIt
   let Inst{11-10} = 0b11;
   let Inst{9-8}   = opcod2;
   let Inst{7-0}   = 0b10011111;
+  let mayLoad = 1;
 }
 class AIstr_ex_or_rel<bits<2> opcod, bits<2> opcod2, dag oops, dag iops, InstrItinClass itin,
               string opc, string asm, list<dag> pattern>
@@ -684,6 +685,7 @@ class AIstr_ex_or_rel<bits<2> opcod, bits<2> opcod2, dag oops, dag iops, InstrIt
   let Inst{9-8}   = opcod2;
   let Inst{7-4}   = 0b1001;
   let Inst{3-0}   = Rt;
+  let mayStore = 1;
 }
 // Atomic load/store instructions
 class AIldrex<bits<2> opcod, dag oops, dag iops, InstrItinClass itin,
@@ -695,6 +697,7 @@ class AIstrex<bits<2> opcod, dag oops, dag iops, InstrItinClass itin,
   : AIstr_ex_or_rel<opcod, 0b11, oops, iops, itin, opc, asm, pattern> {
   bits<4> Rd;
   let Inst{15-12} = Rd;
+  let mayLoad = 1;
 }
 
 // Exclusive load/store instructions
@@ -792,6 +795,8 @@ class AI2ldstidx<bit isLd, bit isByte, bit isPre, dag oops, dag iops,
   let Inst{21}    = isPre; // W bit
   let Inst{20}    = isLd; // L bit
   let Inst{15-12} = Rt;
+  let mayLoad     = isLd;
+  let mayStore    = !eq(isLd, 0);
 }
 class AI2stridx_reg<bit isByte, bit isPre, dag oops, dag iops,
                 IndexMode im, Format f, InstrItinClass itin, string opc,
@@ -809,6 +814,7 @@ class AI2stridx_reg<bit isByte, bit isPre, dag oops, dag iops,
   let Inst{11-5} = offset{11-5};
   let Inst{4} = 0;
   let Inst{3-0} = offset{3-0};
+  let mayStore = 1;
 }
 
 class AI2stridx_imm<bit isByte, bit isPre, dag oops, dag iops,
@@ -825,6 +831,7 @@ class AI2stridx_imm<bit isByte, bit isPre, dag oops, dag iops,
   let Inst{23} = offset{12};
   let Inst{19-16} = Rn;
   let Inst{11-0} = offset{11-0};
+  let mayStore = 1;
 }
 
 
@@ -845,6 +852,7 @@ class AI2stridxT<bit isByte, bit isPre, dag oops, dag iops,
   let Inst{23} = addr{12};
   let Inst{19-16} = addr{17-14};
   let Inst{11-0} = addr{11-0};
+  let mayStore = 1;
 }
 
 // addrmode3 instructions
@@ -865,6 +873,8 @@ class AI3ld<bits<4> op, bit op20, dag oops, dag iops, Format f,
   let Inst{11-8}  = addr{7-4};    // imm7_4/zero
   let Inst{7-4}   = op;
   let Inst{3-0}   = addr{3-0};    // imm3_0/Rm
+  let mayLoad     = op20;
+  let mayStore    = 0;
 
   let DecoderMethod = "DecodeAddrMode3Instruction";
 }
@@ -881,6 +891,8 @@ class AI3ldstidx<bits<4> op, bit op20, bit isPre, dag oops, dag iops,
   let Inst{20}    = op20;         // L bit
   let Inst{15-12} = Rt;           // Rt
   let Inst{7-4}   = op;
+  let mayLoad     = op20;
+  let mayStore    = !if(op20, 0, 1);
 }
 
 // FIXME: Merge with the above class when addrmode2 gets used for LDR, LDRB
@@ -903,6 +915,8 @@ class AI3ldstidxT<bits<4> op, bit isLoad, dag oops, dag iops,
   let Inst{19-16} = addr;         // Rn
   let Inst{15-12} = Rt;           // Rt
   let Inst{7-4}   = op;
+  let mayLoad     = isLoad;
+  let mayStore    = !if(isLoad, 0, 1);
 }
 
 // stores
@@ -924,6 +938,7 @@ class AI3str<bits<4> op, dag oops, dag iops, Format f, InstrItinClass itin,
   let Inst{7-4}   = op;
   let Inst{3-0}   = addr{3-0};    // imm3_0/Rm
   let DecoderMethod = "DecodeAddrMode3Instruction";
+  let mayStore    = 1;
 }
 
 // addrmode4 instructions