Added partial support for compiling C++20 modules and header-units without scanning. #147682
+5,042
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
llvmbot
added
clang
|
@llvm/pr-subscribers-clang-driver Author: Hassan Sajjad (HassanSajjad-302) ChangesThis WIP pull request adds partial support for my draft N2978. In my paper, I have referred to a sample implementation of the paper, In it, I define a type // IPCManagerCompiler class functions
[[nodiscard]] tl::expected<BTCModule, string> receiveBTCModule(const CTBModule &moduleName) const;
[[nodiscard]] tl::expected<BTCNonModule, string> receiveBTCNonModule(const CTBNonModule &nonModule) const;
[[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage) const;
[[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage,
const string &bmiFile, const string &filePath) const;
static tl::expected<ProcessMappingOfBMIFile, string> readSharedMemoryBMIFile(const BMIFile &file);
static tl::expected<void, string> closeBMIFileMapping(const ProcessMappingOfBMIFile &processMappingOfBMIFile);
void closeConnection() const;
// A global function
[[nodiscard]] tl::expected<IPCManagerCompiler, string> makeIPCManagerCompiler(string BMIIfHeaderUnitObjOtherwisePath);
Of these seven functions, It invokes the compilation of a
I opened this half-baked pull-request to request assistance and feedback HMake will also support Clang's 2-phase compilation with this approach. No other build-system can support C++20 modules the way HMake can. Due to these reasons, in the future, I would like to propose it for LLVM as well. @vgvassilev @Bigcheese Patch is 183.41 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/147682.diff 25 Files Affected:
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 77379f1130149..6a6010b4baa44 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -9416,3 +9416,13 @@ def wasm_opt : Flag<["--"], "wasm-opt">,
Group<m_Group>,
HelpText<"Enable the wasm-opt optimizer (default)">,
MarshallingInfoNegativeFlag<LangOpts<"NoWasmOpt">>;
+
+def no_scan_ipc : Flag<["-"], "noScanIPC">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Enable No scan IPC approach">;
+def translate_include : Flag<["-"], "translateInclude">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Consider header includes as header-units">;
+def find_include : Flag<["-"], "findIncludes">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"In No Scan IPC approach, compiler will relay the finding includes responsibility to the build-system">;
diff --git a/clang/include/clang/Frontend/CompilerInstance.h b/clang/include/clang/Frontend/CompilerInstance.h
index 0ae490f0e8073..84a524a549e02 100644
--- a/clang/include/clang/Frontend/CompilerInstance.h
+++ b/clang/include/clang/Frontend/CompilerInstance.h
@@ -16,6 +16,7 @@
#include "clang/Frontend/CompilerInvocation.h"
#include "clang/Frontend/PCHContainerOperations.h"
#include "clang/Frontend/Utils.h"
+#include "clang/IPC2978/IPCManagerCompiler.hpp"
#include "clang/Lex/DependencyDirectivesScanner.h"
#include "clang/Lex/HeaderSearchOptions.h"
#include "clang/Lex/ModuleLoader.h"
@@ -180,6 +181,10 @@ class CompilerInstance : public ModuleLoader {
/// The stream for verbose output.
raw_ostream *VerboseOutputStream = &llvm::errs();
+ /// Pointer for managing communication with build-system if noScan flag is
+ /// set.
+ N2978::IPCManagerCompiler *ipcManager = nullptr;
+
/// Holds information about the output file.
///
/// If TempFilename is not empty we must rename it to Filename at the end.
diff --git a/clang/include/clang/IPC2978/.clang-format-ignore b/clang/include/clang/IPC2978/.clang-format-ignore
new file mode 100644
index 0000000000000..f59ec20aabf58
--- /dev/null
+++ b/clang/include/clang/IPC2978/.clang-format-ignore
@@ -0,0 +1 @@
+*
\ No newline at end of file
diff --git a/clang/include/clang/IPC2978/IPCManagerBS.hpp b/clang/include/clang/IPC2978/IPCManagerBS.hpp
new file mode 100644
index 0000000000000..f9222a55f3fc4
--- /dev/null
+++ b/clang/include/clang/IPC2978/IPCManagerBS.hpp
@@ -0,0 +1,39 @@
+
+#ifndef IPC_MANAGER_BS_HPP
+#define IPC_MANAGER_BS_HPP
+
+#include "clang/IPC2978/Manager.hpp"
+#include "clang/IPC2978/Messages.hpp"
+
+namespace N2978
+{
+
+// IPC Manager BuildSystem
+class IPCManagerBS : Manager
+{
+ friend tl::expected<IPCManagerBS, string> makeIPCManagerBS(string BMIIfHeaderUnitObjOtherwisePath);
+ bool connectedToCompiler = false;
+
+#ifdef _WIN32
+ explicit IPCManagerBS(void *hPipe_);
+#else
+ explicit IPCManagerBS(int fdSocket_);
+#endif
+
+ public:
+ IPCManagerBS(const IPCManagerBS &) = default;
+ IPCManagerBS &operator=(const IPCManagerBS &) = default;
+ IPCManagerBS(IPCManagerBS &&) = default;
+ IPCManagerBS &operator=(IPCManagerBS &&) = default;
+ tl::expected<void, string> receiveMessage(char (&ctbBuffer)[320], CTB &messageType) const;
+ [[nodiscard]] tl::expected<void, string> sendMessage(const BTCModule &moduleFile) const;
+ [[nodiscard]] tl::expected<void, string> sendMessage(const BTCNonModule &nonModule) const;
+ [[nodiscard]] tl::expected<void, string> sendMessage(const BTCLastMessage &lastMessage) const;
+ static tl::expected<ProcessMappingOfBMIFile, string> createSharedMemoryBMIFile(const BMIFile &bmiFile);
+ static tl::expected<void, string> closeBMIFileMapping(const ProcessMappingOfBMIFile &processMappingOfBMIFile);
+ void closeConnection() const;
+};
+
+tl::expected<IPCManagerBS, string> makeIPCManagerBS(string BMIIfHeaderUnitObjOtherwisePath);
+} // namespace N2978
+#endif // IPC_MANAGER_BS_HPP
diff --git a/clang/include/clang/IPC2978/IPCManagerCompiler.hpp b/clang/include/clang/IPC2978/IPCManagerCompiler.hpp
new file mode 100644
index 0000000000000..6aba39822d5ac
--- /dev/null
+++ b/clang/include/clang/IPC2978/IPCManagerCompiler.hpp
@@ -0,0 +1,129 @@
+
+#ifndef IPC_MANAGER_COMPILER_HPP
+#define IPC_MANAGER_COMPILER_HPP
+
+#include "clang/IPC2978/Manager.hpp"
+#include "clang/IPC2978/expected.hpp"
+
+using std::string_view;
+namespace N2978
+{
+
+// IPC Manager Compiler
+class IPCManagerCompiler : Manager
+{
+ template <typename T> tl::expected<T, string> receiveMessage() const;
+ // This is not exposed. sendCTBLastMessage calls this.
+ [[nodiscard]] tl::expected<void, string> receiveBTCLastMessage() const;
+
+ public:
+#ifdef _WIN32
+ explicit IPCManagerCompiler(void *hPipe_);
+#else
+ explicit IPCManagerCompiler(int fdSocket_);
+#endif
+ [[nodiscard]] tl::expected<BTCModule, string> receiveBTCModule(const CTBModule &moduleName) const;
+ [[nodiscard]] tl::expected<BTCNonModule, string> receiveBTCNonModule(const CTBNonModule &nonModule) const;
+ [[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage) const;
+ [[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage,
+ const string &bmiFile, const string &filePath) const;
+ static tl::expected<ProcessMappingOfBMIFile, string> readSharedMemoryBMIFile(const BMIFile &file);
+ static tl::expected<void, string> closeBMIFileMapping(const ProcessMappingOfBMIFile &processMappingOfBMIFile);
+ void closeConnection() const;
+};
+
+template <typename T> tl::expected<T, string> IPCManagerCompiler::receiveMessage() const
+{
+ // Read from the pipe.
+ char buffer[BUFFERSIZE];
+ uint32_t bytesRead;
+ if (const auto &r = readInternal(buffer); !r)
+ {
+ return tl::unexpected(r.error());
+ }
+ else
+ {
+ bytesRead = *r;
+ }
+
+ uint32_t bytesProcessed = 0;
+
+ if constexpr (std::is_same_v<T, BTCModule>)
+ {
+ const auto &r = readProcessMappingOfBMIFileFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r)
+ {
+ return tl::unexpected(r.error());
+ }
+
+ const auto &r2 = readVectorOfModuleDepFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r2)
+ {
+ return tl::unexpected(r2.error());
+ }
+
+ BTCModule moduleFile;
+ moduleFile.requested = *r;
+ moduleFile.deps = *r2;
+ if (bytesRead == bytesProcessed)
+ {
+ return moduleFile;
+ }
+ }
+ else if constexpr (std::is_same_v<T, BTCNonModule>)
+ {
+ const auto &r = readBoolFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r)
+ {
+ return tl::unexpected(r.error());
+ }
+
+ const auto &r2 = readStringFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r2)
+ {
+ return tl::unexpected(r2.error());
+ }
+
+ const auto &r3 = readBoolFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r3)
+ {
+ return tl::unexpected(r3.error());
+ }
+
+ const auto &r4 = readUInt32FromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r4)
+ {
+ return tl::unexpected(r4.error());
+ }
+
+ const auto &r5 = readVectorOfHuDepFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r5)
+ {
+ return tl::unexpected(r5.error());
+ }
+
+ BTCNonModule nonModule;
+ nonModule.isHeaderUnit = *r;
+ nonModule.filePath = *r2;
+ nonModule.angled = *r3;
+ nonModule.fileSize = *r4;
+ nonModule.deps = *r5;
+
+ if (bytesRead == bytesProcessed)
+ {
+ return nonModule;
+ }
+ }
+ else
+ {
+ static_assert(false && "Unknown type\n");
+ }
+
+ if (bytesRead != bytesProcessed)
+ {
+ return tl::unexpected(getErrorString(bytesRead, bytesProcessed));
+ }
+}
+[[nodiscard]] tl::expected<IPCManagerCompiler, string> makeIPCManagerCompiler(string BMIIfHeaderUnitObjOtherwisePath);
+} // namespace N2978
+#endif // IPC_MANAGER_COMPILER_HPP
diff --git a/clang/include/clang/IPC2978/Manager.hpp b/clang/include/clang/IPC2978/Manager.hpp
new file mode 100644
index 0000000000000..580e1878b3ef3
--- /dev/null
+++ b/clang/include/clang/IPC2978/Manager.hpp
@@ -0,0 +1,135 @@
+
+#ifndef MANAGER_HPP
+#define MANAGER_HPP
+
+#include "clang/IPC2978/Messages.hpp"
+#include "clang/IPC2978/expected.hpp"
+
+#include <string>
+#include <vector>
+
+using std::string, std::vector, std::string_view;
+
+#define BUFFERSIZE 4096
+
+#ifdef _WIN32
+// The following variable is used in CreateNamedFunction.
+#define PIPE_TIMEOUT 5000
+#endif
+
+namespace tl
+{
+template <typename T, typename U> class expected;
+}
+
+namespace N2978
+{
+
+enum class ErrorCategory : uint8_t
+{
+ NONE,
+
+ // error-category for API errors
+ READ_FILE_ZERO_BYTES_READ,
+ INCORRECT_BTC_LAST_MESSAGE,
+ UNKNOWN_CTB_TYPE,
+};
+
+string getErrorString();
+string getErrorString(uint32_t bytesRead_, uint32_t bytesProcessed_);
+string getErrorString(ErrorCategory errorCategory_);
+// to facilitate error propagation.
+inline string getErrorString(string err)
+{
+ return err;
+}
+
+struct ProcessMappingOfBMIFile
+{
+ string_view file;
+#ifdef _WIN32
+ void *mapping;
+ void *view;
+#else
+ void *mapping;
+ uint32_t mappingSize;
+#endif
+};
+
+class Manager
+{
+ public:
+#ifdef _WIN32
+ void *hPipe = nullptr;
+#else
+ int fdSocket = 0;
+#endif
+
+ tl::expected<uint32_t, string> readInternal(char (&buffer)[BUFFERSIZE]) const;
+ tl::expected<void, string> writeInternal(const vector<char> &buffer) const;
+
+ static vector<char> getBufferWithType(CTB type);
+ static void writeUInt32(vector<char> &buffer, uint32_t value);
+ static void writeString(vector<char> &buffer, const string &str);
+ static void writeProcessMappingOfBMIFile(vector<char> &buffer, const BMIFile &file);
+ static void writeModuleDep(vector<char> &buffer, const ModuleDep &dep);
+ static void writeHuDep(vector<char> &buffer, const HuDep &dep);
+ static void writeVectorOfStrings(vector<char> &buffer, const vector<string> &strs);
+ static void writeVectorOfProcessMappingOfBMIFiles(vector<char> &buffer, const vector<BMIFile> &files);
+ static void writeVectorOfModuleDep(vector<char> &buffer, const vector<ModuleDep> &deps);
+ static void writeVectorOfHuDep(vector<char> &buffer, const vector<HuDep> &deps);
+
+ tl::expected<bool, string> readBoolFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<uint32_t, string> readUInt32FromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<string, string> readStringFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<BMIFile, string> readProcessMappingOfBMIFileFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<vector<string>, string> readVectorOfStringFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<ModuleDep, string> readModuleDepFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<vector<ModuleDep>, string> readVectorOfModuleDepFromPipe(char (&buffer)[BUFFERSIZE],
+ uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<HuDep, string> readHuDepFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<vector<HuDep>, string> readVectorOfHuDepFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<void, string> readNumberOfBytes(char *output, uint32_t size, char (&buffer)[BUFFERSIZE],
+ uint32_t &bytesRead, uint32_t &bytesProcessed) const;
+};
+
+template <typename T, typename... Args> constexpr T *construct_at(T *p, Args &&...args)
+{
+ return ::new (static_cast<void *>(p)) T(std::forward<Args>(args)...);
+}
+
+template <typename T> T &getInitializedObjectFromBuffer(char (&buffer)[320])
+{
+ T &t = reinterpret_cast<T &>(buffer);
+ construct_at(&t);
+ return t;
+}
+
+inline std::string to16charHexString(const uint64_t v)
+{
+ static auto lut = "0123456789abcdef";
+ std::string out;
+ out.resize(16);
+ for (int i = 0; i < 8; ++i)
+ {
+ // extract byte in big-endian order:
+ const auto byte = static_cast<uint8_t>(v >> ((7 - i) * 8));
+ // high nibble:
+ out[2 * i] = lut[byte >> 4];
+ // low nibble:
+ out[2 * i + 1] = lut[byte & 0xF];
+ }
+ return out;
+}
+
+} // namespace N2978
+#endif // MANAGER_HPP
diff --git a/clang/include/clang/IPC2978/Messages.hpp b/clang/include/clang/IPC2978/Messages.hpp
new file mode 100644
index 0000000000000..c575e2b3fe09d
--- /dev/null
+++ b/clang/include/clang/IPC2978/Messages.hpp
@@ -0,0 +1,122 @@
+#ifndef MESSAGES_HPP
+#define MESSAGES_HPP
+
+#include <cstdint>
+#include <signal.h>
+#include <string>
+#include <vector>
+
+using std::string, std::vector;
+
+namespace N2978
+{
+
+// CTB --> Compiler to Build-System
+// BTC --> Build-System to Compiler
+
+// string is 4 bytes that hold the size of the char array, followed by the array.
+// vector is 4 bytes that hold the size of the array, followed by the array.
+// All fields are sent in declaration order, even if meaningless.
+
+// Compiler to Build System
+// This is the first byte of the compiler to build-system message.
+enum class CTB : uint8_t
+{
+ MODULE = 0,
+ NON_MODULE = 1,
+ LAST_MESSAGE = 2,
+};
+
+// This is sent when the compiler needs a module.
+struct CTBModule
+{
+ string moduleName;
+};
+
+// This is sent when the compiler needs something else than a module.
+// isHeaderUnit is set when the compiler knows that it is a header-unit.
+// If findInclude flag is provided, then the compiler sends logicalName,
+// Otherwise the compiler sends the full path.
+struct CTBNonModule
+{
+ bool isHeaderUnit = false;
+ string str;
+};
+
+// This is the last message sent by the compiler.
+struct CTBLastMessage
+{
+ // Whether the compilation succeeded or failed.
+ bool exitStatus = false;
+ // Following fields are meaningless if the compilation failed.
+ // header-includes discovered during compilation.
+ vector<string> headerFiles;
+ // compiler output
+ string output;
+ // compiler error output.
+ // Any IPC related error output should be reported on stderr.
+ string errorOutput;
+ // exported module name if any.
+ string logicalName;
+ // This is communicated because the receiving process has no
+ // way to learn the shared memory file size on both Windows
+ // and Linux without a filesystem call.
+ // Meaningless if the file compiled is not a module interface unit
+ // or a header-unit.
+ uint32_t fileSize = UINT32_MAX;
+};
+
+// Build System to Compiler
+// Unlike CTB, this is not written as the first byte
+// since the compiler knows what message it will receive.
+enum class BTC : uint8_t
+{
+ MODULE = 0,
+ NON_MODULE = 1,
+ LAST_MESSAGE = 2,
+};
+
+struct BMIFile
+{
+ string filePath;
+ uint32_t fileSize = UINT32_MAX;
+};
+
+struct ModuleDep
+{
+ BMIFile file;
+ string logicalName;
+};
+
+// Reply for CTBModule
+struct BTCModule
+{
+ BMIFile requested;
+ vector<ModuleDep> deps;
+};
+
+struct HuDep
+{
+ BMIFile file;
+ string logicalName;
+ bool angled = false;
+};
+
+// Reply for CTBNonModule
+struct BTCNonModule
+{
+ bool isHeaderUnit = false;
+ string filePath;
+ // if isHeaderUnit == false, the following three are meaning-less.
+ bool angled = false;
+ // if isHeaderUnit == true, fileSize of the requested file.
+ uint32_t fileSize;
+ vector<HuDep> deps;
+};
+
+// Reply for CTBLastMessage if the compilation succeeded.
+struct BTCLastMessage
+{
+};
+} // namespace N2978
+#endif // MESSAGES_HPP
diff --git a/clang/include/clang/IPC2978/expected.hpp b/clang/include/clang/IPC2978/expected.hpp
new file mode 100644
index 0000000000000..1f92b6b3cd85a
--- /dev/null
+++ b/clang/include/clang/IPC2978/expected.hpp
@@ -0,0 +1,2444 @@
+///
+// expected - An implementation of std::expected with extensions
+// Written in 2017 by Sy Brand (tartanllama@gmail.com, @TartanLlama)
+//
+// Documentation available at http://tl.tartanllama.xyz/
+//
+// To the extent possible under law, the author(s) have dedicated all
+// copyright and related and neighboring rights to this software to the
+// public domain worldwide. This software is distributed without any warranty.
+//
+// You should have received a copy of the CC0 Public Domain Dedication
+// along with this software. If not, see
+// <http://creativecommons.org/publicdomain/zero/1.0/>.
+///
+
+#ifndef TL_EXPECTED_HPP
+#define TL_EXPECTED_HPP
+
+#define TL_EXPECTED_VERSION_MAJOR 1
+#define TL_EXPECTED_VERSION_MINOR 1
+#define TL_EXPECTED_VERSION_PATCH 0
+
+#include <exception>
+#include <functional>
+#include <type_traits>
+#include <utility>
+
+#if defined(__EXCEPTIONS) || defined(_CPPUNWIND)
+#define TL_EXPECTED_EXCEPTIONS_ENABLED
+#endif
+
+#if (defined(_MSC_VER) && _MSC_VER == 1900)
+#define TL_EXPECTED_MSVC2015
+#define TL_EXPECTED_MSVC2015_CONSTEXPR
+#else
+#define TL_EXPECTED_MSVC2015_CONSTEXPR constexpr
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 4 && __GNUC_MINOR__ <= 9 && \
+ !defined(__clang__))
+#define TL_EXPECTED_GCC49
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 5 && __GNUC_MINOR__ <= 4 && \
+ !defined(__clang__))
+#define TL_EXPECTED_GCC54
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 5 && __GNUC_MINOR__ <= 5 && \
+ !defined(__clang__))
+#define TL_EXPECTED_GCC55
+#endif
+
+#if !defined(TL_ASSERT)
+//can't have assert in constexpr in C++11 and GCC 4.9 has a compiler bug
+#if (__cplusplus > 201103L) && !defined(TL_EXPECTED_GCC49)
+#include <cassert>
+#define TL_ASSERT(x) assert(x)
+#else
+#define TL_ASSERT(x)
+#endif
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 4 && __GNUC_MINOR__ <= 9 && \
+ !defined(__clang__))
+// GCC < 5 doesn't support overloading on const&& for member functions
+
+#define TL_EXPECTED_NO_CONSTRR
+// GCC < 5 doesn't support some standard C++11 type traits
+#define TL_EXPECTED_IS_TRIVIALLY_COPY_CONSTRUCTIBLE(T) \
+ std::has_trivial_copy_constructor<T>
+#define TL_EXPECTED_IS_TRIVIALLY_COPY_ASSIGNABLE(T) \
+ std::has_trivial_copy_assign<T>
+
+// This one will be different for GCC 5.7 if it's ever supported
+#define TL_EXPECTED_IS_TRIVIALLY_DESTRUCTIBLE(T) \
+ std::is_trivially_destructible<T>
+
+// GCC 5 < v < 8 has a bug in is_trivially_copy_constructible which breaks
+// std::vector for non-copyable types
+#elif (defined(__GNUC__) && __GNUC__ < 8 && !defined(__clang__))
+#ifndef TL_GCC_LESS_8_TRIVIALLY_COPY_CONSTRUCTIBLE_MUTEX
+#define TL_GCC_LESS_8_TRIVIALLY_COPY_CONSTRUCTIBLE_MUTEX
+namespace tl {
+namespace detail {
+template <class T>
+struct is_trivially_copy_constructible
+ : std::is_trivially_copy_constructible<T> {};
+#ifdef _GLIBCXX_VECTOR
+template <class T, class A>
+struct is_trivially_copy_constructible<std::vector<T, A>> : std::false_type {};
+#endif
+} // namespace detail
+} // namespace tl
+#endif
+
+#define TL_EXPECTED_IS_TRIVIALLY_COPY_CONSTRUCTIBLE(T) \
+ tl::detail::is_trivially_copy_constructible<T>
+#define TL_EXPECT...
[truncated]
|
|
@llvm/pr-subscribers-clang Author: Hassan Sajjad (HassanSajjad-302) ChangesThis WIP pull request adds partial support for my draft N2978. In my paper, I have referred to a sample implementation of the paper, In it, I define a type // IPCManagerCompiler class functions
[[nodiscard]] tl::expected<BTCModule, string> receiveBTCModule(const CTBModule &moduleName) const;
[[nodiscard]] tl::expected<BTCNonModule, string> receiveBTCNonModule(const CTBNonModule &nonModule) const;
[[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage) const;
[[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage,
const string &bmiFile, const string &filePath) const;
static tl::expected<ProcessMappingOfBMIFile, string> readSharedMemoryBMIFile(const BMIFile &file);
static tl::expected<void, string> closeBMIFileMapping(const ProcessMappingOfBMIFile &processMappingOfBMIFile);
void closeConnection() const;
// A global function
[[nodiscard]] tl::expected<IPCManagerCompiler, string> makeIPCManagerCompiler(string BMIIfHeaderUnitObjOtherwisePath);
Of these seven functions, It invokes the compilation of a
I opened this half-baked pull-request to request assistance and feedback HMake will also support Clang's 2-phase compilation with this approach. No other build-system can support C++20 modules the way HMake can. Due to these reasons, in the future, I would like to propose it for LLVM as well. @vgvassilev @Bigcheese Patch is 183.41 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/147682.diff 25 Files Affected:
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 77379f1130149..6a6010b4baa44 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -9416,3 +9416,13 @@ def wasm_opt : Flag<["--"], "wasm-opt">,
Group<m_Group>,
HelpText<"Enable the wasm-opt optimizer (default)">,
MarshallingInfoNegativeFlag<LangOpts<"NoWasmOpt">>;
+
+def no_scan_ipc : Flag<["-"], "noScanIPC">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Enable No scan IPC approach">;
+def translate_include : Flag<["-"], "translateInclude">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Consider header includes as header-units">;
+def find_include : Flag<["-"], "findIncludes">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"In No Scan IPC approach, compiler will relay the finding includes responsibility to the build-system">;
diff --git a/clang/include/clang/Frontend/CompilerInstance.h b/clang/include/clang/Frontend/CompilerInstance.h
index 0ae490f0e8073..84a524a549e02 100644
--- a/clang/include/clang/Frontend/CompilerInstance.h
+++ b/clang/include/clang/Frontend/CompilerInstance.h
@@ -16,6 +16,7 @@
#include "clang/Frontend/CompilerInvocation.h"
#include "clang/Frontend/PCHContainerOperations.h"
#include "clang/Frontend/Utils.h"
+#include "clang/IPC2978/IPCManagerCompiler.hpp"
#include "clang/Lex/DependencyDirectivesScanner.h"
#include "clang/Lex/HeaderSearchOptions.h"
#include "clang/Lex/ModuleLoader.h"
@@ -180,6 +181,10 @@ class CompilerInstance : public ModuleLoader {
/// The stream for verbose output.
raw_ostream *VerboseOutputStream = &llvm::errs();
+ /// Pointer for managing communication with build-system if noScan flag is
+ /// set.
+ N2978::IPCManagerCompiler *ipcManager = nullptr;
+
/// Holds information about the output file.
///
/// If TempFilename is not empty we must rename it to Filename at the end.
diff --git a/clang/include/clang/IPC2978/.clang-format-ignore b/clang/include/clang/IPC2978/.clang-format-ignore
new file mode 100644
index 0000000000000..f59ec20aabf58
--- /dev/null
+++ b/clang/include/clang/IPC2978/.clang-format-ignore
@@ -0,0 +1 @@
+*
\ No newline at end of file
diff --git a/clang/include/clang/IPC2978/IPCManagerBS.hpp b/clang/include/clang/IPC2978/IPCManagerBS.hpp
new file mode 100644
index 0000000000000..f9222a55f3fc4
--- /dev/null
+++ b/clang/include/clang/IPC2978/IPCManagerBS.hpp
@@ -0,0 +1,39 @@
+
+#ifndef IPC_MANAGER_BS_HPP
+#define IPC_MANAGER_BS_HPP
+
+#include "clang/IPC2978/Manager.hpp"
+#include "clang/IPC2978/Messages.hpp"
+
+namespace N2978
+{
+
+// IPC Manager BuildSystem
+class IPCManagerBS : Manager
+{
+ friend tl::expected<IPCManagerBS, string> makeIPCManagerBS(string BMIIfHeaderUnitObjOtherwisePath);
+ bool connectedToCompiler = false;
+
+#ifdef _WIN32
+ explicit IPCManagerBS(void *hPipe_);
+#else
+ explicit IPCManagerBS(int fdSocket_);
+#endif
+
+ public:
+ IPCManagerBS(const IPCManagerBS &) = default;
+ IPCManagerBS &operator=(const IPCManagerBS &) = default;
+ IPCManagerBS(IPCManagerBS &&) = default;
+ IPCManagerBS &operator=(IPCManagerBS &&) = default;
+ tl::expected<void, string> receiveMessage(char (&ctbBuffer)[320], CTB &messageType) const;
+ [[nodiscard]] tl::expected<void, string> sendMessage(const BTCModule &moduleFile) const;
+ [[nodiscard]] tl::expected<void, string> sendMessage(const BTCNonModule &nonModule) const;
+ [[nodiscard]] tl::expected<void, string> sendMessage(const BTCLastMessage &lastMessage) const;
+ static tl::expected<ProcessMappingOfBMIFile, string> createSharedMemoryBMIFile(const BMIFile &bmiFile);
+ static tl::expected<void, string> closeBMIFileMapping(const ProcessMappingOfBMIFile &processMappingOfBMIFile);
+ void closeConnection() const;
+};
+
+tl::expected<IPCManagerBS, string> makeIPCManagerBS(string BMIIfHeaderUnitObjOtherwisePath);
+} // namespace N2978
+#endif // IPC_MANAGER_BS_HPP
diff --git a/clang/include/clang/IPC2978/IPCManagerCompiler.hpp b/clang/include/clang/IPC2978/IPCManagerCompiler.hpp
new file mode 100644
index 0000000000000..6aba39822d5ac
--- /dev/null
+++ b/clang/include/clang/IPC2978/IPCManagerCompiler.hpp
@@ -0,0 +1,129 @@
+
+#ifndef IPC_MANAGER_COMPILER_HPP
+#define IPC_MANAGER_COMPILER_HPP
+
+#include "clang/IPC2978/Manager.hpp"
+#include "clang/IPC2978/expected.hpp"
+
+using std::string_view;
+namespace N2978
+{
+
+// IPC Manager Compiler
+class IPCManagerCompiler : Manager
+{
+ template <typename T> tl::expected<T, string> receiveMessage() const;
+ // This is not exposed. sendCTBLastMessage calls this.
+ [[nodiscard]] tl::expected<void, string> receiveBTCLastMessage() const;
+
+ public:
+#ifdef _WIN32
+ explicit IPCManagerCompiler(void *hPipe_);
+#else
+ explicit IPCManagerCompiler(int fdSocket_);
+#endif
+ [[nodiscard]] tl::expected<BTCModule, string> receiveBTCModule(const CTBModule &moduleName) const;
+ [[nodiscard]] tl::expected<BTCNonModule, string> receiveBTCNonModule(const CTBNonModule &nonModule) const;
+ [[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage) const;
+ [[nodiscard]] tl::expected<void, string> sendCTBLastMessage(const CTBLastMessage &lastMessage,
+ const string &bmiFile, const string &filePath) const;
+ static tl::expected<ProcessMappingOfBMIFile, string> readSharedMemoryBMIFile(const BMIFile &file);
+ static tl::expected<void, string> closeBMIFileMapping(const ProcessMappingOfBMIFile &processMappingOfBMIFile);
+ void closeConnection() const;
+};
+
+template <typename T> tl::expected<T, string> IPCManagerCompiler::receiveMessage() const
+{
+ // Read from the pipe.
+ char buffer[BUFFERSIZE];
+ uint32_t bytesRead;
+ if (const auto &r = readInternal(buffer); !r)
+ {
+ return tl::unexpected(r.error());
+ }
+ else
+ {
+ bytesRead = *r;
+ }
+
+ uint32_t bytesProcessed = 0;
+
+ if constexpr (std::is_same_v<T, BTCModule>)
+ {
+ const auto &r = readProcessMappingOfBMIFileFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r)
+ {
+ return tl::unexpected(r.error());
+ }
+
+ const auto &r2 = readVectorOfModuleDepFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r2)
+ {
+ return tl::unexpected(r2.error());
+ }
+
+ BTCModule moduleFile;
+ moduleFile.requested = *r;
+ moduleFile.deps = *r2;
+ if (bytesRead == bytesProcessed)
+ {
+ return moduleFile;
+ }
+ }
+ else if constexpr (std::is_same_v<T, BTCNonModule>)
+ {
+ const auto &r = readBoolFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r)
+ {
+ return tl::unexpected(r.error());
+ }
+
+ const auto &r2 = readStringFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r2)
+ {
+ return tl::unexpected(r2.error());
+ }
+
+ const auto &r3 = readBoolFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r3)
+ {
+ return tl::unexpected(r3.error());
+ }
+
+ const auto &r4 = readUInt32FromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r4)
+ {
+ return tl::unexpected(r4.error());
+ }
+
+ const auto &r5 = readVectorOfHuDepFromPipe(buffer, bytesRead, bytesProcessed);
+ if (!r5)
+ {
+ return tl::unexpected(r5.error());
+ }
+
+ BTCNonModule nonModule;
+ nonModule.isHeaderUnit = *r;
+ nonModule.filePath = *r2;
+ nonModule.angled = *r3;
+ nonModule.fileSize = *r4;
+ nonModule.deps = *r5;
+
+ if (bytesRead == bytesProcessed)
+ {
+ return nonModule;
+ }
+ }
+ else
+ {
+ static_assert(false && "Unknown type\n");
+ }
+
+ if (bytesRead != bytesProcessed)
+ {
+ return tl::unexpected(getErrorString(bytesRead, bytesProcessed));
+ }
+}
+[[nodiscard]] tl::expected<IPCManagerCompiler, string> makeIPCManagerCompiler(string BMIIfHeaderUnitObjOtherwisePath);
+} // namespace N2978
+#endif // IPC_MANAGER_COMPILER_HPP
diff --git a/clang/include/clang/IPC2978/Manager.hpp b/clang/include/clang/IPC2978/Manager.hpp
new file mode 100644
index 0000000000000..580e1878b3ef3
--- /dev/null
+++ b/clang/include/clang/IPC2978/Manager.hpp
@@ -0,0 +1,135 @@
+
+#ifndef MANAGER_HPP
+#define MANAGER_HPP
+
+#include "clang/IPC2978/Messages.hpp"
+#include "clang/IPC2978/expected.hpp"
+
+#include <string>
+#include <vector>
+
+using std::string, std::vector, std::string_view;
+
+#define BUFFERSIZE 4096
+
+#ifdef _WIN32
+// The following variable is used in CreateNamedFunction.
+#define PIPE_TIMEOUT 5000
+#endif
+
+namespace tl
+{
+template <typename T, typename U> class expected;
+}
+
+namespace N2978
+{
+
+enum class ErrorCategory : uint8_t
+{
+ NONE,
+
+ // error-category for API errors
+ READ_FILE_ZERO_BYTES_READ,
+ INCORRECT_BTC_LAST_MESSAGE,
+ UNKNOWN_CTB_TYPE,
+};
+
+string getErrorString();
+string getErrorString(uint32_t bytesRead_, uint32_t bytesProcessed_);
+string getErrorString(ErrorCategory errorCategory_);
+// to facilitate error propagation.
+inline string getErrorString(string err)
+{
+ return err;
+}
+
+struct ProcessMappingOfBMIFile
+{
+ string_view file;
+#ifdef _WIN32
+ void *mapping;
+ void *view;
+#else
+ void *mapping;
+ uint32_t mappingSize;
+#endif
+};
+
+class Manager
+{
+ public:
+#ifdef _WIN32
+ void *hPipe = nullptr;
+#else
+ int fdSocket = 0;
+#endif
+
+ tl::expected<uint32_t, string> readInternal(char (&buffer)[BUFFERSIZE]) const;
+ tl::expected<void, string> writeInternal(const vector<char> &buffer) const;
+
+ static vector<char> getBufferWithType(CTB type);
+ static void writeUInt32(vector<char> &buffer, uint32_t value);
+ static void writeString(vector<char> &buffer, const string &str);
+ static void writeProcessMappingOfBMIFile(vector<char> &buffer, const BMIFile &file);
+ static void writeModuleDep(vector<char> &buffer, const ModuleDep &dep);
+ static void writeHuDep(vector<char> &buffer, const HuDep &dep);
+ static void writeVectorOfStrings(vector<char> &buffer, const vector<string> &strs);
+ static void writeVectorOfProcessMappingOfBMIFiles(vector<char> &buffer, const vector<BMIFile> &files);
+ static void writeVectorOfModuleDep(vector<char> &buffer, const vector<ModuleDep> &deps);
+ static void writeVectorOfHuDep(vector<char> &buffer, const vector<HuDep> &deps);
+
+ tl::expected<bool, string> readBoolFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<uint32_t, string> readUInt32FromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<string, string> readStringFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<BMIFile, string> readProcessMappingOfBMIFileFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<vector<string>, string> readVectorOfStringFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<ModuleDep, string> readModuleDepFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<vector<ModuleDep>, string> readVectorOfModuleDepFromPipe(char (&buffer)[BUFFERSIZE],
+ uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<HuDep, string> readHuDepFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<vector<HuDep>, string> readVectorOfHuDepFromPipe(char (&buffer)[BUFFERSIZE], uint32_t &bytesRead,
+ uint32_t &bytesProcessed) const;
+ tl::expected<void, string> readNumberOfBytes(char *output, uint32_t size, char (&buffer)[BUFFERSIZE],
+ uint32_t &bytesRead, uint32_t &bytesProcessed) const;
+};
+
+template <typename T, typename... Args> constexpr T *construct_at(T *p, Args &&...args)
+{
+ return ::new (static_cast<void *>(p)) T(std::forward<Args>(args)...);
+}
+
+template <typename T> T &getInitializedObjectFromBuffer(char (&buffer)[320])
+{
+ T &t = reinterpret_cast<T &>(buffer);
+ construct_at(&t);
+ return t;
+}
+
+inline std::string to16charHexString(const uint64_t v)
+{
+ static auto lut = "0123456789abcdef";
+ std::string out;
+ out.resize(16);
+ for (int i = 0; i < 8; ++i)
+ {
+ // extract byte in big-endian order:
+ const auto byte = static_cast<uint8_t>(v >> ((7 - i) * 8));
+ // high nibble:
+ out[2 * i] = lut[byte >> 4];
+ // low nibble:
+ out[2 * i + 1] = lut[byte & 0xF];
+ }
+ return out;
+}
+
+} // namespace N2978
+#endif // MANAGER_HPP
diff --git a/clang/include/clang/IPC2978/Messages.hpp b/clang/include/clang/IPC2978/Messages.hpp
new file mode 100644
index 0000000000000..c575e2b3fe09d
--- /dev/null
+++ b/clang/include/clang/IPC2978/Messages.hpp
@@ -0,0 +1,122 @@
+#ifndef MESSAGES_HPP
+#define MESSAGES_HPP
+
+#include <cstdint>
+#include <signal.h>
+#include <string>
+#include <vector>
+
+using std::string, std::vector;
+
+namespace N2978
+{
+
+// CTB --> Compiler to Build-System
+// BTC --> Build-System to Compiler
+
+// string is 4 bytes that hold the size of the char array, followed by the array.
+// vector is 4 bytes that hold the size of the array, followed by the array.
+// All fields are sent in declaration order, even if meaningless.
+
+// Compiler to Build System
+// This is the first byte of the compiler to build-system message.
+enum class CTB : uint8_t
+{
+ MODULE = 0,
+ NON_MODULE = 1,
+ LAST_MESSAGE = 2,
+};
+
+// This is sent when the compiler needs a module.
+struct CTBModule
+{
+ string moduleName;
+};
+
+// This is sent when the compiler needs something else than a module.
+// isHeaderUnit is set when the compiler knows that it is a header-unit.
+// If findInclude flag is provided, then the compiler sends logicalName,
+// Otherwise the compiler sends the full path.
+struct CTBNonModule
+{
+ bool isHeaderUnit = false;
+ string str;
+};
+
+// This is the last message sent by the compiler.
+struct CTBLastMessage
+{
+ // Whether the compilation succeeded or failed.
+ bool exitStatus = false;
+ // Following fields are meaningless if the compilation failed.
+ // header-includes discovered during compilation.
+ vector<string> headerFiles;
+ // compiler output
+ string output;
+ // compiler error output.
+ // Any IPC related error output should be reported on stderr.
+ string errorOutput;
+ // exported module name if any.
+ string logicalName;
+ // This is communicated because the receiving process has no
+ // way to learn the shared memory file size on both Windows
+ // and Linux without a filesystem call.
+ // Meaningless if the file compiled is not a module interface unit
+ // or a header-unit.
+ uint32_t fileSize = UINT32_MAX;
+};
+
+// Build System to Compiler
+// Unlike CTB, this is not written as the first byte
+// since the compiler knows what message it will receive.
+enum class BTC : uint8_t
+{
+ MODULE = 0,
+ NON_MODULE = 1,
+ LAST_MESSAGE = 2,
+};
+
+struct BMIFile
+{
+ string filePath;
+ uint32_t fileSize = UINT32_MAX;
+};
+
+struct ModuleDep
+{
+ BMIFile file;
+ string logicalName;
+};
+
+// Reply for CTBModule
+struct BTCModule
+{
+ BMIFile requested;
+ vector<ModuleDep> deps;
+};
+
+struct HuDep
+{
+ BMIFile file;
+ string logicalName;
+ bool angled = false;
+};
+
+// Reply for CTBNonModule
+struct BTCNonModule
+{
+ bool isHeaderUnit = false;
+ string filePath;
+ // if isHeaderUnit == false, the following three are meaning-less.
+ bool angled = false;
+ // if isHeaderUnit == true, fileSize of the requested file.
+ uint32_t fileSize;
+ vector<HuDep> deps;
+};
+
+// Reply for CTBLastMessage if the compilation succeeded.
+struct BTCLastMessage
+{
+};
+} // namespace N2978
+#endif // MESSAGES_HPP
diff --git a/clang/include/clang/IPC2978/expected.hpp b/clang/include/clang/IPC2978/expected.hpp
new file mode 100644
index 0000000000000..1f92b6b3cd85a
--- /dev/null
+++ b/clang/include/clang/IPC2978/expected.hpp
@@ -0,0 +1,2444 @@
+///
+// expected - An implementation of std::expected with extensions
+// Written in 2017 by Sy Brand (tartanllama@gmail.com, @TartanLlama)
+//
+// Documentation available at http://tl.tartanllama.xyz/
+//
+// To the extent possible under law, the author(s) have dedicated all
+// copyright and related and neighboring rights to this software to the
+// public domain worldwide. This software is distributed without any warranty.
+//
+// You should have received a copy of the CC0 Public Domain Dedication
+// along with this software. If not, see
+// <http://creativecommons.org/publicdomain/zero/1.0/>.
+///
+
+#ifndef TL_EXPECTED_HPP
+#define TL_EXPECTED_HPP
+
+#define TL_EXPECTED_VERSION_MAJOR 1
+#define TL_EXPECTED_VERSION_MINOR 1
+#define TL_EXPECTED_VERSION_PATCH 0
+
+#include <exception>
+#include <functional>
+#include <type_traits>
+#include <utility>
+
+#if defined(__EXCEPTIONS) || defined(_CPPUNWIND)
+#define TL_EXPECTED_EXCEPTIONS_ENABLED
+#endif
+
+#if (defined(_MSC_VER) && _MSC_VER == 1900)
+#define TL_EXPECTED_MSVC2015
+#define TL_EXPECTED_MSVC2015_CONSTEXPR
+#else
+#define TL_EXPECTED_MSVC2015_CONSTEXPR constexpr
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 4 && __GNUC_MINOR__ <= 9 && \
+ !defined(__clang__))
+#define TL_EXPECTED_GCC49
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 5 && __GNUC_MINOR__ <= 4 && \
+ !defined(__clang__))
+#define TL_EXPECTED_GCC54
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 5 && __GNUC_MINOR__ <= 5 && \
+ !defined(__clang__))
+#define TL_EXPECTED_GCC55
+#endif
+
+#if !defined(TL_ASSERT)
+//can't have assert in constexpr in C++11 and GCC 4.9 has a compiler bug
+#if (__cplusplus > 201103L) && !defined(TL_EXPECTED_GCC49)
+#include <cassert>
+#define TL_ASSERT(x) assert(x)
+#else
+#define TL_ASSERT(x)
+#endif
+#endif
+
+#if (defined(__GNUC__) && __GNUC__ == 4 && __GNUC_MINOR__ <= 9 && \
+ !defined(__clang__))
+// GCC < 5 doesn't support overloading on const&& for member functions
+
+#define TL_EXPECTED_NO_CONSTRR
+// GCC < 5 doesn't support some standard C++11 type traits
+#define TL_EXPECTED_IS_TRIVIALLY_COPY_CONSTRUCTIBLE(T) \
+ std::has_trivial_copy_constructor<T>
+#define TL_EXPECTED_IS_TRIVIALLY_COPY_ASSIGNABLE(T) \
+ std::has_trivial_copy_assign<T>
+
+// This one will be different for GCC 5.7 if it's ever supported
+#define TL_EXPECTED_IS_TRIVIALLY_DESTRUCTIBLE(T) \
+ std::is_trivially_destructible<T>
+
+// GCC 5 < v < 8 has a bug in is_trivially_copy_constructible which breaks
+// std::vector for non-copyable types
+#elif (defined(__GNUC__) && __GNUC__ < 8 && !defined(__clang__))
+#ifndef TL_GCC_LESS_8_TRIVIALLY_COPY_CONSTRUCTIBLE_MUTEX
+#define TL_GCC_LESS_8_TRIVIALLY_COPY_CONSTRUCTIBLE_MUTEX
+namespace tl {
+namespace detail {
+template <class T>
+struct is_trivially_copy_constructible
+ : std::is_trivially_copy_constructible<T> {};
+#ifdef _GLIBCXX_VECTOR
+template <class T, class A>
+struct is_trivially_copy_constructible<std::vector<T, A>> : std::false_type {};
+#endif
+} // namespace detail
+} // namespace tl
+#endif
+
+#define TL_EXPECTED_IS_TRIVIALLY_COPY_CONSTRUCTIBLE(T) \
+ tl::detail::is_trivially_copy_constructible<T>
+#define TL_EXPECT...
[truncated]
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This WIP pull request adds partial support for my draft N2978.
This paper allows for building C++20 modules and header-units
without the need to scan them first.
This allows for improved compilation support.
In my paper, I have referred to a sample implementation of the paper,
a library ipc2978api,
that compiler and build-systems can use to achieve
non-scanning builds.
ipc2978api has a complete cross-platform implementation of my paper
with error-handling.
The code coverage is 100%.
In it, I define a type
IPCManagerCompilerthat the compilercan use to manage the interaction with the build-system.
It has seven public methods that are to be used
by the compiler.
tl::expectedis of this library.It is basically
std::expectedfor C++17 which is used for LLVM.While
CTBandBTCmean compiler to build-system andbuild-system to compiler respectively.
These are used to designate the message-type in the communication.
This is mentioned in my paper.
These include fields like exit-status, header-files, output string, logical-name
(If the module-compilation had any) and file output.
This will create a shared memory file.
So other processes don't have to read from the disk.
ProcessMappingOfBMIFilethat can be passed to this function to close the mapping.
Calling Six and Seven isn't necessary since OS closes descriptors on process exit.
Calling
closeBMIFileMappingwill not close the mapping even if it is the onlycompilation currently referencing it.
Since build-system will have it open anyway.
Build-system will close its mapping once all compilations that
might require the mapping have concluded.
makeIPCManagerCompilerfunction can be used to createIPCManagerCompiler.PCM file-path is to be passed if header-unit is being compiled.
And object file-path is to be passed otherwise.
The build-system is required to pass the requisite options to the compiler.
Of these seven functions,
I have added support for the first one in this pull-request.
This does few modifications to the source-code.
Instead, it compiles the ipc2978api as part of
the Clang source code.
It has a
ClangTest.cppwhich is copied to theLLVM repo as the unit test.
This WIP test will test the Clang support for my paper.
Currently, it only tests the first function.
It invokes the compilation of a
main.cppwithnoScanIPCflag.This
main.cppfile depends onmod.cppm,mod1.cppm, andmod2.cppm.These modules are already compiled and are passed to the compiler
by the test using
IPCManagerBS.After compilation completion, it tests for the existence of the
main.owhich marks thesuccessful completion of the test.
Test automatically executes in
build-dir/bindirectorywhere the clang binary exists.
clang/IPC2978/lib/setup.pycopies the test, source-filesand header-files from ipc2978api to the LLVM repo.
ipc2978api and LLVM repo should be in the same directory.
setup.pyadjusts the includes in ipc2978api source-files to point toipc2978api headers in the LLVM repo.
This also uncomments a definition in
ClangTest.cppthat allows itto integrate with gtest.
I opened this half-baked pull-request to request assistance and feedback
to expedite the completion.
Once this is completed, I expect to very quickly add support for this
in my software
HMake.
And then compile Boost with Big header-units without scanning.
Big header-units will be my software feature built on top of this.
It will allow amalgamating all header-files in a directory in a
single heder-unit.
I already compiled 25 Boost libraries with header-units with scanning with MSVC.
HMake will also support Clang's 2-phase compilation with this approach.
Compilation will share the Fat PCM which will be passed to other
compilations if free threads are available.
And this compilation will be signaled to produce the object-file.
So only one process is needed instead of three (scanning, Fat PCM, object-file).
HMake will also support
this
proposed by @ChuanqiXu9 and
this as-well.
While ipc2978api currently does not support 2-phase compilation
and declaration hashing,
I can add this first.
It is just fill in the blanks since the challenging work of
a connection establishment, memory-mapped files, error-handling, etc
is all complete.
And can prioritize this with C++20 modules support before C++20 header-units
support in HMake.
No other build-system can support C++20 modules the way HMake can.
HMake has a next-gen build algorithm that allows for dynamic nodes and edges.
Even with all this stuff, due to its next-gen architecture,
it would still be extremely fast and memory efficient.
It can exploit missed parallelization opportunities in the LLVM build(if any).
Due to these reasons, in the future, I would like to propose it for LLVM as well.
@vgvassilev @Bigcheese