This project was a workshop with the idea to dive into the communication between processes and threads. The problem was simple: How could programs wrote in different languages communicate together on the same machine and what level of pain it is for developers?
So I took my C++ skills because this language is compatible with all others. I've spent few days finding existing libraries and testing them across Windows and Linux platforms. And as planned, no blocking issue was encountered. Just basic compilation things mainly, not even a segmentation fault 🤔.
In less than a week, a simple API was drafted and implemented. It was possible to establish a connection between 2 peers and send messages at lightning speed. And, a security system was designed based on asymmetric keys, backed by a robust solution.
At this point, the responses to the initial question were found. Having another project in mind, I decided to switch and write a new story.
[SPOILER]
The response to the initial problem: The communication between processes and threads is called inter-process communication can be achieved using a named pipe. It's supported for years on Windows and Unix systems including MacOS. The C++ implementation can be done with robust and maintained libraries like boost or libuv and the integration is very easy.
The security is not a concern as there's open source solution like libsodium. It allow developers to implement a robust security layer without prior knowledge in the security domain, except what's hashing, signatures and encryption/decryption with keys.
Finally, inter-process communication is widely used in the software industry. Especially in a scenario where programs require to react from requests or events on the same machine and as fast as possible. (See SQL Server, D-Bus)
The communication is a complex task when it involve exchanging data between processes and threads. This statement is particulary true in a many-to-many scenario. And on top of that, doing it native adds a huge gap of complexity.
The idea is to tackle the complexity by exposing a simple API inspired by the web transport technologies TCP and UDP. An extra requirement is to get a fully secured solution by using some encryption methods.
Just don't reinvent the wheel. There's plenty of existing solution backed by strong actors that can match the technical requirements.
The all-in-one solution doesn't exist and the closer one is libsharedmemory. It's cross platform and use the shared memory for communication. But it miss the security layer, has no binary stream implemented and many more lacks. So, this project will be used as a strong reference and a fallback solution if things goes wrong.
- Cross platform: Windows, MacOS, Linux.
- Cross architecture: x86, x64, arm.
- Act as a sealed box: Encrypted data from peer to peer.
| Solution | Purpose |
|---|---|
| boost | shared memory management, buffering |
| Libsodium | keys generator, encryption, decryption |
| ObjectBox | in-memory database |
The API architecture is traditional, the server is capable of receiving client connections, consume their data and respond with new data. An endpoint is defined by an identifier similarly to the host part inside a URL. There's no mechanism similar to handle a path as it would imply to implement a parameter system which is time consuming. Finally, the encryption is fully transparent for a standard user.
⚠ The following code is not functional and may contain errors. This code is not intended to be compiled.
// Instanciate a server socket.
auto server = new ServerSocket();
// Handle socket errors.
server.OnError([] (std::string error) {
// Consume the errors raised be the server socket.
});
// Handle socket data stream.
server.OnData([] (auto socket, void *data, size_t size) {
// Consume data received from server socket. This is the data sent from a client socket.
auto msg = "Hello from server";
if (socket.Write(msg.c_str(), msg.length()) != 0) {
// Handle write errors.
}
});
// Listen for client connections.
server.Listen("my-unique-socket-identifier");
// Close the socket.
server.Close();// Instanciate a client socket.
auto client = new ClientSocket();
// Handle socket errors.
client.OnError([] (std::string error) {
// Consume the errors raised be the client socket.
});
// Handle socket data stream.
client.OnData([] (auto socket, void *data, size_t size) {
// Consume data received from client socket. This is the data sent from a server socket.
auto msg = "Hello from client";
if (socket.Write(msg.c_str(), msg.length()) == 0) {
// Handle write errors.
}
});
// Connect the socket.
if (client.Connect("my-unique-socket-identifier") != 0) {
// Handle connection error.
}
std::string msg("hello memory world");
if (client->Write(msg.c_str(), msg.length()) == 0) {
// Handle connection error.
}
// Close the socket.
client.Close();- Start a devcontainer. (Building instructions: b3nNn/cpp-devcontainer).
- Open a terminal, then install
boostandlibsodium.
sudo apt-get install libsodium23 libsodium-dev libboost-all-dev- Compile the sources.
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make- (Optional) Install the library
libmemorysocketon the system.
make installA single producer - single consumer program.
- Start the consumer.
./build/example/simple-program- Start the producer with
-pargument.
./build/example/simple-program -pThe consumer's output should be similar to this:
consumed 1k messages
consumed 2k messages
consumed 3k messages
consumed 4k messages
consumed 5k messages
consumed 6k messages
consumed 7k messages
consumed 8k messages
consumed 9k messages
consumed 10k messages
[...]- Boost integration.
- Libsodium integration.
- Objectbox integration.
- Dedicated devcontainer image with dependencies included.
- Linux support.
- MacOS support.
- Windows support. (documentation coming next)
- CMake build on Windows fail.
The licence is and will always remain MIT. The goal of this project is to enable anyone to do anything he wants with the code and it's usage.