Skip to content

Add support for MAP_SHARED for Linux only #429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,13 @@ More detailed documentation is available here: https://oneapi-src.github.io/unif
#### OS memory provider

A memory provider that provides memory from an operating system.
It supports two types of memory mappings
1) private memory mapping (`UMF_MEM_MAP_PRIVATE`)
2) shared memory mapping (`UMF_MEM_MAP_SHARED` - supported on Linux only yet)

If the shared memory mapping is used then an anonymous file descriptor for memory mapping is created using:
1) `memfd_secret()` syscall - (if it is implemented and) if the `UMF_MEM_FD_FUNC` environment variable does not contain the "memfd_create" string or
2) `memfd_create()` syscall - otherwise (and if it is implemented).

##### Requirements

Expand Down Expand Up @@ -226,6 +233,10 @@ In case of Linux it can be done without any code changes using the `LD_PRELOAD`
$ LD_PRELOAD=/usr/lib/libumf_proxy.so myprogram
```

The memory used by the proxy memory allocator is mmap'ed:
1) with the `MAP_PRIVATE` flag by default or
2) with the `MAP_SHARED` flag only if the `UMF_PROXY` environment variable contains the `page.disposition=shared` string.

#### Windows

In case of Windows it requires:
Expand Down
1 change: 1 addition & 0 deletions benchmark/ubench.c
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ UBENCH_EX(simple, glibc_malloc) {

static umf_os_memory_provider_params_t UMF_OS_MEMORY_PROVIDER_PARAMS = {
/* .protection = */ UMF_PROTECTION_READ | UMF_PROTECTION_WRITE,
/* .visibility */ UMF_MEM_MAP_PRIVATE,

// NUMA config
/* .nodemask = */ NULL,
Expand Down
4 changes: 4 additions & 0 deletions include/umf/memory_provider_ops.h
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ typedef struct umf_memory_provider_ext_ops_t {
/// @brief Merges two coarse grain allocations into a single allocation that
/// can be managed (freed) as a whole.
/// allocation_split and allocation_merge should be both set or both NULL.
/// allocation_merge should NOT be called concurrently with allocation_split()
/// with the same pointer.
/// @param hProvider handle to the memory provider
/// @param lowPtr pointer to the first allocation
/// @param highPtr pointer to the second allocation (should be > lowPtr)
Expand All @@ -64,6 +66,8 @@ typedef struct umf_memory_provider_ext_ops_t {
/// @brief Splits a coarse grain allocation into 2 adjacent allocations that
/// can be managed (freed) separately.
/// allocation_split and allocation_merge should be both set or both NULL.
/// allocation_split should NOT be called concurrently with allocation_merge()
/// with the same pointer.
/// @param hProvider handle to the memory provider
/// @param ptr pointer to the beginning of the allocation
/// @param totalSize total size of the allocation to be split
Expand Down
9 changes: 9 additions & 0 deletions include/umf/providers/provider_os_memory.h
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ typedef enum umf_mem_protection_flags_t {
/// @endcond
} umf_mem_protection_flags_t;

/// @brief Memory visibility mode
typedef enum umf_memory_visibility_t {
UMF_MEM_MAP_PRIVATE = 1, ///< private memory mapping
UMF_MEM_MAP_SHARED, ///< shared memory mapping (supported on Linux only)
} umf_memory_visibility_t;

/// @brief Memory binding mode
/// Specifies how memory is bound to NUMA nodes on systems that support NUMA.
/// Not every mode is supported on every system.
Expand Down Expand Up @@ -61,6 +67,8 @@ typedef enum umf_numa_mode_t {
typedef struct umf_os_memory_provider_params_t {
/// Combination of 'umf_mem_protection_flags_t' flags
unsigned protection;
/// memory visibility mode
umf_memory_visibility_t visibility;

// NUMA config
/// ordered list of numa nodes
Expand Down Expand Up @@ -91,6 +99,7 @@ static inline umf_os_memory_provider_params_t
umfOsMemoryProviderParamsDefault(void) {
umf_os_memory_provider_params_t params = {
UMF_PROTECTION_READ | UMF_PROTECTION_WRITE, /* protection */
UMF_MEM_MAP_PRIVATE, /* visibility mode */
NULL, /* numa_list */
0, /* numa_list_len */
UMF_NUMA_MODE_DEFAULT, /* numa_mode */
Expand Down
15 changes: 10 additions & 5 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -66,31 +66,36 @@ set(UMF_SOURCES

set(UMF_SOURCES_LINUX libumf_linux.c)

set(UMF_SOURCES_MACOSX libumf_linux.c)

set(UMF_SOURCES_WINDOWS libumf_windows.c)

# Compile definitions for UMF library.
#
# TODO: Cleanup the compile definitions across all the CMake files
set(UMF_PRIVATE_COMPILE_DEFINITIONS "")

set(UMF_SOURCES_LINUX
${UMF_SOURCES_LINUX}
set(UMF_SOURCES_COMMON_LINUX_MACOSX
provider/provider_os_memory.c
provider/provider_os_memory_linux.c
provider/provider_os_memory_posix.c
memory_targets/memory_target_numa.c
memspaces/memspace_numa.c
memspaces/memspace_host_all.c
memspaces/memspace_highest_capacity.c)

set(UMF_SOURCES_LINUX ${UMF_SOURCES_LINUX} ${UMF_SOURCES_COMMON_LINUX_MACOSX}
provider/provider_os_memory_linux.c)

set(UMF_SOURCES_MACOSX ${UMF_SOURCES_MACOSX} ${UMF_SOURCES_COMMON_LINUX_MACOSX}
provider/provider_os_memory_macosx.c)

set(UMF_SOURCES_WINDOWS ${UMF_SOURCES_WINDOWS} provider/provider_os_memory.c
provider/provider_os_memory_windows.c)

set(UMF_LIBS ${UMF_LIBS} ${LIBHWLOC_LIBRARIES})
set(UMF_PRIVATE_LIBRARY_DIRS ${UMF_PRIVATE_LIBRARY_DIRS}
${LIBHWLOC_LIBRARY_DIRS})

set(UMF_SOURCES_MACOSX ${UMF_SOURCES_MACOSX} ${UMF_SOURCES_LINUX})

if(LINUX)
set(UMF_SOURCES ${UMF_SOURCES} ${UMF_SOURCES_LINUX})
elseif(WINDOWS)
Expand Down
Loading