Skip to content

Commit 4ed8534

Browse files
authored
[SYCL][Graph][OpenCL] Map copy/fill to SVM (#18177)
There is currently no cl_khr_command_buffer or cl_intel_unified_shared_memory entry-point for appending USM copy or fill commands to a command-buffer. This prevents these commands from being added to a graph for the OpenCL backend. The long term solution to this is the OpenCL USVM extension which will align USM and SVM, allowing the existing SVM entry-points to be used. To prepare for this, map the UR entry-points to the cl_khr_command_buffer SVM copy/fill commands. This will work on OpenCL implementations that share an implementation for USM and SVM. Signed-off-by: Ewan Crawford <ewan@codeplay.com>
1 parent b07a0cf commit 4ed8534

27 files changed

+96
-114
lines changed

sycl/doc/design/CommandGraph.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -626,8 +626,8 @@ adapter where there is matching support for each function in the list.
626626
| urCommandBufferReleaseExp | clReleaseCommandBufferKHR | Yes |
627627
| urCommandBufferFinalizeExp | clFinalizeCommandBufferKHR | Yes |
628628
| urCommandBufferAppendKernelLaunchExp | clCommandNDRangeKernelKHR | Yes |
629-
| urCommandBufferAppendUSMMemcpyExp | | No |
630-
| urCommandBufferAppendUSMFillExp | | No |
629+
| urCommandBufferAppendUSMMemcpyExp | clCommandSVMMemcpyKHR | Partial, [see below](#unsupported-command-types) |
630+
| urCommandBufferAppendUSMFillExp | clCommandSVMMemFillKHR | Partial, [see below](#unsupported-command-types) |
631631
| urCommandBufferAppendMembufferCopyExp | clCommandCopyBufferKHR | Yes |
632632
| urCommandBufferAppendMemBufferWriteExp | | No |
633633
| urCommandBufferAppendMemBufferReadExp | | No |
@@ -643,8 +643,6 @@ adapter where there is matching support for each function in the list.
643643
| | clCommandCopyImageToBufferKHR | No |
644644
| | clCommandFillImageKHR | No |
645645
| | clGetCommandBufferInfoKHR | No |
646-
| | clCommandSVMMemcpyKHR | No |
647-
| | clCommandSVMMemFillKHR | No |
648646
| urCommandBufferUpdateKernelLaunchExp | clUpdateMutableCommandsKHR | Partial [See Update Section](#update-support) |
649647

650648
We are looking to address these gaps in the future so that SYCL-Graph can be
@@ -664,17 +662,21 @@ terminate called after throwing an instance of 'sycl::_V1::exception'
664662
what(): USM copy command not supported by graph backend
665663
```
666664

667-
The types of commands which are unsupported, and lead to this exception are:
665+
The types of commands which are unsupported, and may lead to this exception are:
668666
* `handler::copy(src, dest)` - Where `src` is an accessor and `dest` is a pointer.
669667
This corresponds to a memory buffer read command.
670668
* `handler::copy(src, dest)` - Where `src` is an pointer and `dest` is an accessor.
671669
This corresponds to a memory buffer write command.
672670
* `handler::copy(src, dest)` or `handler::memcpy(dest, src)` - Where both `src` and
673-
`dest` are USM pointers. This corresponds to a USM copy command.
671+
`dest` are USM pointers. This corresponds to a USM copy command that is
672+
mapped to `clCommandSVMMemcpyKHR`, which will only work on OpenCL devices
673+
which don't differentiate between USM and SVM.
674674
* `handler::fill(ptr, pattern, count)` - This corresponds to a USM memory
675-
fill command.
675+
fill command that is mapped to `clCommandSVMMemFillKHR`, which will only work
676+
on OpenCL devices which don't differentiate between USM and SVM.
676677
* `handler::memset(ptr, value, numBytes)` - This corresponds to a USM memory
677-
fill command.
678+
fill command that is mapped to `clCommandSVMMemFillKHR`, which will only work
679+
on OpenCL devices which don't differentiate between USM and SVM.
678680
* `handler::prefetch()`.
679681
* `handler::mem_advise()`.
680682

sycl/source/detail/memory_manager.cpp

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1567,9 +1567,17 @@ void MemoryManager::ext_oneapi_fill_usm_cmd_buffer(
15671567
"NULL pointer argument in memory fill operation.");
15681568

15691569
const AdapterPtr &Adapter = Context->getAdapter();
1570-
Adapter->call<UrApiKind::urCommandBufferAppendUSMFillExp>(
1571-
CommandBuffer, DstMem, Pattern.data(), Pattern.size(), Len, Deps.size(),
1572-
Deps.data(), 0, nullptr, OutSyncPoint, nullptr, nullptr);
1570+
ur_result_t Result =
1571+
Adapter->call_nocheck<UrApiKind::urCommandBufferAppendUSMFillExp>(
1572+
CommandBuffer, DstMem, Pattern.data(), Pattern.size(), Len,
1573+
Deps.size(), Deps.data(), 0, nullptr, OutSyncPoint, nullptr, nullptr);
1574+
if (Result == UR_RESULT_ERROR_UNSUPPORTED_FEATURE) {
1575+
throw sycl::exception(
1576+
sycl::make_error_code(sycl::errc::feature_not_supported),
1577+
"USM fill command not supported by graph backend");
1578+
} else {
1579+
Adapter->checkUrResult(Result);
1580+
}
15731581
}
15741582

15751583
void MemoryManager::ext_oneapi_fill_cmd_buffer(

sycl/test-e2e/Graph/Explicit/usm_copy.cpp

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@
44
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=0 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
55
// Extra run to check for immediate-command-list in Level Zero
66
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
7-
//
8-
//
9-
// Intended - USM copy command not supported for OpenCL
10-
// UNSUPPORTED: opencl
117

128
#define GRAPH_E2E_EXPLICIT
139

sycl/test-e2e/Graph/Explicit/usm_fill.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@
44
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=0 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
55
// Extra run to check for immediate-command-list in Level Zero
66
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
7-
//
8-
// Intended - USM fill command not supported for OpenCL
9-
// UNSUPPORTED: opencl
107

118
#define GRAPH_E2E_EXPLICIT
129

sycl/test-e2e/Graph/Explicit/usm_fill_host.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,6 @@
77

88
// REQUIRES: aspect-usm_host_allocations
99

10-
// Intended - USM fill command not supported for OpenCL
11-
// UNSUPPORTED: opencl
12-
1310
#define GRAPH_E2E_EXPLICIT
1411

1512
#include "../Inputs/usm_fill_host.cpp"

sycl/test-e2e/Graph/Explicit/usm_fill_shared.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,6 @@
77

88
// REQUIRES: aspect-usm_shared_allocations
99

10-
// Intended - USM fill command not supported for OpenCL
11-
// UNSUPPORTED: opencl
12-
1310
#define GRAPH_E2E_EXPLICIT
1411

1512
#include "../Inputs/usm_fill_shared.cpp"

sycl/test-e2e/Graph/Explicit/usm_memset.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,6 @@
55
// Extra run to check for immediate-command-list in Level Zero
66
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
77

8-
// Intended - USM memset command not supported for OpenCL
9-
// UNSUPPORTED: opencl
10-
118
#define GRAPH_E2E_EXPLICIT
129

1310
#include "../Inputs/usm_memset.cpp"

sycl/test-e2e/Graph/RecordReplay/barrier_multi_graph.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@
44
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=0 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
55
// Extra run to check for immediate-command-list in Level Zero
66
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
7-
//
8-
// UNSUPPORTED: opencl
9-
// UNSUPPORTED-INTENDED: USM memcpy command not supported for OpenCL
107

118
#include "../graph_common.hpp"
129

sycl/test-e2e/Graph/RecordReplay/barrier_multi_queue.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@
44
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=0 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
55
// Extra run to check for immediate-command-list in Level Zero
66
// RUN: %if level_zero %{env SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 %{l0_leak_check} %{run} %t.out 2>&1 | FileCheck %s --implicit-check-not=LEAK %}
7-
//
8-
// UNSUPPORTED: opencl
9-
// UNSUPPORTED-INTENDED: USM memcpy command not supported for OpenCL
107

118
#include "../graph_common.hpp"
129

sycl/test-e2e/Graph/RecordReplay/ext_oneapi_enqueue_functions.cpp

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,6 @@
77

88
// Tests the enqueue free function kernel shortcuts.
99

10-
// UNSUPPORTED: opencl
11-
// UNSUPPORTED-INTENDED: USM memcpy command not supported for OpenCL
12-
1310
#include "../graph_common.hpp"
1411
#include <sycl/ext/oneapi/experimental/enqueue_functions.hpp>
1512
#include <sycl/properties/all_properties.hpp>

0 commit comments

Comments
 (0)