Skip to content

Commit 9bd8546

Browse files
Gregory Pricedavejiang
authored andcommitted
cxl: docs/linux - overview
Add type-3 device configuration overview that explains the probe process for a type-3 device from early-boot through memory-hotplug. Signed-off-by: Gregory Price <gourry@gourry.net> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Link: https://patch.msgid.link/20250512162134.3596150-8-gourry@gourry.net Signed-off-by: Dave Jiang <dave.jiang@intel.com>
1 parent 8545940 commit 9bd8546

File tree

2 files changed

+105
-1
lines changed

2 files changed

+105
-1
lines changed

Documentation/driver-api/cxl/index.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,10 @@ that have impacts on each other. The docs here break up configurations steps.
3030
platform/example-configs
3131

3232
.. toctree::
33-
:maxdepth: 1
33+
:maxdepth: 2
3434
:caption: Linux Kernel Configuration
3535

36+
linux/overview
3637
linux/access-coordinates
3738

3839

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
.. SPDX-License-Identifier: GPL-2.0
2+
3+
========
4+
Overview
5+
========
6+
7+
This section presents the configuration process of a CXL Type-3 memory device,
8+
and how it is ultimately exposed to users as either a :code:`DAX` device or
9+
normal memory pages via the kernel's page allocator.
10+
11+
Portions marked with a bullet are points at which certain kernel objects
12+
are generated.
13+
14+
1) Early Boot
15+
16+
a) BIOS, Build, and Boot Parameters
17+
18+
i) EFI_MEMORY_SP
19+
ii) CONFIG_EFI_SOFT_RESERVE
20+
iii) CONFIG_MHP_DEFAULT_ONLINE_TYPE
21+
iv) nosoftreserve
22+
23+
b) Memory Map Creation
24+
25+
i) EFI Memory Map / E820 Consulted for Soft-Reserved
26+
27+
* CXL Memory is set aside to be handled by the CXL driver
28+
29+
* Soft-Reserved IO Resource created for CFMWS entry
30+
31+
c) NUMA Node Creation
32+
33+
* Nodes created from ACPI CEDT CFMWS and SRAT Proximity domains (PXM)
34+
35+
d) Memory Tier Creation
36+
37+
* A default memory_tier is created with all nodes.
38+
39+
e) Contiguous Memory Allocation
40+
41+
* Any requested CMA is allocated from Online nodes
42+
43+
f) Init Finishes, Drivers start probing
44+
45+
2) ACPI and PCI Drivers
46+
47+
a) Detects PCI device is CXL, marking it for probe by CXL driver
48+
49+
3) CXL Driver Operation
50+
51+
a) Base device creation
52+
53+
* root, port, and memdev devices created
54+
* CEDT CFMWS IO Resource creation
55+
56+
b) Decoder creation
57+
58+
* root, switch, and endpoint decoders created
59+
60+
c) Logical device creation
61+
62+
* memory_region and endpoint devices created
63+
64+
d) Devices are associated with each other
65+
66+
* If auto-decoder (BIOS-programmed decoders), driver validates
67+
configurations, builds associations, and locks configs at probe time.
68+
69+
* If user-configured, validation and associations are built at
70+
decoder-commit time.
71+
72+
e) Regions surfaced as DAX region
73+
74+
* dax_region created
75+
76+
* DAX device created via DAX driver
77+
78+
4) DAX Driver Operation
79+
80+
a) DAX driver surfaces DAX region as one of two dax device modes
81+
82+
* kmem - dax device is converted to hotplug memory blocks
83+
84+
* DAX kmem IO Resource creation
85+
86+
* hmem - dax device is left as daxdev to be accessed as a file.
87+
88+
* If hmem, journey ends here.
89+
90+
b) DAX kmem surfaces memory region to Memory Hotplug to add to page
91+
allocator as "driver managed memory"
92+
93+
5) Memory Hotplug
94+
95+
a) mhp component surfaces a dax device memory region as multiple memory
96+
blocks to the page allocator
97+
98+
* blocks appear in :code:`/sys/bus/memory/devices` and linked to a NUMA node
99+
100+
b) blocks are onlined into the requested zone (NORMAL or MOVABLE)
101+
102+
* Memory is marked "Driver Managed" to avoid kexec from using it as region
103+
for kernel updates

0 commit comments

Comments
 (0)