|
| 1 | +.. SPDX-License-Identifier: GPL-2.0 |
| 2 | +
|
| 3 | +======== |
| 4 | +Overview |
| 5 | +======== |
| 6 | + |
| 7 | +This section presents the configuration process of a CXL Type-3 memory device, |
| 8 | +and how it is ultimately exposed to users as either a :code:`DAX` device or |
| 9 | +normal memory pages via the kernel's page allocator. |
| 10 | + |
| 11 | +Portions marked with a bullet are points at which certain kernel objects |
| 12 | +are generated. |
| 13 | + |
| 14 | +1) Early Boot |
| 15 | + |
| 16 | + a) BIOS, Build, and Boot Parameters |
| 17 | + |
| 18 | + i) EFI_MEMORY_SP |
| 19 | + ii) CONFIG_EFI_SOFT_RESERVE |
| 20 | + iii) CONFIG_MHP_DEFAULT_ONLINE_TYPE |
| 21 | + iv) nosoftreserve |
| 22 | + |
| 23 | + b) Memory Map Creation |
| 24 | + |
| 25 | + i) EFI Memory Map / E820 Consulted for Soft-Reserved |
| 26 | + |
| 27 | + * CXL Memory is set aside to be handled by the CXL driver |
| 28 | + |
| 29 | + * Soft-Reserved IO Resource created for CFMWS entry |
| 30 | + |
| 31 | + c) NUMA Node Creation |
| 32 | + |
| 33 | + * Nodes created from ACPI CEDT CFMWS and SRAT Proximity domains (PXM) |
| 34 | + |
| 35 | + d) Memory Tier Creation |
| 36 | + |
| 37 | + * A default memory_tier is created with all nodes. |
| 38 | + |
| 39 | + e) Contiguous Memory Allocation |
| 40 | + |
| 41 | + * Any requested CMA is allocated from Online nodes |
| 42 | + |
| 43 | + f) Init Finishes, Drivers start probing |
| 44 | + |
| 45 | +2) ACPI and PCI Drivers |
| 46 | + |
| 47 | + a) Detects PCI device is CXL, marking it for probe by CXL driver |
| 48 | + |
| 49 | +3) CXL Driver Operation |
| 50 | + |
| 51 | + a) Base device creation |
| 52 | + |
| 53 | + * root, port, and memdev devices created |
| 54 | + * CEDT CFMWS IO Resource creation |
| 55 | + |
| 56 | + b) Decoder creation |
| 57 | + |
| 58 | + * root, switch, and endpoint decoders created |
| 59 | + |
| 60 | + c) Logical device creation |
| 61 | + |
| 62 | + * memory_region and endpoint devices created |
| 63 | + |
| 64 | + d) Devices are associated with each other |
| 65 | + |
| 66 | + * If auto-decoder (BIOS-programmed decoders), driver validates |
| 67 | + configurations, builds associations, and locks configs at probe time. |
| 68 | + |
| 69 | + * If user-configured, validation and associations are built at |
| 70 | + decoder-commit time. |
| 71 | + |
| 72 | + e) Regions surfaced as DAX region |
| 73 | + |
| 74 | + * dax_region created |
| 75 | + |
| 76 | + * DAX device created via DAX driver |
| 77 | + |
| 78 | +4) DAX Driver Operation |
| 79 | + |
| 80 | + a) DAX driver surfaces DAX region as one of two dax device modes |
| 81 | + |
| 82 | + * kmem - dax device is converted to hotplug memory blocks |
| 83 | + |
| 84 | + * DAX kmem IO Resource creation |
| 85 | + |
| 86 | + * hmem - dax device is left as daxdev to be accessed as a file. |
| 87 | + |
| 88 | + * If hmem, journey ends here. |
| 89 | + |
| 90 | + b) DAX kmem surfaces memory region to Memory Hotplug to add to page |
| 91 | + allocator as "driver managed memory" |
| 92 | + |
| 93 | +5) Memory Hotplug |
| 94 | + |
| 95 | + a) mhp component surfaces a dax device memory region as multiple memory |
| 96 | + blocks to the page allocator |
| 97 | + |
| 98 | + * blocks appear in :code:`/sys/bus/memory/devices` and linked to a NUMA node |
| 99 | + |
| 100 | + b) blocks are onlined into the requested zone (NORMAL or MOVABLE) |
| 101 | + |
| 102 | + * Memory is marked "Driver Managed" to avoid kexec from using it as region |
| 103 | + for kernel updates |
0 commit comments