Skip to content

Commit 699ea52

Browse files
shijujose4bp3tk0v
shijujose4
authored and
bp3tk0v
committed
EDAC: Add a memory repair control feature
Add a generic EDAC memory repair control driver to manage memory repairs in the system, such as CXL Post Package Repair (PPR) and other soft and hard PPR features. For example, a CXL device with DRAM components that support PPR features may implement PPR maintenance operations. DRAM components may support two types of PPR: - hard PPR, for a permanent row repair, and - soft PPR, for a temporary row repair. Soft PPR is much faster than hard PPR, but the repair is lost with a power cycle. When a CXL device detects an error in a memory, it may report the need for a repair maintenance operation by using an event record where the "maintenance needed" flag is set. The event records contain the device physical address (DPA) and other optional attributes of the memory to repair. The kernel will report the corresponding CXL general media or DRAM trace event to userspace, and userspace tools (e.g. rasdaemon) will initiate a repair operation in response to the device request via the sysfs repair control. Device with memory repair features registers with EDAC device driver, which retrieves a memory repair descriptor from EDAC memory repair driver and exposes the sysfs repair control attributes to userspace in /sys/bus/edac/devices/<dev-name>/mem_repairX/. The common memory repair control interface abstracts the control of arbitrary memory repair functionality into a standardized set of functions. The sysfs memory repair attribute nodes are only available if the client driver has implemented the corresponding attribute callback function and provided operations to the EDAC device driver during registration. [ bp: Massage, fixup edac_dev_register() retvals, merge write_overflow fix to mem_repair_create_desc() ] Signed-off-by: Shiju Jose <shiju.jose@huawei.com> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Link: https://lore.kernel.org/r/20250212143654.1893-5-shiju.jose@huawei.com
1 parent bcbd069 commit 699ea52

File tree

9 files changed

+668
-0
lines changed

9 files changed

+668
-0
lines changed

‎Documentation/ABI/testing/sysfs-edac-memory-repair

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX
2+
Date: March 2025
3+
KernelVersion: 6.15
4+
Contact: linux-edac@vger.kernel.org
5+
Description:
6+
The sysfs EDAC bus devices /<dev-name>/mem_repairX subdirectory
7+
pertains to the memory media repair features control, such as
8+
PPR (Post Package Repair), memory sparing etc, where <dev-name>
9+
directory corresponds to a device registered with the EDAC
10+
device driver for the memory repair features.
11+
12+
Post Package Repair is a maintenance operation requests the memory
13+
device to perform a repair operation on its media. It is a memory
14+
self-healing feature that fixes a failing memory location by
15+
replacing it with a spare row in a DRAM device. For example, a
16+
CXL memory device with DRAM components that support PPR features may
17+
implement PPR maintenance operations. DRAM components may support
18+
two types of PPR functions: hard PPR, for a permanent row repair, and
19+
soft PPR, for a temporary row repair. Soft PPR may be much faster
20+
than hard PPR, but the repair is lost with a power cycle.
21+
22+
The sysfs attributes nodes for a repair feature are only
23+
present if the parent driver has implemented the corresponding
24+
attr callback function and provided the necessary operations
25+
to the EDAC device driver during registration.
26+
27+
In some states of system configuration (e.g. before address
28+
decoders have been configured), memory devices (e.g. CXL)
29+
may not have an active mapping in the main host address
30+
physical address map. As such, the memory to repair must be
31+
identified by a device specific physical addressing scheme
32+
using a device physical address(DPA). The DPA and other control
33+
attributes to use will be presented in related error records.
34+
35+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/repair_type
36+
Date: March 2025
37+
KernelVersion: 6.15
38+
Contact: linux-edac@vger.kernel.org
39+
Description:
40+
(RO) Memory repair type. For eg. post package repair,
41+
memory sparing etc. Valid values are:
42+
43+
- ppr - Post package repair.
44+
45+
- All other values are reserved.
46+
47+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/persist_mode
48+
Date: March 2025
49+
KernelVersion: 6.15
50+
Contact: linux-edac@vger.kernel.org
51+
Description:
52+
(RW) Get/Set the current persist repair mode set for a
53+
repair function. Persist repair modes supported in the
54+
device, based on a memory repair function, either is temporary,
55+
which is lost with a power cycle or permanent. Valid values are:
56+
57+
- 0 - Soft memory repair (temporary repair).
58+
59+
- 1 - Hard memory repair (permanent repair).
60+
61+
- All other values are reserved.
62+
63+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/repair_safe_when_in_use
64+
Date: March 2025
65+
KernelVersion: 6.15
66+
Contact: linux-edac@vger.kernel.org
67+
Description:
68+
(RO) True if memory media is accessible and data is retained
69+
during the memory repair operation.
70+
The data may not be retained and memory requests may not be
71+
correctly processed during a repair operation. In such case
72+
repair operation can not be executed at runtime. The memory
73+
must be taken offline.
74+
75+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/hpa
76+
Date: March 2025
77+
KernelVersion: 6.15
78+
Contact: linux-edac@vger.kernel.org
79+
Description:
80+
(RW) Host Physical Address (HPA) of the memory to repair.
81+
The HPA to use will be provided in related error records.
82+
83+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/dpa
84+
Date: March 2025
85+
KernelVersion: 6.15
86+
Contact: linux-edac@vger.kernel.org
87+
Description:
88+
(RW) Device Physical Address (DPA) of the memory to repair.
89+
The specific DPA to use will be provided in related error
90+
records.
91+
92+
In some states of system configuration (e.g. before address
93+
decoders have been configured), memory devices (e.g. CXL)
94+
may not have an active mapping in the main host address
95+
physical address map. As such, the memory to repair must be
96+
identified by a device specific physical addressing scheme
97+
using a DPA. The device physical address(DPA) to use will be
98+
presented in related error records.
99+
100+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/nibble_mask
101+
Date: March 2025
102+
KernelVersion: 6.15
103+
Contact: linux-edac@vger.kernel.org
104+
Description:
105+
(RW) Read/Write Nibble mask of the memory to repair.
106+
Nibble mask identifies one or more nibbles in error on the
107+
memory bus that produced the error event. Nibble Mask bit 0
108+
shall be set if nibble 0 on the memory bus produced the
109+
event, etc. For example, CXL PPR and sparing, a nibble mask
110+
bit set to 1 indicates the request to perform repair
111+
operation in the specific device. All nibble mask bits set
112+
to 1 indicates the request to perform the operation in all
113+
devices. Eg. for CXL memory repair, the specific value of
114+
nibble mask to use will be provided in related error records.
115+
For more details, See nibble mask field in CXL spec ver 3.1,
116+
section 8.2.9.7.1.2 Table 8-103 soft PPR and section
117+
8.2.9.7.1.3 Table 8-104 hard PPR, section 8.2.9.7.1.4
118+
Table 8-105 memory sparing.
119+
120+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/min_hpa
121+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/max_hpa
122+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/min_dpa
123+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/max_dpa
124+
Date: March 2025
125+
KernelVersion: 6.15
126+
Contact: linux-edac@vger.kernel.org
127+
Description:
128+
(RW) The supported range of memory address that is to be
129+
repaired. The memory device may give the supported range of
130+
attributes to use and it will depend on the memory device
131+
and the portion of memory to repair.
132+
The userspace may receive the specific value of attributes
133+
to use for a repair operation from the memory device via
134+
related error records and trace events, for eg. CXL DRAM
135+
and CXL general media error records in CXL memory devices.
136+
137+
What: /sys/bus/edac/devices/<dev-name>/mem_repairX/repair
138+
Date: March 2025
139+
KernelVersion: 6.15
140+
Contact: linux-edac@vger.kernel.org
141+
Description:
142+
(WO) Issue the memory repair operation for the specified
143+
memory repair attributes. The operation may fail if resources
144+
are insufficient based on the requirements of the memory
145+
device and repair function.
146+
147+
- 1 - Issue the repair operation.
148+
149+
- All other values are reserved.

‎Documentation/edac/features.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,3 +97,7 @@ RAS features
9797
1. Memory Scrub
9898

9999
Memory scrub features are documented in `Documentation/edac/scrub.rst`.
100+
101+
2. Memory Repair
102+
103+
Memory repair features are documented in `Documentation/edac/memory_repair.rst`.

‎Documentation/edac/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,5 @@ EDAC Subsystem
88
:maxdepth: 1
99

1010
features
11+
memory_repair
1112
scrub

‎Documentation/edac/memory_repair.rst

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.2-no-invariants-or-later
2+
3+
==========================
4+
EDAC Memory Repair Control
5+
==========================
6+
7+
Copyright (c) 2024-2025 HiSilicon Limited.
8+
9+
:Author: Shiju Jose <shiju.jose@huawei.com>
10+
:License: The GNU Free Documentation License, Version 1.2 without
11+
Invariant Sections, Front-Cover Texts nor Back-Cover Texts.
12+
(dual licensed under the GPL v2)
13+
:Original Reviewers:
14+
15+
- Written for: 6.15
16+
17+
Introduction
18+
------------
19+
20+
Some memory devices support repair operations to address issues in their
21+
memory media. Post Package Repair (PPR) and memory sparing are examples of
22+
such features.
23+
24+
Post Package Repair (PPR)
25+
~~~~~~~~~~~~~~~~~~~~~~~~~
26+
27+
Post Package Repair is a maintenance operation which requests the memory
28+
device to perform repair operation on its media. It is a memory self-healing
29+
feature that fixes a failing memory location by replacing it with a spare row
30+
in a DRAM device.
31+
32+
For example, a CXL memory device with DRAM components that support PPR
33+
features implements maintenance operations. DRAM components support those
34+
types of PPR functions:
35+
36+
- hard PPR, for a permanent row repair, and
37+
- soft PPR, for a temporary row repair.
38+
39+
Soft PPR is much faster than hard PPR, but the repair is lost after a power
40+
cycle.
41+
42+
The data may not be retained and memory requests may not be correctly
43+
processed during a repair operation. In such case, the repair operation should
44+
not be executed at runtime.
45+
46+
For example, for CXL memory devices, see CXL spec rev 3.1 [1]_ sections
47+
8.2.9.7.1.1 PPR Maintenance Operations, 8.2.9.7.1.2 sPPR Maintenance Operation
48+
and 8.2.9.7.1.3 hPPR Maintenance Operation for more details.
49+
50+
Memory Sparing
51+
~~~~~~~~~~~~~~
52+
53+
Memory sparing is a repair function that replaces a portion of memory with
54+
a portion of functional memory at a particular granularity. Memory
55+
sparing has cacheline/row/bank/rank sparing granularities. For example, in
56+
rank memory-sparing mode, one memory rank serves as a spare for other ranks on
57+
the same channel in case they fail.
58+
59+
The spare rank is held in reserve and not used as active memory until
60+
a failure is indicated, with reserved capacity subtracted from the total
61+
available memory in the system.
62+
63+
After an error threshold is surpassed in a system protected by memory sparing,
64+
the content of a failing rank of DIMMs is copied to the spare rank. The
65+
failing rank is then taken offline and the spare rank placed online for use as
66+
active memory in place of the failed rank.
67+
68+
For example, CXL memory devices can support various subclasses for sparing
69+
operation vary in terms of the scope of the sparing being performed.
70+
71+
Cacheline sparing subclass refers to a sparing action that can replace a full
72+
cacheline. Row sparing is provided as an alternative to PPR sparing functions
73+
and its scope is that of a single DDR row. Bank sparing allows an entire bank
74+
to be replaced. Rank sparing is defined as an operation in which an entire DDR
75+
rank is replaced.
76+
77+
See CXL spec 3.1 [1]_ section 8.2.9.7.1.4 Memory Sparing Maintenance
78+
Operations for more details.
79+
80+
.. [1] https://computeexpresslink.org/cxl-specification/
81+
82+
Use cases of generic memory repair features control
83+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
84+
85+
1. The soft PPR, hard PPR and memory-sparing features share similar control
86+
attributes. Therefore, there is a need for a standardized, generic sysfs
87+
repair control that is exposed to userspace and used by administrators,
88+
scripts and tools.
89+
90+
2. When a CXL device detects an error in a memory component, it informs the
91+
host of the need for a repair maintenance operation by using an event
92+
record where the "maintenance needed" flag is set. The event record
93+
specifies the device physical address (DPA) and attributes of the memory
94+
that requires repair. The kernel reports the corresponding CXL general
95+
media or DRAM trace event to userspace, and userspace tools (e.g.
96+
rasdaemon) initiate a repair maintenance operation in response to the
97+
device request using the sysfs repair control.
98+
99+
3. Userspace tools, such as rasdaemon, request a repair operation on a memory
100+
region when maintenance need flag set or an uncorrected memory error or
101+
excess of corrected memory errors above a threshold value is reported or an
102+
exceed corrected errors threshold flag set for that memory.
103+
104+
4. Multiple PPR/sparing instances may be present per memory device.
105+
106+
5. Drivers should enforce that live repair is safe. In systems where memory
107+
mapping functions can change between boots, one approach to this is to log
108+
memory errors seen on this boot against which to check live memory repair
109+
requests.
110+
111+
The File System
112+
---------------
113+
114+
The control attributes of a registered memory repair instance could be
115+
accessed in the /sys/bus/edac/devices/<dev-name>/mem_repairX/
116+
117+
sysfs
118+
-----
119+
120+
Sysfs files are documented in
121+
`Documentation/ABI/testing/sysfs-edac-memory-repair`.

‎drivers/edac/Kconfig

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,16 @@ config EDAC_ECS
9393
into a unified set of functions.
9494
Say 'y/n' to enable/disable EDAC ECS feature.
9595

96+
config EDAC_MEM_REPAIR
97+
bool "EDAC memory repair feature"
98+
help
99+
The EDAC memory repair feature is optional and is designed to control
100+
the memory devices with repair features, such as Post Package Repair
101+
(PPR), memory sparing etc. The common sysfs memory repair interface
102+
abstracts the control of various memory repair functionalities into
103+
a unified set of functions.
104+
Say 'y/n' to enable/disable EDAC memory repair feature.
105+
96106
config EDAC_AMD64
97107
tristate "AMD64 (Opteron, Athlon64)"
98108
depends on AMD_NB && EDAC_DECODE_MCE

‎drivers/edac/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ edac_core-y += edac_module.o edac_device_sysfs.o wq.o
1414
edac_core-$(CONFIG_EDAC_DEBUG) += debugfs.o
1515
edac_core-$(CONFIG_EDAC_SCRUB) += scrub.o
1616
edac_core-$(CONFIG_EDAC_ECS) += ecs.o
17+
edac_core-$(CONFIG_EDAC_MEM_REPAIR) += mem_repair.o
1718

1819
ifdef CONFIG_PCI
1920
edac_core-y += edac_pci.o edac_pci_sysfs.o

‎drivers/edac/edac_device.c

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -575,6 +575,7 @@ static void edac_dev_release(struct device *dev)
575575
{
576576
struct edac_dev_feat_ctx *ctx = container_of(dev, struct edac_dev_feat_ctx, dev);
577577

578+
kfree(ctx->mem_repair);
578579
kfree(ctx->scrub);
579580
kfree(ctx->dev.groups);
580581
kfree(ctx);
@@ -613,6 +614,7 @@ int edac_dev_register(struct device *parent, char *name,
613614
const struct attribute_group **ras_attr_groups;
614615
struct edac_dev_data *dev_data;
615616
struct edac_dev_feat_ctx *ctx;
617+
int mem_repair_cnt = 0;
616618
int attr_gcnt = 0;
617619
int ret = -ENOMEM;
618620
int scrub_cnt = 0;
@@ -631,6 +633,10 @@ int edac_dev_register(struct device *parent, char *name,
631633
case RAS_FEAT_ECS:
632634
attr_gcnt += ras_features[feat].ecs_info.num_media_frus;
633635
break;
636+
case RAS_FEAT_MEM_REPAIR:
637+
attr_gcnt++;
638+
mem_repair_cnt++;
639+
break;
634640
default:
635641
return -EINVAL;
636642
}
@@ -650,8 +656,15 @@ int edac_dev_register(struct device *parent, char *name,
650656
goto groups_free;
651657
}
652658

659+
if (mem_repair_cnt) {
660+
ctx->mem_repair = kcalloc(mem_repair_cnt, sizeof(*ctx->mem_repair), GFP_KERNEL);
661+
if (!ctx->mem_repair)
662+
goto data_mem_free;
663+
}
664+
653665
attr_gcnt = 0;
654666
scrub_cnt = 0;
667+
mem_repair_cnt = 0;
655668
for (feat = 0; feat < num_features; feat++, ras_features++) {
656669
switch (ras_features->ft_type) {
657670
case RAS_FEAT_SCRUB:
@@ -688,6 +701,25 @@ int edac_dev_register(struct device *parent, char *name,
688701

689702
attr_gcnt += ras_features->ecs_info.num_media_frus;
690703
break;
704+
case RAS_FEAT_MEM_REPAIR:
705+
if (!ras_features->mem_repair_ops ||
706+
mem_repair_cnt != ras_features->instance) {
707+
ret = -EINVAL;
708+
goto data_mem_free;
709+
}
710+
711+
dev_data = &ctx->mem_repair[mem_repair_cnt];
712+
dev_data->instance = mem_repair_cnt;
713+
dev_data->mem_repair_ops = ras_features->mem_repair_ops;
714+
dev_data->private = ras_features->ctx;
715+
ret = edac_mem_repair_get_desc(parent, &ras_attr_groups[attr_gcnt],
716+
ras_features->instance);
717+
if (ret)
718+
goto data_mem_free;
719+
720+
mem_repair_cnt++;
721+
attr_gcnt++;
722+
break;
691723
default:
692724
ret = -EINVAL;
693725
goto data_mem_free;
@@ -714,6 +746,7 @@ int edac_dev_register(struct device *parent, char *name,
714746
return devm_add_action_or_reset(parent, edac_dev_unreg, &ctx->dev);
715747

716748
data_mem_free:
749+
kfree(ctx->mem_repair);
717750
kfree(ctx->scrub);
718751
groups_free:
719752
kfree(ras_attr_groups);

0 commit comments

Comments
 (0)