kdump on arm64

Refer to: https://www.kernel.org/doc/Documentation/admin-guide/kdump/kdump.rst

There are two ways to reserve crashkernel memory:

1. crashkernel=size. The crashkernel region can be automatically placed by the system kernel at run time.
2. crashkernel=size@offset. User specify offset. If memory [offset, offset+size] is used, reservation may fail.

In case 1), arm64 kernel by default reserves memory in region bellow 4GB (a.k.a. low region). If for some reason there is no enough space in low region, reservation fails.

We backported bellow patches from: https://lkml.org/lkml/2021/1/30/53, to make 5.4 and 5.10 LTS kernel supports reservation above 4GB. (These patches are not merged to upstream):

0001-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
0002-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
0003-arm64-kdump-reimplement-crashkernel-X.patch
0004-x86-arm64-Add-ARCH_WANT_RESERVE_CRASH_KERNEL-config.patch
0005-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
0006-kdump-update-Documentation-about-crashkernel.patch
0007-arm64-kdump-remove-dependency-on-arm64_dma32_phys_li.patch

Recommended kernel option:

"crashkernel=512M-12G:128M,12G-64G:256M,64G-128G:512M,128G-:768M crashkernel=16M,low" This option tries to reserved 16MB bellow 4G, and 768MB above 4G. From dmesg:

[    0.000000] Reserving 16MB of low memory at 3052MB for crashkernel (low RAM limit: 4096MB) //Note, offset and size may be different
[    0.000000] Reserving 768MB of memory at 67370240MB for crashkernel (System RAM: 523009MB) //Note, offset and size may be different

Optional patch:

Centos-8.3 kexec-tools(2.0.22) by default uses kexec_file_load() syscall. These two patches add Kexec_file crashdump support in kernel. Otherwise, kexec cannot load crashkernel.

0008-libfdt-include-fdt_addresses.c.patch 0009-arm64-kexec_file-add-crash-dump-support.patch

If your kexec-tools does not use kexec_file_load() to load crashkernel, just ignore these two patches.

To support kexec_file_load(), also enable 'CONFIG_KEXEC_FILE'.

kernel option details:

1. If set option: "crashkernel=512M-12G:128M,12G-64G:256M,64G-128G:512M,128G-:768M" By default, "Kernel will allocate at least 256M memory below 4G automatically if crashkernel=Y,low is not specified." [1] On some platforms, this automatical allocation may fail: "Cannot reserve 256MB crashkernel low memory, please try smaller size".

2. If above error, try allocating smaller size in low region: E.g: "crashkernel=512M-12G:128M,12G-64G:256M,64G-128G:512M,128G-:768M crashkernel=16M,low" This size depends on how much memory the crash kernel may use in low region, e.g, for DMA buffer.

Reference:

[1]: kernel by default reserve 256MB in low region

	/* crashkernel=Y,low */
	ret = parse_crashkernel_low(boot_command_line, low_mem_limit, &low_size, &base);
	if (ret) {
		/*
		 * two parts from kernel/dma/swiotlb.c:
		 * -swiotlb size: user-specified with swiotlb= or default.
		 *
		 * -swiotlb overflow buffer: now hardcoded to 32k. We round it
		 * to 8M for other buffers that may need to stay low too. Also
		 * make sure we allocate enough extra low memory so that we
		 * don't run out of DMA buffers for 32-bit devices.
		 */
		low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);

[2]: LTS 5.4 kernel patches:

12b62d24d2f6 arm64: kexec_file: add crash dump support dc68409ae05a libfdt: include fdt_addresses.c 2471c317ff78 arm64: kdump: remove dependency on arm64_dma32_phys_limit da09afd8cc1c kdump: update Documentation about crashkernel 57917df7bfba arm64: kdump: add memory for devices by DT property linux,usable-memory-range ab5cd9074801 x86, arm64: Add ARCH_WANT_RESERVE_CRASH_KERNEL config 039507e1290e arm64: kdump: reimplement crashkernel=X 99bff284ee7a arm64: kdump: introduce some macroes for crash kernel reservation 3121eb3b635a x86: kdump: move reserve_crashkernel_low into crash_core.c