Weird behavior with MPU alignment in flash section on ARM Cortex-R5 on ZynqMP

[robhancocksed]

We have a Zephyr application running on a Xilinx MPSoC (ZynqMP) platform on the Cortex-R5 (RPU) cores. Essentially the memory layout/configuration is that we have:

• CONFIG_XIP=y
• zephyr,flash pointing at the TCM internal memory at address 0
• zephyr,sram pointing to DDR memory
• Another memory region defined with various subsystems relocated into DDR (as the code does not all fit in TCM) using the NOKEEP and NOCOPY flags

The problem I am seeing is that if I end up using more than 50% (i.e. 128KB) of the TCM space for text and rodata space, then things immediately start to overflow:

/home/hancock/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/zephyr_pre0.elf section `datas' will not fit in region `FLASH'
/home/hancock/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: region `FLASH' overflowed by 8784 bytes

From what I can see from looking at the generated zephyr.map file, in this state, a bunch of padding is being inserted at the end of the rodata section:

 *(SORT_BY_ALIGNMENT(.gnu.linkonce.r.*))
                0x00000000000203e8                . = ALIGN (0x4)
 *fill*         0x00000000000203e7        0x1 
                0x00000000000203e8                __rodata_region_end = .
                0x00000000000203e8                __rom_region_end = .
                0x0000000000020400                . = ALIGN (_region_min_align)
                0x0000000000040000                . = ALIGN ((0x1 << LOG2CEIL ((__rodata_region_end - __rom_region_start))))
                0x0000000000000022                __rom_region_mpu_size_bits = ((LOG2CEIL ((__rodata_region_end - __rom_region_start)) - 0x1) << 0x1)

This appears to be coming from include/zephyr/arch/arm/cortex_a_r/scripts/linker.ld:

    SECTION_PROLOGUE(_RODATA_SECTION_NAME,,)
    {
        *(.rodata)
        *(".rodata.*")
        *(.gnu.linkonce.r.*)

/* Located in generated directory. This file is populated by the
 * zephyr_linker_sources() Cmake function.
 */
#include <snippets-rodata.ld>

#include <zephyr/linker/kobject-rom.ld>

        /*
         * For XIP images, in order to avoid the situation when __data_rom_start
         * is 32-bit aligned, but the actual data is placed right after rodata
         * section, which may not end exactly at 32-bit border, pad rodata
         * section, so __data_rom_start points at data and it is 32-bit aligned.
         *
         * On non-XIP images this may enlarge image size up to 3 bytes. This
         * generally is not an issue, since modern ROM and FLASH memory is
         * usually 4k aligned.
         */
        . = ALIGN(4);

        /*
         * RODATA must be the last section so that the size of the entire read
         * only area will be filled to a power of 2.
         */
        MPU_ALIGN(ABSOLUTE(.) - __rom_region_start);
    } GROUP_LINK_IN(ROMABLE_REGION)

It looks like what this is trying to do is align the end of the rodata section (supposedly the last) to a power of 2 boundary. The problem is that (at least in this configuration?) that is NOT the last section in the flash segment, as the data section seems to be appended to it (and gets copied to DDR on startup). So if we are already past the 128KB boundary, doing this adds a bunch of padding up to the 256KB boundary before the data section, causing the data section to no longer fit in TCM.

Something does not seem right with this code. In my application, simply deleting the MPU_ALIGN seems to resolve this problem, but I assume this is not a proper solution generically. Does someone know what is going on here or the proper way to fix it?