MCUboot incompatible with image encryption on ESP32S3 (possibly more espressif boards)

[hobbes-400]

Performing updates when flash encryption is enabled on an ESP32s3 results in the image trailer (specifically image_ok) getting into a bad state. MCUboot attempts to perform a decrypted flash read on an erased trailer (raw 0xFF), causing it to think that the erased trailer has valid data in it (decrypted 0xFF). Some details:

In MCUBoot's bootutil_public.c, the boot_read_flag function attempts to read a trailer flag and determine its state:

    static int
    boot_read_flag(const struct flash_area *fap, uint8_t *flag, uint32_t off)
    {
        int rc;
        rc = flash_area_read(fap, off, flag, sizeof *flag);
        if (rc < 0) {
            return BOOT_EFLASH;
        }
        if (bootutil_buffer_is_erased(fap, flag, sizeof *flag)) {
            *flag = BOOT_FLAG_UNSET;
        } else {
            *flag = boot_flag_decode(*flag);
        }
        return 0;
    }

This function relies on bootutil_buffer_is_erased() to check whether the trailer flag buffer is erased. If the buffer is erased, it assumes the flag is unset.

The problem arises because flash_area_read() performs a decrypted flash read. When the flash is freshly erased, which happens when image swapping occurs, its raw contents are 0xFF. However, on an encrypted board, this erased flash (0xFF) is attempted to be decrypted. Decrypting 0xFF does not necessarily yield 0xFF — it produces arbitrary data. This causes MCUBoot to incorrectly interpret the trailer as containing valid data, rather than being erased, when flash encryption is enabled. Of course, the "valid data" is decrypted garbage, getting the trailer into a corrupted state.

Despite this fact, the decrypted flash read is required in some cases. For instance, imgtool creates an "erased" trailer filled with 0xFF when OTA assets are first generated. When this asset is flashed over-the-air, the 0xFF values are automatically encrypted. Upon reading them, MCUBoot correctly decrypts the contents back to 0xFF and correctly recognizes them as erased, setting the flag to BOOT_FLAG_UNSET for newly flashed update assets (that have not been swapped / moved yet).

In summary, it appears boot_read_flag only supports decrypted flash flows OR encrypted flash flows that have been pre populated with valid data. It does not support flows that have encrypted flash that have been erased. More concretely, when flash encryption is enabled, bootutil_buffer_is_erased is more accurately bootutil_buffer_is_unset as the read APIs do not take into account the raw flash contents.

We were able to verify this theory with the following patch:

     static int
     boot_read_flag(const struct flash_area *fap, uint8_t *flag, uint32_t off)
     {
         int rc;
         rc = flash_area_read(fap, off, flag, sizeof *flag);
         if (rc < 0) {
             return BOOT_EFLASH;
         }
    -    if (bootutil_buffer_is_erased(fap, flag, sizeof *flag)) {
    -        BOOT_LOG_INF("buffer erased");
    +    uint8_t raw_flag;
    +    rc = flash_area_read_raw(fap, off, &raw_flag, sizeof *flag); // perform raw / non-decrypted flash read
    +    if (rc < 0) {
    +        return BOOT_EFLASH;
    +    }
    +
    +    bool raw_val_erased = bootutil_buffer_is_erased(fap, &raw_flag, sizeof *flag);
    +    bool decrypt_val_erased = bootutil_buffer_is_erased(fap, flag, sizeof *flag);
    +    if (raw_val_erased || decrypt_val_erased) {
    +        BOOT_LOG_INF("buffer erased. raw_val_erased:%d val_erased:%d", raw_val_erased, val_erased);

              *flag = BOOT_FLAG_UNSET;

Potential solution:
For flash encryption flows, manually set image_ok and other relevant flags to UNSET after they have been erased / copied over. A flow similar to this is already used for confirmed image updates. In this case, image_ok is set to 1 after the copy is done.

Of note:
If we attempt to OTA a confirmed image, the OTA works as expected as image_ok is overwritten during the swap process, leaving it in a known, good state.

This flow occurs both when the OTA asset is updated via BLE (smpmgr) and when hardflashed using esptool. Information for the latter is provided below.

Commits used (patches for the commits linked below):
MCUBoot: commit 346f737
Zephyr: commit 7823374e872145b5bd018bfe447839eb36042611 (tag: v4.1.0)
ESP-IDF: commit d7b0a45ddbddbac53afb4fc28168f9f9259dbb79 (HEAD, tag: v5.1.4)

mcuboot_patch.txt

zephyr_patch.txt

I believe all required updates for flash encryption to work are addressed in the patches (updating write-block-size, ensuring MCU_BOOT_MAX_ALIGN is set appropriately, and image is padded / aligned as expected)

Build MCUBoot:

cd bootloader/mcuboot/boot/espressif
cmake -DCMAKE_TOOLCHAIN_FILE=tools/toolchain-esp32s3.cmake -DMCUBOOT_TARGET=esp32s3  -DMCUBOOT_FLASH_PORT=/dev/ttyACM1 -B build -GNinja
ninja -C build

Build Zephyr app:

cd zephyr/samples/subsys/mgmt/mcumgr/smp_svr/
west build -b esp32s3_devkitm/esp32s3/procpu -p --  -DEXTRA_CONF_FILE="overlay-bt.conf"

Flashing commands:

esptool.py -p $DEVICE -b 2000000 --no-stub --after no_reset write_flash  --flash_mode dio  --flash_freq 40m --encrypt 0x0000 build/mcuboot_esp32s3.bin --force
esptool.py -p $DEVICE -b 2000000 --no-stub --after no_reset write_flash  --flash_mode dio  --flash_freq 40m --encrypt 0x0020000  build/zephyr/zephyr.signed.bin --force
esptool.py -p $DEVICE -b 2000000 --no-stub --after no_reset write_flash  --flash_mode dio  --flash_freq 40m --encrypt 0x00170000  build/zephyr/zephyr.signed.bin --force

Security EFuses:

Security fuses:
DIS_DOWNLOAD_ICACHE (BLOCK0)                       Set this bit to disable Icache in download mode (b = False R/- (0b0)
                                                   oot_mode[3:0] is 0; 1; 2; 3; 6; 7)                
DIS_DOWNLOAD_DCACHE (BLOCK0)                       Set this bit to disable Dcache in download mode (  = False R/- (0b0)
                                                   boot_mode[3:0] is 0; 1; 2; 3; 6; 7)              
DIS_FORCE_DOWNLOAD (BLOCK0)                        Set this bit to disable the function that forces c = False R/- (0b0)
                                                   hip into download mode                            
DIS_DOWNLOAD_MANUAL_ENCRYPT (BLOCK0)               Set this bit to disable flash encryption when in d = False R/- (0b0)
                                                   ownload boot modes                                
SPI_BOOT_CRYPT_CNT (BLOCK0)                        Enables flash encryption when 1 or 3 bits are set  = Enable R/W (0b111)
                                                   and disabled otherwise                            
SECURE_BOOT_KEY_REVOKE0 (BLOCK0)                   Revoke 1st secure boot key                         = False R/W (0b0)
SECURE_BOOT_KEY_REVOKE1 (BLOCK0)                   Revoke 2nd secure boot key                         = False R/W (0b0)
SECURE_BOOT_KEY_REVOKE2 (BLOCK0)                   Revoke 3rd secure boot key                         = False R/W (0b0)
KEY_PURPOSE_0 (BLOCK0)                             Purpose of Key0                                    = XTS_AES_128_KEY R/- (0x4)
KEY_PURPOSE_1 (BLOCK0)                             Purpose of Key1                                    = USER R/W (0x0)
KEY_PURPOSE_2 (BLOCK0)                             Purpose of Key2                                    = USER R/W (0x0)
KEY_PURPOSE_3 (BLOCK0)                             Purpose of Key3                                    = USER R/W (0x0)
KEY_PURPOSE_4 (BLOCK0)                             Purpose of Key4                                    = USER R/W (0x0)
KEY_PURPOSE_5 (BLOCK0)                             Purpose of Key5                                    = USER R/W (0x0)
SECURE_BOOT_EN (BLOCK0)                            Set this bit to enable secure boot                 = False R/W (0b0)
SECURE_BOOT_AGGRESSIVE_REVOKE (BLOCK0)             Set this bit to enable revoking aggressive secure  = False R/W (0b0)
                                                   boot                                              
DIS_DOWNLOAD_MODE (BLOCK0)                         Set this bit to disable download mode (boot_mode[3 = False R/W (0b0)
                                                   :0] = 0; 1; 2; 3; 6; 7)                          
ENABLE_SECURITY_DOWNLOAD (BLOCK0)                  Set this bit to enable secure UART download mode   = False R/W (0b0)
SECURE_VERSION (BLOCK0)                            Secure version (used by ESP-IDF anti-rollback feat = 0 R/W (0x0000)
                                                   ure)                                              
BLOCK_KEY0 (BLOCK4)
  Purpose: XTS_AES_128_KEY
    Key0 or user data                                
   = ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? -/-
BLOCK_KEY1 (BLOCK5)
  Purpose: USER
               Key1 or user data                                
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W
BLOCK_KEY2 (BLOCK6)
  Purpose: USER
               Key2 or user data                                
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W
BLOCK_KEY3 (BLOCK7)
  Purpose: USER
               Key3 or user data                                
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W
BLOCK_KEY4 (BLOCK8)
  Purpose: USER
               Key4 or user data                                
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W
BLOCK_KEY5 (BLOCK9)
  Purpose: USER
               Key5 or user data                                
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W

Zephyr logs from OTA attempt

SP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40375e30
SPIWP:0xee
mode:DIO, clock div:2
load:0x3fcd37f0,len:0x2b50
load:0x403b0000,len:0x1ff0
load:0x403ba000,len:0x492c
entry 0x403be854
[esp32s3] [INF] *** Booting MCUboot build v2.1.0-rc1-234-gcbe9b48d ***
[esp32s3] [INF] [boot] chip revision: v0.2
[esp32s3] [INF] [boot.esp32s3] Boot SPI Speed : 40MHz
[esp32s3] [INF] [boot.esp32s3] SPI Mode       : DIO
[esp32s3] [INF] [boot.esp32s3] SPI Flash Size : 8MB
[esp32s3] [INF] [boot] Enabling RNG early entropy source...
[esp32s3] [INF] Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x2
[esp32s3] [INF] Scratch: magic=bad, swap_type=0x1, copy_done=0x2, image_ok=0x2
[esp32s3] [INF] Boot source: primary slot
[esp32s3] [INF] Image index: 0, Swap type: test
[esp32s3] [INF] Starting swap using scratch algorithm.
[esp32s3] [INF] Checking flash encryption...
[esp32s3] [INF] [flash_encrypt] flash encryption is enabled (0 plaintext flashes left)
[esp32s3] [INF] Disabling RNG early entropy source...
[esp32s3] [INF] br_image_off = 0x20000
[esp32s3] [INF] ih_hdr_size = 0x20
[esp32s3] [INF] Loading image 0 - slot 0 from flash, area id: 1
[esp32s3] [INF] DRAM segment: start=0x316c0, size=0x261c, vaddr=0x3fc95650
[esp32s3] [INF] IRAM segment: start=0x20080, size=0x11640, vaddr=0x40374000
[esp32s3] [INF] start=0x403803c0
I (9282) boot: IROM segment: paddr=00040000h, vaddr=42000000h, size=3548Eh (218254) map
I (9283) boot: DROM segment: paddr=00080000h, vaddr=3c040000h, size=0A1A0h ( 41376) map
I (9298) boot: libc heap size 240 kB.
I (9298) spi_flash: detected chip: generic
I (9299) spi_flash: flash io: dio

*** Booting Zephyr OS build v4.1.0-4-gecd4e9c68a7b ***
[00:00:09.680,000] <inf> littlefs: LittleFS version 2.10, disk version 2.1
[00:00:09.681,000] <inf> littlefs: FS at flash-controller@60002000:0x3b0000 is 48 0x1000-byte blocks with 512 cycle
[00:00:09.681,000] <inf> littlefs: partition sizes: rd 16 ; pr 16 ; ca 64 ; la 32
[00:00:09.682,000] <inf> esp32_bt_adapter: BT controller compile version [fd62b31]
[00:00:09.720,000] <inf> bt_hci_core: Identity: CC:8D:A2:ED:C5:64 (public)
[00:00:09.720,000] <inf> bt_hci_core: HCI: version 5.0 (0x09) revision 0x0016, manufacturer 0x02e5
[00:00:09.720,000] <inf> bt_hci_core: LMP: version 5.0 (0x09) subver 0x0016
[00:00:09.721,000] <inf> smp_bt_sample: Advertising successfully started
[00:00:09.721,000] <inf> smp_sample: build time: Apr 28 2025 23:16:30
[00:00:09.722,000] <inf> smp_sample: boot_write_img_confirmed() appears to have been successful
[00:00:09.722,000] <err> smp_sample: boot_write_img_confirmed did not work -- this shouldn't happen

[hobbes-400]

@rftafas moving this discussion here

[almir-okato]

@hobbes-400 I've opened two draft PRs for two potential solutions, the branches needed by them for Zephyr and hal_espressif are indicated on each PR description:

#2321
DRAFT: ESP32-XX hardware flash encryption issue when updating images - "bootutil layer" solution

#2320
DRAFT: ESP32-XX hardware flash encryption issue when updating images - "flash imp layer" solution

Please note that there are still scenarios that need testing, and also some discussion may raise regarding which kind of solution fits MCUboot and Zephyr better.

Could you please test your scenario with these?