Challenge !!!! Enabling flash encryption OTA !!!! (IDFGH-16646) #17745
Description
Answers checklist.
- I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
- I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
- I have searched the issue tracker for a similar issue and not found a similar issue.
General issue report
I currently have units in the field that have Secure Boot enabled and have been operating that way for some time on ESP32. I would like to increase the security of these units by enabling Flash Encryption over-the-air (OTA). However, I am facing a couple of issues.
To give you some context, the units in the field have been using SPIFFS as their storage system. They also have the partition table located at offset 0x8000, with Secure Boot enabled.
Old partition table :
nvs, data, nvs, 0x9000, 16k,
otadata, data, ota, 0xd000, 8k,
phy_init, data, phy, 0xf000, 4k,
factory, app, factory, 0x10000, 2M,
ota_0, app, ota_0, 0x210000, 2M,
ota_1, app, ota_1, 0x410000, 2M,
storage, data, spiffs, , 1M,
My new partitions able (bundled as a binary with the firmware) :
otadata, data, ota, 0xd000, 8K,
phy_init, data, phy, 0xf000, 4k,
factory, app, factory, 0x10000, 2M,
ota_0, app, ota_0, 0x210000, 2M,
ota_1, app, ota_1, 0x410000, 2M,
nvs, data, nvs, 0x610000, 256K
nvs_keys, data, nvs_keys,0x650000, 4K, encrypted
To enable Flash Encryption, I need a new bootloader bundled with the firmware (I have already successfully updated bootloaders OTA in the past), as well as an updated partition table bundled with the firmware. This is necessary because I need to move the partition table to a different location to accommodate the larger bootloader. I am also switching from SPIFFS to NVS.
I have built two migration firmware versions:
Firmware 1 (no flash encryption, partition table offset set to 0x8000):
Exports the old configuration to an unused area of flash.
Writes the new partition table to a new location at 0xc000.
Once this is done, Firmware 1 updates to Firmware 2 (flash encryption enabled, partition table offset set to 0xc000).
Firmware 2 successfully reads the new partition table at 0xc000, which was written by Firmware 1.
This firmware then updates the bootloader to a new version that is built with a partition table offset of 0xc000 and includes flash encryption support.
The issue arises after reboot:
The ROM bootloader fails to read the second-stage bootloader’s Secure Boot magic byte, even though the bootloader is 100% correctly signed (I have triple-checked this I also used espsecure to verify that its correctly signed).
I’m not sure what I’m doing wrong, but any help would be hugely appreciated this issue has been driving me mad.
The code I am using for the re partition + update : Worth mentioning everything returns ESP_OK so no errors really
static esp_err_t
MovePartitionTable(void)
{
esp_err_t err = ESP_OK;
esp_flash_t *const p_chip = esp_flash_default_chip;
const esp_flash_os_functions_t *const p_originalFunctions = p_chip->os_func;
const size_t partitionTableSize = (size_t)(partitionImageEnd - partitionImageStart);
ESP_LOGI(LOG_TAG_REPARTITION, "Starting partition table move (size=%zu bytes)", partitionTableSize);
// Copy the existing hooks into our RAM instance so we can patch out the protection checks temporarily.
s_flashOsFunctions = *p_chip->os_func;
s_flashOsFunctions.region_protected = NULL;
p_chip->os_func = &s_flashOsFunctions;
ESP_LOGI(LOG_TAG_REPARTITION, "Temporarily disabled region protection for partition update");
// Round up to whole flash sectors before erasing the partition table region.
const size_t eraseLength = (partitionTableSize + (FLASH_SECTOR_SIZE - 1u)) & ~(FLASH_SECTOR_SIZE - 1u);
ESP_LOGI(LOG_TAG_REPARTITION, "Erasing previous partition table at 0x%06X", NEW_PARTITION_TABLE_ADDR);
err = esp_flash_erase_region(esp_flash_default_chip, NEW_PARTITION_TABLE_ADDR, eraseLength);
if (ESP_OK == err)
{
ESP_LOGI(LOG_TAG_REPARTITION, "Partition table region erased");
ESP_LOGI(LOG_TAG_REPARTITION, "Writing new partition table");
err = esp_flash_write(esp_flash_default_chip, (const void *)partitionImageStart, NEW_PARTITION_TABLE_ADDR, partitionTableSize);
if (ESP_OK == err)
{
ESP_LOGI(LOG_TAG_REPARTITION, "Partition table sucessfully written");
// Update bootloader next to support flash enc
const esp_err_t bootloaderErr = BOOTLDR_Update();
if (ESP_OK != bootloaderErr)
{
ESP_LOGE(LOG_TAG_REPARTITION, "Bootloader update failed: %s", esp_err_to_name(bootloaderErr));
err = bootloaderErr;
}
}
else
{
ESP_LOGE(LOG_TAG_REPARTITION, "Failed to write partition table: %s", esp_err_to_name(err));
}
}
else
{
ESP_LOGE(LOG_TAG_REPARTITION, "Failed to erase previous partition table: %s", esp_err_to_name(err));
}
p_chip->os_func = p_originalFunctions;
ESP_LOGI(LOG_TAG_REPARTITION, "Flash region protection restored");
if (ESP_OK == err)
{
MigrationState_t nextState = { 0u };
nextState.magic = MIGRATION_STATE_MAGIC;
nextState.phase = (uint8_t)RESTORE_SPIFFS_DATA;
err = MigrationStateWrite(&nextState);
if (ESP_OK == err)
{
const esp_partition_t *const p_running = esp_ota_get_running_partition();
if (NULL == p_running)
{
ESP_LOGE(LOG_TAG_REPARTITION, "Failed to get running partition");
err = ESP_ERR_NOT_FOUND;
}
else
{
ESP_LOGI(LOG_TAG_REPARTITION, "Pinning boot to running partition: label=%s subtype=0x%02x addr=0x%06x",
p_running->label, p_running->subtype, (unsigned int)p_running->address);
const esp_err_t set_ret = esp_ota_set_boot_partition(p_running);
if (ESP_OK != set_ret)
{
ESP_LOGE(LOG_TAG_REPARTITION, "esp_ota_set_boot_partition failed: %s", esp_err_to_name(set_ret));
err = set_ret;
}
else
{
const esp_partition_t *const p_boot = esp_ota_get_boot_partition();
if (NULL == p_boot)
{
ESP_LOGE(LOG_TAG_REPARTITION, "Failed to read boot partition after set");
err = ESP_ERR_NOT_FOUND;
}
else
{
ESP_LOGI(LOG_TAG_REPARTITION, "Boot partition now: label=%s subtype=0x%02x addr=0x%06x",
p_boot->label, p_boot->subtype, (unsigned int)p_boot->address);
const esp_err_t mark_ret = esp_ota_mark_app_valid_cancel_rollback();
if (ESP_OK != mark_ret)
{
ESP_LOGW(LOG_TAG_REPARTITION, "esp_ota_mark_app_valid_cancel_rollback returned: %s",
esp_err_to_name(mark_ret));
}
if (ESP_OK == err)
{
ESP_LOGI(LOG_TAG_REPARTITION, "Migration state set to RESTORE_SPIFFS_DATA, restarting");
esp_restart();
}
}
}
}
}
}
return err;
}
Updating the bootloader is done through :
I also tried using esp_flash_write_encrypted instead of esp_flash_write and no luck, ROM Bootloader still wasn't able to read the second stage bootloader
#define BOOTLOADER_ERASE_START_SECTOR (0x1000)
#define BOOTLOADER_ERASE_SECTORS_NUM (9u)
#define BOOTLOADER_ERASE_SECTOR_SIZE (4096u)
#define BOOTLOADER_WRITE_START_ADDRESS (0x1000)
esp_err_t
BOOTLDR_Update(void)
{
esp_err_t err = ESP_OK;
// First check that new bootloader is written or not.
// This fuse gets written in the new bootloader.
if (false == esp_flash_encryption_enabled())
{
ESP_LOGI(LOG_TAG_BOOTLDR, "Updating bootloader to support flash enc");
ESP_LOGI(LOG_TAG_BOOTLDR, "new bootloader size to be written is %d bytes", (bootloader_image_end - bootloader_image_start));
esp_flash_t *p_chip = esp_flash_default_chip;
//Copy the existing hooks into our RAM instance
s_osFunctions = *p_chip->os_func;
// Patch out the protection check
s_osFunctions.region_protected = NULL;
//Point the driver at our patched table
p_chip->os_func = &s_osFunctions;
// erase old bootloader
ESP_LOGI(LOG_TAG_BOOTLDR, "erasing previous bootloader ...");
err = esp_flash_erase_region(esp_flash_default_chip, BOOTLOADER_ERASE_START_SECTOR, BOOTLOADER_ERASE_SECTOR_SIZE * BOOTLOADER_ERASE_SECTORS_NUM);
if (ESP_OK == err)
{
ESP_LOGI(LOG_TAG_BOOTLDR, "writing bootloader with flash enc support ...");
// write new bootloader
err = esp_flash_write(esp_flash_default_chip, (const void *)bootloader_image_start, BOOTLOADER_WRITE_START_ADDRESS,
(uint32_t)(bootloader_image_end - bootloader_image_start));
if (ESP_OK == err)
{
ESP_LOGI(LOG_TAG_BOOTLDR, "bootloader with flash enc support updated sucessfully");
}
else
{
ESP_LOGE(LOG_TAG_BOOTLDR, "failed to write bootloader with flash enc support :%d", err);
}
}
else
{
ESP_LOGE(LOG_TAG_BOOTLDR, "failed to erase previous bootloader :%d", err);
}
}
else
{
ESP_LOGI(LOG_TAG_BOOTLDR, "Flash enc already enabled, nothing to do");
}
return err;
}
The logs I am getting
I (1708) nvs: Generating NVS encr-keys...
I (3738) REPARTITION: Migration workflow triggered
I (3738) REPARTITION: Loading migration state from flash
I (3738) REPARTITION: Migration state valid (phase=1)
I (3748) REPARTITION: Migration state load finished (err=ESP_OK, valid=true)
I (3758) REPARTITION: Migration state load complete (valid=true)
I (3758) REPARTITION: Continuing migration from phase 1
I (3768) REPARTITION: Executing phase MOVE_PARTITION_TABLE
I (3778) REPARTITION: Starting partition table move (size=3072 bytes)
I (3778) REPARTITION: Temporarily disabled region protection for partition update
I (3788) REPARTITION: Erasing location 0x00C000 for the partition table
I (3798) REPARTITION: Partition table region erased
I (3798) REPARTITION: Writing new partition table
I (3818) REPARTITION: Partition table sucessfully written
I (3818) boot: Updating bootloader to support flash enc
I (3818) boot: new bootloader size to be written is 32768 bytes
I (3828) boot: erasing previous bootloader ...
I (4268) boot: writing bootloader with flash enc support ...
I (4578) boot: bootloader with flash enc support updated sucessfully
I (4578) REPARTITION: Flash region protection restored
I (4578) REPARTITION: Writing migration state (phase=2)
I (4588) REPARTITION: Computed migration state CRC=0x6CB5BF55
I (4638) REPARTITION: Migration state sector erased
I (4648) REPARTITION: Migration state written successfully
I (4648) REPARTITION: Pinning boot to running partition: label=ota_1 subtype=0x11 addr=0x410000
I (4648) esp_image: segment 0: paddr=00410020 vaddr=3f400020 size=81730h (530224) map
I (4828) esp_image: segment 1: paddr=00491758 vaddr=3ffbdb60 size=0591ch ( 22812)
I (4848) esp_image: segment 2: paddr=0049707c vaddr=40080000 size=08f9ch ( 36764)
I (4868) esp_image: segment 3: paddr=004a0020 vaddr=400d0020 size=d3268h (864872) map
I (5148) esp_image: segment 4: paddr=00573290 vaddr=40088f9c size=0c798h ( 51096)
I (5178) esp_image: segment 5: paddr=0057fa30 vaddr=00000000 size=005a0h ( 1440)
I (5188) esp_image: Verifying image signature...
I (5208) secure_boot_v2: Take trusted digest key(s) from eFuse block(s)
I (5208) secure_boot_v2: #0 app key digest == #0 trusted key digest
I (5208) secure_boot_v2: Verifying with RSA-PSS...
I (5278) secure_boot_v2_rsa: Signature verified successfully!
I (5338) REPARTITION: Boot partition now: label=ota_1 subtype=0x11 addr=0x410000
I (5388) REPARTITION: Migration state set to RESTORE_SPIFFS_DATA, restarting
ets Jul 29 2019 12:21:46
rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:2, clock div:2
secure boot v2 enabled
secure boot verification succeeded
load:0x3fff00c0 len:0xa8c
load:0x40078000 len:0x5170
load:0x40080400 len:0x4
--- 0x40080400: _init at ??:?
load:0x40080404 len:0xbe4
entry 0x4008057c
No signature block magic byte found at signature sector (found 0xff not 0xe7). Image not V2 signed?
ets Jul 29 2019 12:21:46