Skip to content

Commit 25e2761

Browse files
michalek-nonvlsianpu
michalek-no
authored and
nvlsianpu
committed
doc: compression format
describes internals of signed image with compressed payload. Signed-off-by: Mateusz Michalek <mateusz.michalek@nordicsemi.no> Signed-off-by: Anna Wojdylo <anna.wojdylo@nordicsemi.no>
1 parent 724ee7e commit 25e2761

File tree

3 files changed

+263
-35
lines changed

3 files changed

+263
-35
lines changed

‎docs/compression_format.md

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# Compressed binary file internals
2+
3+
This article describes the structure of the
4+
`zephyr.signed.bin` file when image
5+
compression is enabled. You do not need to know these details to use the
6+
image compression subsystem, but they can be beneficial if you want to
7+
use them for verification or custom integration purposes.
8+
9+
For an example, see the following structure of the file:
10+
11+
![LZMA header](./images/decomp.png)
12+
13+
## [LZMA Header](#LZMA-Header)
14+
15+
The Lempel-Ziv-Markov chain Algorithm (LZMA) header is crucial for files
16+
compressed using the LZMA method. It contains metadata essential for
17+
decompression. The `lzma2_header` encodes compression parameters using
18+
two bytes.
19+
20+
### [Calculating compression parameters](#Calculating-compression-parameters)
21+
22+
Compression parameters can be calculated, retrieved, or changed
23+
depending on your needs. For details, see the following sections.
24+
25+
#### [Default values](#Default-values)
26+
27+
Compression parameters have the following default values:
28+
29+
- `dict_size`: 131072
30+
- `pb`: 2
31+
- `lc`: 3
32+
- `lp`: 1
33+
34+
#### [Adjusting dictionary size](#Adjusting-dictionary-size)
35+
36+
You can calculate the `dict_size` using the following method:
37+
38+
``` {.c}
39+
unsigned int i = 0;
40+
41+
for (i = 0; i < 40; i++) {
42+
if (raw_dict_size <= (((uint32_t)2 | ((i) & 1)) << ((i) / 2 + 11))) {
43+
break;
44+
}
45+
}
46+
dict_size = (uint8_t)i;
47+
```
48+
49+
With this method, `dict_size` can have one of the following values:
50+
51+
|Hex Value | Size |
52+
|-----------|------------|
53+
|0x00 | 4096 |
54+
|0x01 | 6144 |
55+
|0x02 | 8192 |
56+
|0x03 | 12288 |
57+
|0x04 | 16384 |
58+
|0x05 | 24576 |
59+
|0x06 | 32768 |
60+
|0x07 | 49152 |
61+
|0x08 | 65536 |
62+
|0x09 | 98304 |
63+
|0x0a | 131072 |
64+
|0x0b | 196608 |
65+
|0x0c | 262144 |
66+
|0x0d | 393216 |
67+
|0x0e | 524288 |
68+
|0x0f | 786432 |
69+
|0x10 | 1048576 |
70+
|0x11 | 1572864 |
71+
|0x12 | 2097152 |
72+
|0x13 | 3145728 |
73+
|0x14 | 4194304 |
74+
|0x15 | 6291456 |
75+
|0x16 | 8388608 |
76+
|0x17 | 12582912 |
77+
|0x18 | 16777216 |
78+
|0x19 | 25165824 |
79+
|0x1a | 33554432 |
80+
|0x1b | 50331648 |
81+
|0x1c | 67108864 |
82+
|0x1d | 100663296 |
83+
|0x1e | 134217728 |
84+
|0x1f | 201326592 |
85+
|0x20 | 268435456 |
86+
|0x21 | 402653184 |
87+
|0x22 | 536870912 |
88+
|0x23 | 805306368 |
89+
|0x24 | 1073741824 |
90+
|0x25 | 1610612736 |
91+
|0x26 | 2147483648 |
92+
|0x27 | 3221225472 |
93+
94+
#### [Calculating literal context, literal pos, and pos bits](#Calculating-literal-context-literal-pos-and-pos-bits)
95+
96+
The second byte of the `lzma2_header` carries the following parameters:
97+
98+
- `lc`, which specifies a number of literal context bits
99+
100+
- `lp`, which specifies a number of literal pos bits
101+
102+
- `pb`, which specifies a number of pos bits
103+
104+
These parameters are encoded with the following formula:
105+
106+
``` {.c}
107+
pb_lp_lc = (uint8_t)((pb * 5 + lp) * 9 + lc);
108+
```
109+
110+
To decode these values from the combined `pb_lp_lc` byte, run the
111+
following code:
112+
113+
``` {.c}
114+
lc = pb_lp_lc % 9;
115+
pb_lp_lc /= 9;
116+
pb = pb_lp_lc / 5;
117+
lp = pb_lp_lc % 5;
118+
```
119+
120+
## [Extracting LZMA stream from image](#Extracting-LZMA-stream-from-image)
121+
122+
To extract and decompress the LZMA stream from the image, follow these
123+
steps:
124+
125+
1. Determine the offset of the compressed stream by adding the
126+
`lzma2_header` size and the value stored under
127+
`image_header.ih_hdr_size`. For the size of the compressed stream,
128+
see `image_header.ih_img_size`.
129+
2. If the compressed stream is isolated and stored in a file named
130+
`raw.lzma`, you can perform
131+
decompression using the following commands:
132+
133+
- Without an ARM thumb filter:
134+
135+
``` {.bash}
136+
unlzma --lzma2 --format=raw --suffix=.lzma raw.lzma
137+
```
138+
139+
- With an ARM thumb filter:
140+
141+
``` {.bash}
142+
unlzma --armthumb --lzma2 --format=raw --suffix=.lzma raw.lzma
143+
```
144+
145+
Once the command is executed you will see a newly created file named
146+
`raw`, which is identical to the
147+
image before compression.
148+
149+
## [TLVs](#TLVs)
150+
151+
The following Type-Length-Values (TLVs) are used in the context of
152+
decompressed images:
153+
154+
- `DECOMP_SIZE (0x70)`: Specifies the size of the decompressed image.
155+
- `DECOMP_SHA (0x71)`: Contains the hash of the decompressed image.
156+
- `DECOMP_SIGNATURE (0x72)`: Holds the signature of either the hash or
157+
the entire image.
158+
159+
These TLVs are placed in the protected TLV section, ensuring they are
160+
included in the hashing and signature calculations during the
161+
verification process. The process for choosing the type of cryptographic
162+
signature and hash algorithm used for securing the image is the same,
163+
regardless of whether the image has undergone compression.
164+
165+
## [Sample](#Sample)
166+
167+
For practical implementation, you can find a simple stand-alone
168+
verification program under the following path
169+
`bootloader/mcuboot/samples/compression_test/independent_cmp.c`
170+
171+
This program demonstrates how to independently verify the integrity and
172+
authenticity of a decompressed image using the specified TLVs.

‎docs/images/decomp.png

40.7 KB
Loading

‎docs/imgtool.md

Lines changed: 91 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -51,50 +51,100 @@ enabled, this last step is unnecessary and can be skipped.
5151
Image signing takes an image in binary or Intel Hex format intended for the
5252
primary slot and adds a header and trailer that the bootloader is expecting:
5353

54-
Usage: imgtool.py sign [OPTIONS] INFILE OUTFILE
54+
Usage: imgtool sign [OPTIONS] INFILE OUTFILE
5555

5656
Create a signed or unsigned image
5757

5858
INFILE and OUTFILE are parsed as Intel HEX if the params have .hex
5959
extension, otherwise binary format is used
6060

6161
Options:
62+
--vector-to-sign [payload|digest]
63+
send to OUTFILE the payload or payloads
64+
digest instead of complied image. These data
65+
can be used for external image signing
66+
--sha [auto|256|384|512] selected sha algorithm to use; defaults to
67+
"auto" which is 256 if no cryptographic
68+
signature is used, or default for signature
69+
type
70+
--sig-out filename Path to the file to which signature will be
71+
written. The image signature will be encoded
72+
as base64 formatted string
73+
--pure Expected Pure variant of signature; the Pure
74+
variant is expected to be signature done
75+
over an image rather than hash of that
76+
image.
77+
--fix-sig-pubkey filename public key relevant to fixed signature
78+
--fix-sig filename fixed signature for the image. It will be
79+
used instead of the signature calculated
80+
using the public key
6281
-k, --key filename
6382
--public-key-format [hash|full]
64-
--align [1|2|4|8|16|32] Alignment used by swap update modes.
65-
-v, --version TEXT [required]
66-
-s, --security-counter TEXT Specify the value of security counter. Use
67-
the `auto` keyword to automatically generate
68-
it from the image version.
69-
-d, --dependencies TEXT
70-
--pad-sig Add 0-2 bytes of padding to ECDSA signature
71-
(for MCUboot <1.5)
72-
-H, --header-size INTEGER [required]
73-
--pad-header Add --header-size zeroed bytes at the
74-
beginning of the image
75-
-S, --slot-size INTEGER Size of the slot where the image will be
76-
written [required]
77-
--pad Pad image to --slot-size bytes, adding
78-
trailer magic
79-
--confirm When padding the image, mark it as confirmed
80-
-M, --max-sectors INTEGER When padding allow for this amount of
81-
sectors (defaults to 128)
82-
--boot-record sw_type Create CBOR encoded boot record TLV. The
83-
sw_type represents the role of the software
84-
component (e.g. CoFM for coprocessor
85-
firmware). [max. 12 characters]
86-
--overwrite-only Use overwrite-only instead of swap upgrades
87-
-e, --endian [little|big] Select little or big endian
88-
-E, --encrypt filename Encrypt image using the provided public key
89-
--save-enctlv When upgrading, save encrypted key TLVs
90-
instead of plain keys. Enable when
91-
BOOT_SWAP_SAVE_ENCTLV config option was set.
92-
-L, --load-addr INTEGER Load address for image when it should run
93-
from RAM.
94-
-x, --hex-addr INTEGER Adjust address in hex output file.
95-
-R, --erased-val [0|0xff] The value that is read back from erased
96-
flash.
97-
-h, --help Show this message and exit.
83+
In what format to add the public key to the
84+
image manifest: full key or hash of the key.
85+
--max-align [8|16|32] Maximum flash alignment. Set if flash
86+
alignment of the primary and secondary slot
87+
differ and any of them is larger than 8.
88+
--align [1|2|4|8|16|32] Alignment used by swap update modes.
89+
-v, --version TEXT [required]
90+
-s, --security-counter TEXT Specify the value of security counter. Use
91+
the `auto` keyword to automatically generate
92+
it from the image version.
93+
-d, --dependencies TEXT Add dependence on another image, format:
94+
"(<image_ID>,<image_version>), ... "
95+
--pad-sig Add 0-2 bytes of padding to ECDSA signature
96+
(for mcuboot <1.5)
97+
-H, --header-size INTEGER [required]
98+
--pad-header Add --header-size zeroed bytes at the
99+
beginning of the image
100+
-S, --slot-size INTEGER Size of the slot. If the slots have
101+
different sizes, use the size of the
102+
secondary slot. [required]
103+
--pad Pad image to --slot-size bytes, adding
104+
trailer magic
105+
--confirm When padding the image, mark it as confirmed
106+
(implies --pad)
107+
-M, --max-sectors INTEGER When padding allow for this amount of
108+
sectors (defaults to 128)
109+
--boot-record sw_type Create CBOR encoded boot record TLV. The
110+
sw_type represents the role of the software
111+
component (e.g. CoFM for coprocessor
112+
firmware). [max. 12 characters]
113+
--overwrite-only Use overwrite-only instead of swap upgrades
114+
-e, --endian [little|big] Select little or big endian
115+
-c, --clear Output a non-encrypted image with encryption
116+
capabilities,so it can be installed in the
117+
primary slot, and encrypted when swapped to
118+
the secondary.
119+
--skip-encryption Set encryption flags and TLV's without
120+
applying encryption.
121+
--compression [disabled|lzma2|lzma2armthumb]
122+
Enable image compression using specified
123+
type. Will fall back without image
124+
compression automatically if the compression
125+
increases the image size.
126+
--encrypt-keylen [128|256] When encrypting the image using AES, select
127+
a 128 bit or 256 bit key len.
128+
-E, --encrypt filename Encrypt image using the provided public key.
129+
(Not supported in direct-xip or ram-load
130+
mode.)
131+
--save-enctlv When upgrading, save encrypted key TLVs
132+
instead of plain keys. Enable when
133+
BOOT_SWAP_SAVE_ENCTLV config option was set.
134+
-F, --rom-fixed INTEGER Set flash address the image is built for.
135+
-L, --load-addr INTEGER Load address for image when it should run
136+
from RAM.
137+
-x, --hex-addr INTEGER Adjust address in hex output file.
138+
-R, --erased-val [0|0xff] The value that is read back from erased
139+
flash.
140+
--custom-tlv [tag] [value] Custom TLV that will be placed into
141+
protected area. Add "0x" prefix if the value
142+
should be interpreted as an integer,
143+
otherwise it will be interpreted as a
144+
string. Specify the option multiple times to
145+
add multiple TLVs.
146+
--non-bootable Mark the image as non-bootable.
147+
-h, --help Show this message and exit.
98148

99149
The main arguments given are the key file generated above, a version
100150
field to place in the header (1.2.3 for example), the alignment of the
@@ -111,6 +161,12 @@ the load address (in Intel Hex terms, the Extended Linear Address record) to
111161
adjust for the new bytes prepended to the file. The load address of all data
112162
existing in the file should not change.
113163

164+
The `--compression` option enables LZMA compression over payload. Details
165+
about internals of image generated with this option can be found here
166+
[here](./compression_format.md)
167+
This isn't fully supported on the embedded side but can be utilised when
168+
project is built on top of the mcuboot.
169+
114170
The `--slot-size` argument is required and used to check that the firmware
115171
does not overflow into the swap status area (metadata). If swap upgrades are
116172
not being used, `--overwrite-only` can be passed to avoid adding the swap

0 commit comments

Comments
 (0)