Better documentation for compiling optiboot

[mcuee]

I think Wiki is a good start as well as README.TXT.
https://github.com/Optiboot/optiboot/wiki/CompilingOptiboot
https://github.com/Optiboot/optiboot/blob/master/optiboot/bootloaders/optiboot/README.TXT

But there are a few questions.
Toolchain -- is WinAVR really supported now? Is Atmel toolchain (avr-gcc 5.4 based and 7.3 based) supported?

I believe WinAVR is no longer supported. So it is better to remove it from the Wiki.
I think Atmel toolchain links should be provided in the Wiki and they should work based on my testing.

https://www.microchip.com/en-us/tools-resources/develop/microchip-studio/gcc-compilers
Atmel version 3.7.0 --> avr-gcc 7.3.0
Atmel version 3.6.2 --> avr-gcc 5.4.0

README.TXT -- which part is still relevant?

Build environment under Windows -- is CMD really supported? What about PowerShell and MSYS2 mingw?

I have no issued building under Linux and macOS. Windows is not a big problem but the building process needs to be better documented. I need to use git bash to get it work or use MSYS2.

Build environment under macOS -- CrossPacks is too old to work for newe version of macOS. It is better to only mention Arduino toolchains which works out of the box.

[mcuee]

Example under Windows using CMD or PowerShell.

c:\work\avr\avrdude_test\others\optiboot\optiboot\bootloaders\optiboot>make
avr-gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

process_begin: CreateProcess(NULL, bash baudcheck.tmp.sh, ...) failed.
make (e=2): The system cannot find the file specified.
make: [Makefile:684: baudcheck] Error 2 (ignored)
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
 #warning BAUD_RATE off by greater than 2%
  ^~~~~~~
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num' -Wl,-Tlink_optiboot.ld -Wl,--relax -nostartfiles -o optiboot_virboot8.elf optiboot.o
avr-size optiboot_virboot8.elf
   text    data     bss     dec     hex filename
    602       0       0     602     25a optiboot_virboot8.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_virboot8.elf optiboot_virboot8.hex
avr-objdump -h -S optiboot_virboot8.elf > optiboot_virboot8.lst
rm optiboot.o

c:\work\avr\avrdude_test\others\optiboot\optiboot\bootloaders\optiboot>make clean
Too many tokens found on command line.  First unexpected token was
"__temp_.elf".
*** Program aborted on error. ***
make: *** [Makefile:699: clean] Error 3

c:\work\avr\avrdude_test\others\optiboot\optiboot\bootloaders\optiboot>make clean
Too many tokens found on command line.  First unexpected token was
"__temp_.elf".
*** Program aborted on error. ***
make: *** [Makefile:699: clean] Error 3

[mcuee]

Under Windows I need to use git bash to get it working,

$ make clean
rm -rf *.o *.elf *.lst *.map *.sym *.lss *.eep *.srec *.bin *.hex *.tmp.sh


$ make
avr-gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

BAUD RATE CHECK: Desired: 115200, Real: 117647, UBRRL = 16, Difference=2.1%
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
 #warning BAUD_RATE off by greater than 2%
  ^~~~~~~
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num' -Wl,-Tlink_optiboot.ld -Wl,--relax -nostartfiles -o optiboot_virboot8.elf optiboot.o
avr-size optiboot_virboot8.elf
   text    data     bss     dec     hex filename
    602       0       0     602     25a optiboot_virboot8.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_virboot8.elf optiboot_virboot8.hex
avr-objdump -h -S optiboot_virboot8.elf > optiboot_virboot8.lst
rm optiboot.o

[mcuee]

MSYS2 works as well with its bundled toolchain.

$ make clean
rm -rf *.o *.elf *.lst *.map *.sym *.lss *.eep *.srec *.bin *.hex *.tmp.sh

$ make
avr-gcc.exe (GCC) 8.5.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

BAUD RATE CHECK: Desired: 115200, Real: 117647, UBRRL = 16, Difference=2.1%
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
 #warning BAUD_RATE off by greater than 2%
  ^~~~~~~
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num' -Wl,-Tlink_optiboot.ld -Wl,--relax -nostartfiles -o optiboot_virboot8.elf optiboot.o
avr-size optiboot_virboot8.elf
   text    data     bss     dec     hex filename
    602       0       0     602     25a optiboot_virboot8.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_virboot8.elf optiboot_virboot8.hex
avr-objdump -h -S optiboot_virboot8.elf > optiboot_virboot8.lst
rm optiboot.o

$ avr-gcc -v
Using built-in specs.
Reading specs from c:/msys64/mingw64/bin/../lib/gcc/avr/8.5.0/device-specs/specs-avr2
COLLECT_GCC=C:\msys64\mingw64\bin\avr-gcc.exe
COLLECT_LTO_WRAPPER=c:/msys64/mingw64/bin/../libexec/gcc/avr/8.5.0/lto-wrapper.exe
Target: avr
Configured with: ../gcc-8.5.0/configure --build=x86_64-w64-mingw32 --prefix=/mingw64 --target=avr --enable-languages=c,c++ --disable-nls --disable-libssp --disable-shared --disable-threads --disable-libgomp --disable-libada --with-dwarf2 --enable-mingw-wildcard
Thread model: single
gcc version 8.5.0 (GCC)

[mcuee]

And I believe WinAVR is no longer supported.

$ make clean
rm -rf *.o *.elf *.lst *.map *.sym *.lss *.eep *.srec *.bin *.hex *.tmp.sh

$ make
avr-gcc (WinAVR 20100110) 4.3.3
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

BAUD RATE CHECK: Desired: 115200, Real: 117647, UBRRL = 16, Difference=2.1%
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2%
optiboot.c:498: warning: 'leaf' attribute directive ignored
optiboot.c:499: warning: 'leaf' attribute directive ignored
optiboot.c: In function 'main':
optiboot.c:830: error: 'EE_RDY_vect_num' undeclared (first use in this function)
optiboot.c:830: error: (Each undeclared identifier is reported only once
optiboot.c:830: error: for each function it appears in.)
optiboot.c: In function 'read_mem':
optiboot.c:1535: error: 'EE_RDY_vect_num' undeclared (first use in this function)
make: *** [<builtin>: optiboot.o] Error 1

[mcuee]

For macOS, I have encountered issues with Atmel provided toolchain due to macOS security thingy.

I have figured the solution.

• Better documentation of bootloader firmware building process stefanrueger/urboot#8 (comment)
• https://support.blackfire.io/en/articles/3669492-issues-with-macos-catalina

[mcuee]

It would also be good to have some benchmarks about using different toolchains. I've seen some old 2018 threads with some data. It would be good to compare the code size generated using different version of the toolchain, at least the avr-gcc 5.4 based version and 7.3 based version.

Example output for hex files generated by different toolchain using makeall.arduino.sh. We can see that older avr-gcc 5.4.0 based version will usually produce the smallest code size with some exceptions. And avr-gcc 12.1.0 usually produces the biggest code size with a few exceptions.

hexls is a neat utility from urboot project.
https://github.com/stefanrueger/urboot/blob/main/src/hexls

1. Using avr-gcc 5.4.0 Atmel toolchain (overall best)
2. Using avr-gcc 7.3, current version of Arduino toolchain or Atmel toolchain
3. MSYS2 mingw64 avr-gcc 8.5.0 (overall second best)
4. avr-gcc 12.1.0. (overall worst)

hex file	Size (5.4.0)	Space (5.4.0)	Size (7.3.0)	Space (7.3.0)	Size (8.5.0)	Space (8.5.0)	Size (12.1.0)	Space (12.1.0)
optiboot_atmega1280.hex	787	1024	789	1024	791	1024	783	1024
optiboot_atmega1284p.hex	784	1024	790	1024	792	1024	788	1024
optiboot_atmega168.hex	474	512	490	512	476	512	494	512
optiboot_atmega168p_16MHz.hex	474	512	490	512	476	512	494	512
optiboot_atmega168p_UART__.hex	474	512	490	512	476	512	494	512
optiboot_atmega32.hex	458	512	466	512	470	512	484	512
optiboot_atmega328.hex	474	512	490	512	476	512	494	512
optiboot_atmega328_pro_8MHz.hex	474	512	490	512	476	512	494	512
optiboot_atmega8.hex	458	512	466	512	470	512	484	512
optiboot_atmega88.hex	474	512	490	512	476	512	494	512
optiboot_bobuino.hex	784	1024	790	1024	792	1024	788	1024
optiboot_diecimila.hex	474	512	490	512	476	512	494	512
optiboot_lilypad.hex	474	512	490	512	476	512	494	512
optiboot_luminet.hex	556	576	564	576	564	576	572	576
optiboot_pro_16MHz.hex	474	512	490	512	476	512	494	512
optiboot_pro_20mhz.hex	474	512	490	512	476	512	494	512
optiboot_pro_8MHz.hex	474	512	490	512	476	512	494	512
optiboot_sanguino.hex	747	1024	749	1024	751	1024	747	1024
optiboot_wildfirev2.hex	785	1024	791	1024	793	1024	789	1024
optiboot_xplained168pb.hex	474	512	490	512	476	512	494	512
optiboot_xplained328p.hex	474	512	490	512	476	512	494	512
optiboot_xplained328pb.hex	474	512	490	512	476	512	494	512

[mcuee]

The older toolchain does not work with the custom linker script, so I checked out one commit earlier than that (68b7a55)

This may not be comparable with the above git main results though.

1. Using avr-gcc 4.8.1 from urboot under Linux.
2. Using the 4.9.2 toolchain from urboot under Linux.
3. Using avr-gcc 5.4.0 Atmel toolchain (overall best)
4. avr-gcc 7.3 Current version of Arduino toolchain or Atmel toolchain
5. MSYS2 mingw64 avr-gcc 8.5.0 (overall second best)
6. avr-gcc 12.1.0. (overall worst)
   

hex file	Size (4.8.1)	Space (4.8.1)	Size (4.9.2)	Space (4.9.2)	Size (5.4.0)	Space (5.4.0)	Size (7.3.0)	Space (7.3.0)	Size (8.5.0)	Space (8.5.0)	Size (12.1.0)	Space (12.1.0)
optiboot_atmega1280.hex	818	1024	823	1024	797	1024	795	1024	798	1024	792	1024
optiboot_atmega1284p.hex	816	1024	820	1024	792	1024	796	1024	798	1024	794	1024
optiboot_atmega168.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega168p_16MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega168p_UART__.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega32.hex	470	512	466	512	464	512	472	512	476	512	490	512
optiboot_atmega328.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega328_pro_8MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega8.hex	472	512	468	512	466	512	472	512	476	512	490	512
optiboot_atmega88.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_bobuino.hex	816	1024	820	1024	792	1024	796	1024	798	1024	794	1024
optiboot_diecimila.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_lilypad.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_luminet.hex	570	640	570	640	564	640	570	640	570	640	578	640
optiboot_pro_16MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_pro_20mhz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_pro_8MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_sanguino.hex	776	1024	775	1024	755	1024	755	1024	758	1024	754	1024
optiboot_wildfirev2.hex	817	1024	821	1024	793	1024	797	1024	800	1024	796	1024
optiboot_xplained168pb.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_xplained328p.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_xplained328pb.hex	490	512	488	512	482	512	496	512	482	512	500	512

7. WinAVR gcc version 4.3.3 (WinAVR 20100110) can only build a few hex files.

$ ./hexls
856 1024 o8.2 -.s-.-r-- optiboot_atmega1280.hex
856 1024 o8.2 -.s-.-r-- optiboot_atmega1284p.hex
494 512 o8.2 -.s-.-r-- optiboot_atmega32.hex
494 512 o8.2 -.s-.-r-- optiboot_atmega8.hex
856 1024 o8.2 -.s-.-r-- optiboot_bobuino.hex
816 1024 o8.2 -.s-.-r-- optiboot_sanguino.hex
857 1024 o8.2 -.s-.-r-- optiboot_wildfirev2.hex

[mcuee]

Nice refernce from here, back in August 2018.
#142 (comment)

Atmel avr-gcc 5.4.0 based toolchain was the winner at that time. I think it is still the overall best now.

target \ compiler	1.0.6	1.6.9	1.6.13	1.8.5	microchip avr-gcc 5.4
atmega128	672	654	666	666	642
atmega1280	680	666	680	680	654
atmega1281	684	666	678	678	652
atmega1284	682	662	674	674	646
atmega1284p	682	662	674	674	646
atmega16	474	436	434	434	432
atmega162	478	434	432	432	426
atmega164a	x	456	454	454	448
atmega164p	x	456	454	454	448
atmega168	502	456	454	454	448
atmega168p	502	456	454	454	448
atmega168pb LED=B5	x	456	454	454	448
atmega169	500	454	452	452	446
atmega169p	500	454	452	452	446
atmega2560	684	666	678	678	652
atmega2561	684	666	678	678	652
atmega32	478	438	434	434	432
atmega324a	x	456	454	454	448
atmega324p	x	456	454	454	448
atmega324pa	x	454	452	452	446
atmega324pb	x	x	x	x	x
atmega328	502	456	454	454	448
atmega328 BIGBOOT=1	590	576	578	578	570
atmega328 LED_START_FLASHES=0	456	410	410	410	404
atmega328 LED_START_FLASHES=0 LED_START_ON=1 SUPPORT_EEPROM=1	x	512	x	x	508
atmega328 LED_START_FLASHES=0 SUPPORT_EEPROM=1	x	508	512	512	504
atmega328 SOFT_UART=1	502	458	458	458	452
atmega328_pro8	502	456	454	454	448
atmega328pb LED=B5	x	x	x	x	x
atmega329	498	454	454	454	448
atmega3290	502	456	454	454	448
atmega3290p	502	456	454	454	448
atmega329p	498	456	454	454	448
atmega64	632	610	618	618	598
atmega640	x	662	674	674	648
atmega644p	644	620	628	628	608
atmega644p	644	620	628	628	608
atmega649	638	614	624	624	604
atmega6490	642	618	626	626	606
atmega6490p	x	618	626	626	606
atmega649p	x	618	626	626	606
atmega8	474	436	432	432	430
atmega8515	x	438	434	434	432
atmega8535	x	438	434	434	432
atmega88	500	456	454	454	448
atmega88p LED=B5	500	456	454	454	448
atmega88pb LED=B5	x	456	454	454	448
attiny1634	x	546	546	546	536
attiny828	x	x	x	x	x
attiny84	660	546	546	546	548
attiny841	x	580	578	578	576
bobuino	682	662	674	674	646
diecimila	502	456	454	454	448
lilypad	502	456	454	454	448
lilypad_resonator	502	456	454	454	448
luminet	660	546	546	546	548
mega1280	680	666	680	680	654
mega2560	684	666	678	678	652
mighty1284	682	662	674	674	646
pro16	502	456	454	454	448
pro20	502	456	454	454	448
pro8	502	456	454	454	448
sanguino	644	620	628	628	608
virboot328	x	570	570	570	560
virboot8	x	572	568	568	570
wildfire	682	662	674	674	646
wildfirev2	682	662	674	674	646
wildfirev3	682	662	674	674	646
xplained168pb	502	456	454	454	448
xplained328p	502	456	454	454	448
xplained328pb	502	456	454	454	448

[mcuee]

@WestfW @MCUdude and @stefanrueger

Maybe you will be interested in the results above.

Interestingly the conclusion is a bit different from urboot test results below. For urboot, 4.8.1 performs better than 5.4.0.

• Better documentation of bootloader firmware building process stefanrueger/urboot#8 (comment)

[hecko]

Im happy to help here in some capacity, however It seems that at the moment there is way too many combinations of compilers , compiler versions, environments and target platforms to make some baseline for CI/CD. Im kind of inclined to suggest having one or two environments fully officially supported and clean-up the whole repo, Makefiles etc.

The Microchip version of gcc toolchain for Linux seems like one of those envs that make sense to be officially supported by Optiboot. Im not sure about the other ones as it seems they lack some uC support that Microchip version has (like atmega328pb).

I have made a reference Dockerfile for compiling against Microchip toolchain in https://github.com/hecko/optiboot_docker_build - this can be easily ran using GitHub Actions and hex files can be automagically published / analyzed somewhere.

Someone authoritative should really make a decision here. Especially now that HEAD version of Optiboot has different compilation options to the 8.0 version that will definitely confuse many people / compilation systems once 9.0 or whatever is released.

[mcuee]

Someone authoritative should really make a decision here. Especially now that HEAD version of Optiboot has different compilation options to the 8.0 version that will definitely confuse many people / compilation systems once 9.0 or whatever is released.

@WestfW
It seems that the current git head is of version 8.3, just wondering if you want release it as Optiboot 8.3. That may help to reduce the confusions.