|
| 1 | +:orphan: |
| 2 | + |
| 3 | +mkbuildoptglobals.py internals |
| 4 | +============================== |
| 5 | + |
| 6 | +Sketch header aka SKETCH.ino.globals.h |
| 7 | +-------------------------------------- |
| 8 | + |
| 9 | +.. hint:: |
| 10 | + |
| 11 | + ``SKETCH.ino.globals.h`` is expected to be created by the user. |
| 12 | + |
| 13 | +It is always located in the root of the SKETCH directory, and must use the |
| 14 | +the actual name of the sketch program (``SKETCH.ino``) in the its name. |
| 15 | + |
| 16 | +Header file format is used because IDE only manages source files it actually |
| 17 | +recognizes as valid C / C++ file formats. When building and re-building the |
| 18 | +sketch, only valid file formats are taken into an account when building source |
| 19 | +code dependencies tree. |
| 20 | + |
| 21 | +Command-line options file |
| 22 | +------------------------- |
| 23 | + |
| 24 | + .. hint:: |
| 25 | + |
| 26 | + This file is created by the script. |
| 27 | + |
| 28 | +Options file is generated based on the contents of the sketch header comment block, |
| 29 | +and its contents are then used as gcc command-line options (``@file``) |
| 30 | + |
| 31 | + If file does not exist, or cannot be read, then the option will be treated literally, and not removed. |
| 32 | + |
| 33 | + Options in file are separated by whitespace. A whitespace character may be included |
| 34 | + in an option by surrounding the entire option in either single or double quotes. |
| 35 | + Any character (including a backslash) may be included by prefixing the character |
| 36 | + to be included with a backslash. |
| 37 | + The file may itself contain additional @file options; any such options will be processed recursively. |
| 38 | + |
| 39 | + --- https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html |
| 40 | + |
| 41 | + |
| 42 | +Arduino build system uses timestamps to determine which files should be rebuilt. |
| 43 | +``@file`` cannot be a direct dependency, only other source code files can. |
| 44 | + |
| 45 | +Thus, command-line options file is *always* created with at least one ``-include`` |
| 46 | + |
| 47 | +.. code-block:: console |
| 48 | +
|
| 49 | + -include "PLATFORM_PATH / COMMON_HEADER_PATH" |
| 50 | +
|
| 51 | +When matching directive is found in the sketch header, path to its copy in the build directory is also added |
| 52 | + |
| 53 | +.. code-block:: console |
| 54 | +
|
| 55 | + -include "BUILD_DIRECTORY / SKETCH_HEADER_COPY_PATH" |
| 56 | +
|
| 57 | +Common header |
| 58 | +------------- |
| 59 | + |
| 60 | +.. note:: |
| 61 | + |
| 62 | + This file is also created by the script. |
| 63 | + |
| 64 | +It is used as a means of triggering core rebuild, because modern Arduino build systems |
| 65 | +are agressively caching it and attempt to re-use existing ``core.a`` whenever possible. |
| 66 | + |
| 67 | +This file would contain path to the currently used command-line options file extracted |
| 68 | +from the sketch header. It remains empty otherwise. |
| 69 | + |
| 70 | +Build directory |
| 71 | +--------------- |
| 72 | + |
| 73 | +Arduino build process copies every valid source file from the source (sketch) |
| 74 | +directory into the build directory. This script is expected to be launched in |
| 75 | +the "prebuild" stage. At that point, build directory should already exist, but |
| 76 | +it may not yet contain any of the sketch source files. |
| 77 | + |
| 78 | +Script would always attempt to copy sketch header from the source (sketch) |
| 79 | +directory to the build one. If it does not exist, a placeholder would be created. |
| 80 | + |
| 81 | +Script would always synchronize atime & mtime of every file. When sketch header |
| 82 | +exists, stats are taken from it. When it doesn't, stats for the generated common |
| 83 | +header are used instead. |
| 84 | + |
| 85 | +Configuration |
| 86 | +------------- |
| 87 | + |
| 88 | +``platform.txt`` is expected to have this script listed as a tool |
| 89 | + |
| 90 | +.. code-block:: ini |
| 91 | +
|
| 92 | + runtime.tools.mkbuildoptglobals={runtime.platform.path}/tools/mkbuildoptglobals.py |
| 93 | +
|
| 94 | +Paths are always provided as Fully Qualified File Names (FQFNs): |
| 95 | + |
| 96 | +.. code-block:: ini |
| 97 | +
|
| 98 | + globals.h.source.fqfn={build.source.path}/{build.project_name}.globals.h |
| 99 | + globals.h.common.fqfn={build.core.path}/__common_globals.h |
| 100 | + build.opt.fqfn={build.path}/core/build.opt |
| 101 | + mkbuildoptglobals.extra_flags= |
| 102 | +
|
| 103 | +`"prebuild" hook <https://docs.arduino.cc/arduino-cli/platform-specification/#pre-and-post-build-hooks-since-arduino-ide-165>`__ must be used, |
| 104 | +allowing this script to run *before* build process creates and / or copies them. |
| 105 | +Both Arduino IDE 1.x and 2.x generate |
| 106 | +`prerequisite makefiles (files with a .d suffix) <https://www.gnu.org/software/make/manual/html_node/Automatic-Prerequisites.html>`__ |
| 107 | +at some point in "discovery phase". |
| 108 | + |
| 109 | +.. code-block:: ini |
| 110 | +
|
| 111 | + recipe.hooks.prebuild.#.pattern= |
| 112 | + "{runtime.tools.python3.path}/python3" -I |
| 113 | + "{runtime.tools.mkbuildoptglobals}" {mkbuildoptglobals.extra_flags} build |
| 114 | + --build-path "{build.path}" |
| 115 | + --build-opt "{build.opt.fqfn}" |
| 116 | + --sketch-header "{globals.h.source.fqfn}" |
| 117 | + --common-header "{commonhfile.fqfn}" |
| 118 | +
|
| 119 | +Command-line options file is then shared between other recipes by including it in |
| 120 | +the "cpreprocessor" flags. |
| 121 | + |
| 122 | +.. code-block:: ini |
| 123 | +
|
| 124 | + compiler.cpreprocessor.flags=... @{build.opt.path} ... |
| 125 | +
|
| 126 | +After that point, prerequisite makefiles should contain either only the common header, |
| 127 | +or both the common header and the build sketch header. When any of included headers is |
| 128 | +modified, every file in the dependency chain would be rebuilt. This allows us to keep |
| 129 | +existing ``core.a`` cache when command-line options file is not used by the sketch. |
| 130 | + |
| 131 | +Example |
| 132 | +------- |
| 133 | + |
| 134 | +Sketch header file with embedded command-line options file might look like this |
| 135 | + |
| 136 | +.. code-block:: c++ |
| 137 | + :emphasize-lines: 1,2,3,4,5,6,7,8,9,10,11 |
| 138 | + |
| 139 | + /*@create-file:build.opt@ |
| 140 | + // An embedded "build.opt" file using a "C" block comment. The starting signature |
| 141 | + // must be on a line by itself. The closing block comment pattern should be on a |
| 142 | + // line by itself. Each line within the block comment will be space trimmed and |
| 143 | + // written to build.opt, skipping blank lines and lines starting with '//', '*' |
| 144 | + // or '#'. |
| 145 | + -DMYDEFINE="\"Chimichangas do not exist\"" |
| 146 | + -O3 |
| 147 | + -fanalyzer |
| 148 | + -DUMM_STATS=2 |
| 149 | + */ |
| 150 | +
|
| 151 | + #ifndef SKETCH_INO_GLOBALS_H |
| 152 | + #define SKETCH_INO_GLOBALS_H |
| 153 | + |
| 154 | + #if defined(__cplusplus) |
| 155 | + // Defines kept private to .cpp modules |
| 156 | + //#pragma message("__cplusplus has been seen") |
| 157 | + #endif |
| 158 | + |
| 159 | + #if !defined(__cplusplus) && !defined(__ASSEMBLER__) |
| 160 | + // Defines kept private to .c modules |
| 161 | + #endif |
| 162 | + |
| 163 | + #if defined(__ASSEMBLER__) |
| 164 | + // Defines kept private to assembler modules |
| 165 | + #endif |
| 166 | + |
| 167 | + #endif |
| 168 | + |
| 169 | + |
| 170 | +Caveats, Observations, and Ramblings |
| 171 | +------------------------------------ |
| 172 | + |
| 173 | +1. Edits to ``platform.txt`` or ``platform.local.txt`` force a complete rebuild that |
| 174 | + removes the core folder. Not a problem, just something to be aware of when |
| 175 | + debugging this script. Similarly, changes on the IDE Tools selection cause a |
| 176 | + complete rebuild. |
| 177 | + |
| 178 | + In contrast, the core directory is not deleted when the rebuild occurs from |
| 179 | + changing a file with an established dependency (inspect .d in the build path) |
| 180 | + |
| 181 | +2. Renaming files does not change the last modified timestamp, possibly causing |
| 182 | + issues when adding or replacing files by renaming and rebuilding. |
| 183 | + |
| 184 | + A good example of this problem is when you correct the spelling of sketch |
| 185 | + header file. You must update mtime (e.g. call touch) of the file. |
| 186 | + |
| 187 | +3. ``-include ".../Sketch.ino.globals.h"`` is conditionally added to every compilation command, |
| 188 | + so it may be reasonable to expect that ``#include "Sketch.ino.globals.h"`` is no longer necessary. |
| 189 | + |
| 190 | + However, it may not be the case when `create-file:...` directive is missing or does not match. |
| 191 | + |
| 192 | + When explicit ``#include "Sketch.ino.globals.h"`` is used in the code, it must always be guarded against including it twice: |
| 193 | + |
| 194 | + .. code-block:: c++ |
| 195 | + :emphasize-lines: 1 |
| 196 | + |
| 197 | + #pragma once |
| 198 | + |
| 199 | + Or, by using classic header guards: |
| 200 | + |
| 201 | + .. code-block:: c++ |
| 202 | + :emphasize-lines: 1,2,4 |
| 203 | + |
| 204 | + #infdef SKETCH_GLOBALS_H |
| 205 | + #define SKETCH_GLOBALS_H |
| 206 | + ... file contents ... |
| 207 | + #endif |
| 208 | + |
| 209 | +4. ``build.opt`` itself is not listed as a dependency in .d, .h files are used |
| 210 | + because this is the only obvious way to force arduino-builder / arduino-cli |
| 211 | + into tracking it. |
| 212 | + |
| 213 | +5. When not using ``--build-path``, ``core.a`` is cached and shared. |
| 214 | + CI sometimes uses `ARDUINO_BUILD_CACHE_PATH environment variable <https://arduino.github.io/arduino-cli/1.2/configuration/>`__. |
| 215 | + This allows to have a private core cache, separate from the system one. |
| 216 | + |
| 217 | +6. `Referencing upstream arduino-cli code (v1.2.2) <https://github.com/arduino/arduino-cli/blob/1.2.2/internal/arduino/builder/core.go#L88-L110>`__, ``core.a`` cache key is based on: |
| 218 | + |
| 219 | + * `ESP8266 Platform Path`, and depends on installation method |
| 220 | + |
| 221 | + * `Installing ESP8266 Core <../installing.rst>`__ |
| 222 | + * `Arduino Platform Installation Directories <https://docs.arduino.cc/arduino-cli/platform-specification/#platform-installation-directories>`__ |
| 223 | + |
| 224 | + * `FQBN` |
| 225 | + |
| 226 | + See `Arduino Custom Board Options <https://docs.arduino.cc/arduino-cli/platform-specification/#custom-board-options>`__). |
| 227 | + |
| 228 | + * `Optimization flags` |
| 229 | + |
| 230 | + .. attention:: |
| 231 | + |
| 232 | + ``{compiler.optimization_flags}`` is not currently used in the ESP8266 Core |
| 233 | + |
| 234 | + See `Arduino Optimization Level setting <https://docs.arduino.cc/arduino-cli/platform-specification/#optimization-level-for-debugging>`__). |
| 235 | + |
0 commit comments