Target mcuboot-image does not upload the signed executable

[cfoucher-laas]

Describe the bug
On STSTM32 platform, we use the mcuboot-image target to sign the image after building. We had trouble making this target to work flawlessly, as while the firmware is correctly built, it is the pre-mcuboot firmware that gets uploaded to the board. We finally found some workarounds, but they all have drawbacks. Here are the different steps we took:

First try

We started by using the following parameters in platformio.ini:

[env]
board_build.zephyr.bootloader.header_len = 0x200
board_build.zephyr.bootloader.flash_alignment = 8
board_build.zephyr.bootloader.slot_size = 0x37800
board_build.zephyr.bootloader.secondary_slot = 1
board_upload.offset_address = 0x8047800

targets = mcuboot-image

In this case, the firmware is correctly built, but the upload phase will not upload the signed firmware (firmware.mcuboot.bin), only the pre-mcuboot file firmware.bin. At this stage, the workaround was to use an external tool to upload the firmware to the board.

Second try

We identified the following code snippet in STSTM32's main.py:

    if "zephyr" in frameworks and "mcuboot-image" in COMMAND_LINE_TARGETS:
        target_firm = env.MCUbootImage(
            join("$BUILD_DIR", "${PROGNAME}.mcuboot.bin"), target_firm)

[...]

upload_source = target_firm

This lead us to a second try by ensuring the upload phase is done in a single step with the build/sign process. Here, we modified the targets in platformio.ini as follows:

targets = mcuboot-image,upload

There, it works but this is an akward configuration because the "build" button in the bottom bar of VS Code will also upload the file (thus would result in an error when not connected to the board), while the "upload" button still uploads the wrong file.

Third try

Finally, we resorted in a trick consisting in forcing pio scripts use the mcuboot-image even with the single "upload" step, by altering the COMMAND_LINE_TARGETS variable. This uses a "pre" hook as follows:

extra_scripts = pre:upload_signed_firmware.py
targets = mcuboot-image

With the following content in upload_signed_firmware.py:

Import("env")

# Make sure mcuboot-image is in the target list when uploading,
# because pio scripts will behave incorrectly if it isn't
# (wrong file gets uploaded)
if "mcuboot-image" not in COMMAND_LINE_TARGETS:
	COMMAND_LINE_TARGETS.insert(0, "mcuboot-image")

To Reproduce
Steps to reproduce the behavior:

1. Add targets = mcuboot-image to platformio.ini
2. Click on the "build" button in VS Code and observe that it produces a file firmware.mcuboot.bin
3. Click on the "upload" button in VS Code
4. Watch the uploader upload the wrong file firmware.bin

Expected behavior
When using the mcuboot-image target in platformio.ini, the "upload" phase should upload the final, signed, firmware, not the one produced at an earlier phase of the build process.

Impact
The impact was a time-consuming process of going through the pio scripts and understanding Scons in order to find a workaround. Now that it works, the impact is minimal (only an additional script to ship in our repo).

Environment

• OS: Linux
• Toolchain: Platformio toolchain within the STSTM32 platform, using VS Code
• Commit SHA or Version used: framework-zephyr version 2.30400.230914

[valeros]

Hi @cfoucher-laas, that's somewhat expected behavior. When you press the Upload button in the IDE, PlatformIO CLI is invoked with the only --target upload argument. The CLI options have a higher priority, so the targets = mcuboot-image option set in your platformio.ini is overridden. Your workaround is just fine, as an alternative you can simply add a new environment with proper target set by default:

[env]
...

[env:upload_mcuboot_image]
extends = env:your_default_env_name
targets = mcuboot-image, upload

[cfoucher-laas]

@valeros: thank you for this quick answer. What I understood from this dive in pio scripts is that the targets option in platformio.ini actually refers to build targets only. Perhaps it would be useful to allow describing upload targets in platformio.ini as well?

Also, can you confirm me that I'm not breaking anything else by overriding the COMMAND_LINE_TARGETS variable?

[valeros]

What I understood from this dive in pio scripts is that the targets option in platformio.ini actually refers to build targets only.

The targets option was introduced to allow developers set default targets per environment, not necessarily "build" targets as you can see there is the upload target in the code snippet above targets = mcuboot-image, upload.

Also, can you confirm me that I'm not breaking anything else by overriding the COMMAND_LINE_TARGETS variable?

SCons documentation doesn't forbid it explicitly, so I believe you should be fine, although I'd recommend using the approach with a separate environment and default targets or a precise CLI command pio run -t mcuboot-image -t upload in the IDE terminal.

[cfoucher-laas]

I didn't include the full platformio.ini file I work with in the issue, but I already defined various targets. Here is a longer excerpt from this file:

[platformio]
default_envs = spin_bootloader
boards_dir = zephyr/boards

[env]
platform = [email protected]
framework = zephyr

[bootloader]
board_build.zephyr.bootloader.header_len = 0x200
board_build.zephyr.bootloader.flash_alignment = 8
board_build.zephyr.bootloader.slot_size = 0x37800
board_build.zephyr.bootloader.secondary_slot = 1
board_upload.offset_address = 0x8047800

extra_scripts = pre:zephyr/scripts/upload_signed_firmware.py
targets = mcuboot-image

[env:spin]
board = spin

[env:spin_bootloader]
extends = bootloader
board = spin

[env:nucleo_g474re]
board = nucleo_g474re

[env:nucleo_g474re_bootloader]
extends = bootloader
board = nucleo_g474re

As you can see, we already define multiple targets, so we are fine working in command line. The issue arises when we use the VS Code buttons provided by pio, which are very handy for us as we work with people not familiar with command line. In that case, the "build" button correctly interprets the targets option, but the "upload" button completely ignores it.

This is why I think it would be a pretty good feature to allow defining targets for the default upload process in pio configuration file.

[valeros]

The issue arises when we use the VS Code buttons provided by pio, which are very handy for us as we work with people not familiar with command line.

Your workaround is just fine for such cases, if it also won't work for any reason, you can always fall back to a standalone environment with two default targets targets = mcuboot-image, upload as I mentioned above.

This is why I think it would be a pretty good feature to allow defining targets for the default upload process in pio configuration file.

That's the goal the extra scripts were designed for, if the default behavior of PlatformIO doesn't suit your needs, you can fine tune it any way you like.

[cfoucher-laas]

Thank you for these precisions regarding the intended behavior, this is now more clear to me. I understand that this is not a bug, but the intended behavior as per the way the tool is built.

However, let aside these VS Code related stuff, I cannot help to think that even in command line, a user would expect that using the target mcuboot-image, then the target upload will behave the same as using the targets mcuboot-image, upload at once. In any case, I think a user building the image using this specific target probably intends at using the signed firmware on the board, and people often decouple the build process from the upload process.

Hence, I had a few other ideas that may be more in line with the project guidelines. I apologize if I'm bothering you, but only see my insistence as aiming at improving things for other users that could fall in the same pitfall as me.

Let me expose my ideas to you, then I promise I'll stop bothering you.

Idea 1: Replace/rename the file in the process

In STSTM32 platform, the mcuboot-image target is defined as "do a build, then do an extra step with MCUBoot", as stated by this cone snippet:

    target_elf = env.BuildProgram()
    target_firm = env.ElfToBin(join("$BUILD_DIR", "${PROGNAME}"), target_elf)

    if "zephyr" in frameworks and "mcuboot-image" in COMMAND_LINE_TARGETS:
        target_firm = env.MCUbootImage(
            join("$BUILD_DIR", "${PROGNAME}.mcuboot.bin"), target_firm)

We could simply target ${PROGNAME}.bin instead of ${PROGNAME}.mcuboot.bin, or rename the file after processing it. However, I can understand that this is a bit too much, as I can see scenarios in which one would want to preserve the original firmware, or would need both files.

Idea 2: Use a way of detecting the file to upload

Another approach would be to have a detection that the user has built a mcuboot firmware. Currently, the upload stage only knows the correct file by the fact that the above code overwrites target_firm within the process, hence no other way exists of uploading the correct file than having mcuboot-image in COMMAND_LINE_TARGETS. However, a few lines later, when upload_source is set to target_firm, it could be a conditional assignment, where if ${PROGNAME}.mcuboot.bin exists in the build directory, we would set upload_source to use it instead of target_firm.

This part has only one drawback I can think of: if building mcuboot-image once, then building without it, if the build directory wasn't cleaned in between, we would target the wrong file. This is however the case with many things in Zephyr, such as when modifying the device tree: if not cleaned before build, that will not be taken into account.

Idea 3: Remember the latest built firmware

Another way of achieving the same result as the previous idea would be to "remember" the latest firmware image built. For example, the build process could leave a trace within the build directory in the form of a text file such as "latest_built_firmware". The upload process could then, when this file is present, use its content to define the target_firm variable.

I cannot see any drawbacks to this approach, except that the code of the python scripts would be slightly more complicated.

Idea 4: Define another target such as mcuboot-upload

Finally, another option could be to define another target in platformio-build.py, namely mcuboot-upload or something of this kind. It would simply do what it says: upload the .mcuboot.bin file to the board, instead of the .bin file.

That's all, folks!

Thank you for reading so far, and thanks for this very good tool that is PlatformIO. I use it both in research and in teaching with my students, and it's always a pleasure to see how things are simple and seamless using it when you know the complexity of what it does!