Skip to content

Commit

Permalink
Track .ioc checksums to remove code generation from CI (#1087)
Browse files Browse the repository at this point in the history
### Summary
<!-- Quick summary of changes, optional -->

We currently re-generate STM32CubeMX code during CI, which is flaky and
slow. This is because the binary builds depend on a checksum of the
`.ioc` files, which are generated during STM32CubeMX code generation.
These are currently git ignored, forcing code regeneration in CI.

However, we can instead track the checksums, and add a new job to verify
the `.ioc` matches the tracked checksum. This gives us the best of both
worlds:
- We can verify that the `.ioc` file is up to date with the code
generated by STM32CubeMX by comparing the checksums. If they don't
match, you will need to re-generate code locally.
- CI is fast.

### Changelist 
<!-- Give a list of the changes covered in this PR. This will help both
you and the reviewer keep this PR within scope. -->

- Track `.ioc` MD5 checksums
- Add H7 dev board (I forgot to do this before whoops)

### Testing Done
<!-- Outline the testing that was done to demonstrate the changes are
solid. This could be unit tests, integration tests, testing on the car,
etc. Include relevant code snippets, screenshots, etc as needed. -->

### Resolved Issues
<!-- Link any issues that this PR resolved like so: `Resolves #1, #2,
and #5` (Note: Using this format, Github will automatically close the
issue(s) when this PR is merged in). -->

### Checklist
*Please change `[ ]` to `[x]` when you are ready.*
- [x] I have read and followed the code conventions detailed in
[README.md](../README.md) (*This will save time for both you and the
reviewer!*).
- [x] If this pull request is longer then **500** lines, I have provided
*explicit* justification in the summary above explaining why I *cannot*
break this up into multiple pull requests (*Small PR's are faster and
less painful for everyone involved!*).
  • Loading branch information
gtaharaedmonds authored Nov 18, 2023
1 parent eb6c4b0 commit 5432fbb
Show file tree
Hide file tree
Showing 13 changed files with 30 additions and 21 deletions.
30 changes: 14 additions & 16 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ jobs:
ioc_dir: firmware/thruna/BMS
- board: f4dev
ioc_dir: firmware/dev/f4dev/src/cubemx
- board: h7dev
ioc_dir: firmware/dev/h7dev/src/cubemx

container:
image: ubcformulaelectric/environment:latest
Expand All @@ -66,20 +68,15 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

# xvfb-run is required to prefix any commands that run STM32CubeMX
# It fakes out a display, since STM32CubeMX throws an error if it doesn't detect a display
- name: Generate code
# To ensure that the generated STM32CubeMX code is up to date with the .ioc file, we
# we also generate a checksum from the .ioc file when code is generated during builds.
# During CI, we re-generate the checksum. If it matches, then the code is up to date,
# otherwise, you need to re-generate STM32CubeMX code locally.
- name: Generate checksum
run: |
echo "test: $GITHUB_WORKSPACE"
xvfb-run python3 scripts/utilities/generate_cube_code.py \
--board ${{ matrix.board }} \
--log4j_properties_output ~/.stm32cubemx/log4j.properties \
--ioc $GITHUB_WORKSPACE/${{ matrix.ioc_dir }}/${{ matrix.board }}.ioc \
--codegen_output_dir $GITHUB_WORKSPACE/${{ matrix.ioc_dir }} \
--cube_bin /usr/local/STM32CubeMX/STM32CubeMX
- name: Fix formatting
run: python3 scripts/clang_format/fix_formatting.py
python3 scripts/utilities/generate_md5_checksum.py \
$GITHUB_WORKSPACE/${{ matrix.ioc_dir }}/${{ matrix.board }}.ioc \
$GITHUB_WORKSPACE/${{ matrix.ioc_dir }}/${{ matrix.board }}.ioc.md5
- name: Check for differences
run: git --no-pager diff --color --exit-code
Expand All @@ -98,6 +95,7 @@ jobs:
- DCM
- DIM
- f4dev
- h7dev
container:
image: ubcformulaelectric/environment:latest
credentials:
Expand Down Expand Up @@ -133,7 +131,7 @@ jobs:
password: ${{ secrets.DOCKER_TOKEN }}

env:
BUILD_DIR: build_x86
BUILD_DIR: build_fw_test

steps:

Expand Down Expand Up @@ -168,7 +166,7 @@ jobs:
password: ${{ secrets.DOCKER_TOKEN }}

env:
BUILD_DIR: build_x86
BUILD_DIR: build_fw_test

steps:
# Fix to ensure that commands like git status work in child directories
Expand Down Expand Up @@ -202,7 +200,7 @@ jobs:
password: ${{ secrets.DOCKER_TOKEN }}

env:
BUILD_DIR: build_x86
BUILD_DIR: build_fw_test

steps:
# Fix to ensure that commands like git status work in child directories
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/master_cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ jobs:
build_dir: firmware/thruna/BMS/BMS.elf
- board: f4dev
build_dir: firmware/dev/f4dev/f4dev.elf
- board: h7dev
build_dir: firmware/dev/f4dev/h7dev.elf
container:
image: ubcformulaelectric/environment:latest
credentials:
Expand Down Expand Up @@ -72,7 +74,7 @@ jobs:
uses: ./.github/actions/push_release
with:
# https://github.com/actions/download-artifact#download-all-artifacts
files: "FSM.elf/FSM.elf BMS.elf/BMS.elf DCM.elf/DCM.elf DIM.elf/DIM.elf PDM.elf/PDM.elf f4dev.elf/f4dev.elf CanMsgs.dbc/CanMsgs.dbc"
files: "FSM.elf/FSM.elf BMS.elf/BMS.elf DCM.elf/DCM.elf DIM.elf/DIM.elf PDM.elf/PDM.elf f4dev.elf/f4dev.elf h7dev.elf/h7dev.elf CanMsgs.dbc/CanMsgs.dbc"
latest: true
title: "Stable Binaries and DBCs"
body: "This release contains the latest stable binaries and DBCs."
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,6 @@ cmake-build-*
*.l1p
*.l2p
*.iex
*.md5

# STM32CubeMX
.mxproject
Expand Down
2 changes: 1 addition & 1 deletion .vscode/cmake-variants.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
platform:
default: arm
default: firmware_deploy
choices:
firmware_deploy:
short: Firmware Deploy
Expand Down
1 change: 1 addition & 0 deletions firmware/dev/f4dev/src/cubemx/f4dev.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
eeda1272ba9532f2df9a7512c39e8643
3 changes: 3 additions & 0 deletions firmware/dev/h7dev/src/cubemx/Src/main.c
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
/* Private includes ----------------------------------------------------------*/
/* USER CODE BEGIN Includes */
#include "string.h"
#include "hw_hardFaultHandler.h"
/* USER CODE END Includes */

/* Private typedef -----------------------------------------------------------*/
Expand Down Expand Up @@ -109,6 +110,8 @@ int main(void)
/* USER CODE BEGIN 2 */
// __HAL_DBGMCU_FREEZE_IWDG();

hw_hardFaultHandler_handleFault();

// Configure and initialize SEGGER SystemView.
SEGGER_SYSVIEW_Conf();
/* USER CODE END 2 */
Expand Down
4 changes: 2 additions & 2 deletions firmware/dev/h7dev/src/cubemx/Src/stm32h7xx_it.c
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
#include "stm32h7xx_it.h"
/* Private includes ----------------------------------------------------------*/
/* USER CODE BEGIN Includes */
#include "Io_SharedHardFaultHandler.h"
#include "hw_hardFaultHandler.h"
/* USER CODE END Includes */

/* Private typedef -----------------------------------------------------------*/
Expand Down Expand Up @@ -86,7 +86,7 @@ void NMI_Handler(void)
void HardFault_Handler(void)
{
/* USER CODE BEGIN HardFault_IRQn 0 */
Io_SharedHardFaultHandler_HandleHardFault();
hw_hardFaultHandler_handleFault();
/* USER CODE END HardFault_IRQn 0 */
while (1)
{
Expand Down
1 change: 1 addition & 0 deletions firmware/dev/h7dev/src/cubemx/h7dev.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3c0783d07b4f61d0b1f0c73c7592fcca
1 change: 1 addition & 0 deletions firmware/thruna/BMS/BMS.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
07584c0edf7569aaf2d92d1a4cfd039c
1 change: 1 addition & 0 deletions firmware/thruna/DCM/DCM.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
d260ccc38dd514961b5318c4a6b673e3
1 change: 1 addition & 0 deletions firmware/thruna/DIM/src/cubemx/DIM.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bb13c45725652542bc8d2e080e9b0a06
1 change: 1 addition & 0 deletions firmware/thruna/FSM/FSM.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
66b685c1e9c81840c30fa4a2e30cf328
1 change: 1 addition & 0 deletions firmware/thruna/PDM/PDM.ioc.md5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
4def9a455a9cc19cb897bc45ccee1923

0 comments on commit 5432fbb

Please sign in to comment.