Skip to content

Commit

Permalink
chore: add docs for versioning and testing (#21)
Browse files Browse the repository at this point in the history
* chore: add docs for versioning and testing

* update release info / bundle info

* update release info / bundle info
  • Loading branch information
Racer159 authored Mar 12, 2024
1 parent 5265bf8 commit 8aae272
Show file tree
Hide file tree
Showing 2 changed files with 76 additions and 0 deletions.
40 changes: 40 additions & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# SWF Testing

Currently we test the UDS Software Factory at two different levels, the overall `bundle`, and the individual `package`. Each level has different tests that it performs in order to save time while still ensuring that the necessary configurations are covered.

## Bundle Testing

Bundle testing is performed in the main [UDS Software Factory repository](https://github.com/defenseunicorns/uds-software-factory/) and tests the following:

1. Overall health of each package within the deployment
2. Integration of individual packages with each other
3. Integration of packages with all of `uds-core`
4. Integration of packages with clusters beyond `k3d`

It does not test:

1. Upgrades of individual packages
2. Package flavors other than `upstream`

## Package Testing

Package testing is performed in each package repository and tests the following:

1. Health of the Package deployment (including its individual components)
2. Package upgrades from the previous version to the current version
3. The full matrix of package flavors
4. Integration of a package with `k3d-core-istio`

It does not test:

1. Integration between SWF packages (with the exception of add-ons like GitLab Runner)
2. Integration of a package with all of `uds-core`
3. Integration of packages with clusters beyond `k3d`

> :warning: **Note:** In order to ensure that we can be reasonably confident in the above, package flavors should deviate only by their selected _images_ and not by other configuration like their Helm Chart. This ensures that when we test the images with the given chart / configuration and then test that configuration within a bundle, we can be reasonably certain that each combination of flavors would work within the bundle.
## Running Tests

In order to keep them portable, tests at all levels will be implemented as `uds` tasks that create the necessary packages/bundles, setup the cluster, deploy the packages/bundles and then perform the necessary checks on them. These will then be wrapped in GitHub actions to run on Pull Requests / Releases.

> :warning: **Note:** You can see the list of available tasks in a given repository by running `uds run --list` at the repository root.
36 changes: 36 additions & 0 deletions docs/versioning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# SWF Versioning

> :warning: **Note:** Packages are the only production release artifacts published from the UDS Software Factory repositories, as bundles are meant to be _created_, not consumed, by users. Repos may publish bundles for demo purposes, but those are not intended for production consumption though will loosely follow standard semantic versioning.
## Versioning

UDS Software Factory `packages` are versioned as follows:

```
<upstream-app-version>-uds.<uds-sub-version>
```

Where,

- `upstream-app-version`: is the version of the main application in the package (i.e. `16.9.2` for GitLab)
- `uds-sub-version`: is the number of releases since the last main application version bump (starting at `0`)

In practice, this results in the following for the second release of a `package` for version `16.9.2` of GitLab:

```
16.9.2-uds.1
```

## Releases

Releases themselves are cut with [Release Please](https://github.com/googleapis/release-please), but due to the fact that we don't follow semantic versioning precisely, specific version numbers are manually bumped in the release please PRs or via [empty commits](https://github.com/googleapis/release-please). Specific lines to update within `zarf.yaml` or `uds-bundle.yaml` files are controlled via the following:

```yaml
# x-release-please-start-version
version: "0.2.1"
# x-release-please-end
```

Once a release is cut by merging a release-please PR, a GitHub action takes over and publishes the resulting package to `ghcr.io/defenseunicorns/packages` where it is available in Zarf and UDS CLI via the `defenseunicorns` and `🦄` shorthands.

> :warning: **Note:** If a release is broken or has known issues that the team feels are severe enough, the `CHANGELOG.md` and GitHub release notes should be updated denoting those facts, and a new release should be issued as soon as the team is able.

0 comments on commit 8aae272

Please sign in to comment.