Skip to content

Commit

Permalink
Allow Butane configs with basic Go templating
Browse files Browse the repository at this point in the history
* Recommend preparing Ignition configs external to Matchbox (e.g.
with Terraform poseidon/terraform-provider-ct)
* Document that Matchbox Rendering is discouraged
* Add Butane Config support as a replacement for dropping Container
Linux Config support. Perform Matchbox Go template evaluation,
translation to Ignition, and parsing to a forward compatible version
  • Loading branch information
dghubble committed Sep 26, 2022
1 parent 2301b48 commit 3893b7f
Show file tree
Hide file tree
Showing 12 changed files with 293 additions and 267 deletions.
18 changes: 18 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,24 @@ Notable changes between releases.

## Latest

* Upgrade Ignition from v0.35.0 (spec v2.x) to v2.14.0 (spec v3.x)
* Parse Ignition and render forward to the current Ignition spec version
* Ignition is [forward](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61) compatible (a `v3.1` spec can be rendered as `v3.3` safely)
* Recommend preparing Ignition externally (**action required**)
* Use Terraform with [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct)
* For a CLI, use Butane
* Remove support for Container Linux Configs (**action required**)
* Flatcar Linux now supports Ignition v2 (spec v3.x)
* Butane is a suitable YAML format that renders Ignition v2 (spec v3.x)
* Add limited support for Matchbox rendering Butane configs

For those still templating Container Linux Configs, you may be able to convert to Butane by prepending:

```yaml
variant: flatcar
version: 1.0.0
```
## v0.9.1
* Add dependabot Go module update automation ([#833](https://github.com/poseidon/matchbox/pull/833))
Expand Down
8 changes: 4 additions & 4 deletions docs/api-http.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,9 @@ coreos:
command: start
```
## Container Linux Config / Ignition Config
## Ignition Config
Finds the profile matching the machine and renders the corresponding Ignition Config with group metadata, selectors, and query params.
Finds the profile matching the machine and renders the corresponding Ignition for machine consumption.
```
GET http://matchbox.foo/ignition?label=value
Expand All @@ -121,11 +121,11 @@ GET http://matchbox.foo/ignition?label=value

```json
{
"ignition": { "version": "2.0.0" },
"ignition": { "version": "3.3.0" },
"systemd": {
"units": [{
"name": "example.service",
"enable": true,
"enabled": true,
"contents": "[Service]\nType=oneshot\nExecStart=/usr/bin/echo Hello World\n\n[Install]\nWantedBy=multi-user.target"
}]
}
Expand Down
2 changes: 1 addition & 1 deletion docs/cloud-config.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Cloud Config

!!! warning
Migrate to [Container Linux Configs](container-linux-config.md). Cloud-Config support will be removed in the future.
Migrate to [Ignition configs](ignition.md). Cloud-Config support will be removed in the future.

CoreOS Cloud-Config is a system for configuring machines with a Cloud-Config file or executable script from user-data. Cloud-Config runs in userspace on each boot and implements a subset of the [cloud-init spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data). See the cloud-config [docs](https://coreos.com/os/docs/latest/cloud-config.html) for details.

Expand Down
144 changes: 0 additions & 144 deletions docs/container-linux-config.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ terraform {

### Profiles

Machine profiles specify the kernel, initrd, kernel args, Ignition Config, and other configs (e.g. templated Container Linux Config, Cloud-config, generic) used to network boot and provision a bare-metal machine. The profile below would PXE boot machines using a Fedora CoreOS kernel and initrd (see [assets](api-http.md#assets) to learn about caching for speed), perform a disk install, reboot (first boot from disk), and use a [Fedora CoreOS Config](https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md) to generate an Ignition config to provision.
Machine profiles specify the kernel, initrd, kernel args, Ignition Config, or other configs (e.g. templated Butane Config, generic) used to network boot and provision a bare-metal machine. The profile below would PXE boot machines using a Fedora CoreOS kernel and initrd (see [assets](api-http.md#assets) to learn about caching for speed), perform a disk install, reboot (first boot from disk), and use a [Fedora CoreOS Config](https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md) to generate an Ignition config to provision.

```tf
// Fedora CoreOS profile
Expand Down
160 changes: 160 additions & 0 deletions docs/ignition.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# Ignition Configs

[Ignition](https://coreos.github.io/ignition/) configs define how disks should be provisioned (on network boot and first-boot from disk) to partition disks, write files (regular files, systemd units, networkd units, etc.), and configure users. Ignition is used by:

* Fedora CoreOS
* RHEL CoreOS
* Flatcar Linux

See the Ignition Config v3.x [specs](https://coreos.github.io/ignition/specs/) for details.

## Usage

Ignition configs can be added to the `/var/lib/matchbox/ignition` directory or in an `ignition` subdirectory of a custom `-data-path`. Ignition configs must end in `.ign` or `ignition`.

```
/var/lib/matchbox
├── ignition
│   └── k8s-controller.ign
│   └── k8s-worker.ign
└── profiles
```

Matchbox Profiles can set an Ignition config for provisioning machines. Specify the Ignition config in a [Profile](matchbox.md#profiles) with `ignition_id`.

```json
{
"id": "worker",
"name": "My Profile",
"boot": {
...
},
"ignition_id": "my-ignition.ign"
}
```

When PXE booting, set kernel arguments depending on the OS (e.g. `ignition.firstboot` on FCOS, `flatcar.first_boot=yes` on Flatcar).

* [Fedora CoreOS](https://github.com/poseidon/matchbox/blob/main/examples/profiles/fedora-coreos.json)
* [Flatcar Linux](https://github.com/poseidon/matchbox/blob/main/examples/profiles/flatcar.json)

Point the `ignition.config.url` or `flatcar.config.url` to point to the `matchbox` [Ignition endpoint](api-http.md#ignition-config).

Matchbox parses Ignition configs (e.g. `.ign` or `.ignition`) at spec v3.3 or below and renders to the current supported version (v3.3). This relies on Ignition's [forward compatibility](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61).

## Writing Configs

Ignition configs can be prepared externally and loaded via the gRPC API, rather than writing Ignition by hand.

### Terraform

Terraform can be used to prepare Ignition configs, while providing integrations with external systems and rich templating. Using tools like [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct), you can write Butane config (an easier YAML format), validate configs, and load Ignition into Matchbox ([examples](https://github.com/poseidon/matchbox/tree/main/examples/terraform)).

Define a Butane config for Fedora CoreOS or Flatcar Linux:

```yaml
variant: fcos
version: 1.4.0
passwd:
users:
- name: core
ssh_authorized_keys:
- ssh-key foo
```
```yaml
variant: flatcar
version: 1.0.0
passwd:
users:
- name: core
ssh_authorized_keys:
- ssh-key foo
```
Define a `ct_config` data source with strict validation. Optionally use Terraform [templating](https://github.com/poseidon/terraform-provider-ct).

```tf
data "ct_config" "worker" {
content = file("worker.yaml")
strict = true
pretty_print = false
snippets = [
file("units.yaml"),
file("storage.yaml"),
]
}
```

Then render the Butane config to Ignition and use it in a Matchbox Profile.

```tf
resource "matchbox_profile" "fedora-coreos-install" {
name = "worker"
kernel = "/assets/fedora-coreos/fedora-coreos-${var.os_version}-live-kernel-x86_64"
initrd = [
"--name main /assets/fedora-coreos/fedora-coreos-${var.os_version}-live-initramfs.x86_64.img"
]
args = [
"initrd=main",
"coreos.live.rootfs_url=${var.matchbox_http_endpoint}/assets/fedora-coreos/fedora-coreos-${var.os_version}-live-rootfs.x86_64.img",
"coreos.inst.install_dev=/dev/vda",
"coreos.inst.ignition_url=${var.matchbox_http_endpoint}/ignition?uuid=$${uuid}&mac=$${mac:hexhyp}",
]
raw_ignition = data.ct_config.worker.rendered
}
```

See the Terraform [examples](https://github.com/poseidon/matchbox/tree/main/examples#terraform-examples) for details.

### Butane

The [Butane](https://coreos.github.io/butane/) command line tool can be used to convert Butane configs (an easier YAML format) to Ignition. Then you can use the Matchbox gRPC API to upload the rendered Ignition to Matchbox for serving to machines on boot.

See [examples/ignition](../examples/ignition) for Butane config examples.

### Matchbox Rendering

While Matchbox recommends preparing Ignition configs externally (e.g. using Terraform's rich templating), Matchbox does still support limited templating and translation features with a builtin Butane converter.

Specify a Butane config in a [Profile](matchbox.md#profiles) with `ignition_id` (file must not end in `.ign` or `.ignition`).

```json
{
"id": "worker",
"name": "My Profile",
"boot": {
...
},
"ignition_id": "butane.yaml"
}
```

Here is an example Butane config with Matchbox template elements. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be interpreted using group metadata, selectors, and query params.

```yaml
variant: flatcar
version: 1.0.0
storage:
files:
- path: /var/home/core/foo
mode: 0644
contents:
inline: |
{{.example_contents}}
{{ if index . "ssh_authorized_keys" }}
passwd:
users:
- name: core
ssh_authorized_keys:
{{ range $element := .ssh_authorized_keys }}
- {{$element}}
{{end}}
{{end}}
```

Matchbox will use the Butane library to config to the current supported Ignition version. This relies on Ignition's [forward compatibility](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61).
Loading

0 comments on commit 3893b7f

Please sign in to comment.