From 81a6b55d4ccb5d2d826cfef0c529337d5f11e269 Mon Sep 17 00:00:00 2001 From: Benjamin Gilbert Date: Wed, 27 May 2020 21:31:52 -0400 Subject: [PATCH 1/5] docs: add migrating-configs.md Straightforward port of the Ignition spec 3.1.0 migrating-configs content to FCCs. Users shouldn't need to read Ignition docs to learn about changes in the new FCC spec. --- docs/migrating-configs.md | 115 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 docs/migrating-configs.md diff --git a/docs/migrating-configs.md b/docs/migrating-configs.md new file mode 100644 index 00000000..952522b2 --- /dev/null +++ b/docs/migrating-configs.md @@ -0,0 +1,115 @@ +# Migrating Between Configuration Versions + +Occasionally, there are changes made to Fedora CoreOS configuration that break backward compatibility. While this is not a concern for running machines (since Ignition only runs one time during first boot), it is a concern for those who maintain configuration files. This document serves to detail each of the breaking changes and tries to provide some reasoning for the change. This does not cover all of the changes to the spec - just those that need to be considered when migrating from one version to the next. + +## From Version 1.0.0 to 1.1.0 + +There are no breaking changes between versions 1.0.0 and 1.1.0 of the configuration specification. Any valid 1.0.0 configuration can be updated to a 1.1.0 configuration by changing the version string in the config. + +The following is a list of notable new features, deprecations, and changes. + +### SHA-256 resource verification + +All `verification.hash` fields now support the `sha256` hash type. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +storage: + files: + - path: /etc/hosts + mode: 644 + contents: + source: https://example.com/etc/hosts + verification: + hash: sha256-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +``` + +### Compression support for certificate authorities and merged configs + +The config `merge` and `replace` sections and the `certificate_authorities` section now support gzip-compressed resources via the `compression` field. `gzip` compression is supported for all URL schemes except `s3`. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +ignition: + config: + merge: + - source: https://secure.example.com/example.ign.gz + compression: gzip + security: + tls: + certificate_authorities: + - source: https://example.com/ca.pem.gz + compression: gzip +``` + +### Filesystem mount options + +The `filesystems` section gained a new `mount_options` field. It is a list of options Ignition should pass to `mount -o` when mounting the specified filesystem. This is useful for mounting btrfs subvolumes. This field only affects mounting performed by Ignition while it is running; it does not affect mounting of the filesystem by the provisioned system. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +storage: + filesystems: + - path: /var/data + device: /dev/vdb1 + wipe_filesystem: false + format: btrfs + mount_options: + - subvolid=5 +``` + +### Custom HTTP headers + +The sections which allow fetching a remote URL — config `merge` and `replace`, `certificate_authorities`, and file `contents` and `append` — gained a new field called `http_headers`. This field can be set to an array of HTTP headers which will be added to an HTTP or HTTPS request. Custom headers can override Ignition's default headers, and will not be retained across HTTP redirects. + +During config merging, if a child config specifies a header `name` but not a corresponding `value`, any header with that `name` in the parent config will be removed. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +storage: + files: + - path: /etc/hosts + mode: 0644 + contents: + source: https://example.com/etc/hosts + http_headers: + - name: Authorization + value: Basic YWxhZGRpbjpvcGVuc2VzYW1l + - name: User-Agent + value: Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 5.1) +``` + +### HTTP proxies + +The `ignition` section gained a new field called `proxy`. It allows configuring proxies for HTTP and HTTPS requests, as well as exempting certain hosts from proxying. + +The `https_proxy` field specifies the proxy URL for HTTPS requests. The `http_proxy` field specifies the proxy URL for HTTP requests, and also for HTTPS requests if `https_proxy` is not specified. The `no_proxy` field lists specifiers of hosts that should not be proxied, in any of several formats: + +- An IP address prefix (`1.2.3.4`) +- An IP address prefix in CIDR notation (`1.2.3.4/8`) +- A domain name, matching the domain and its subdomains (`example.com`) +- A domain name, matching subdomains only (`.example.com`) +- A wildcard matching all hosts (`*`) + +IP addresses and domain names can also include a port number (`1.2.3.4:80`). + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +ignition: + proxy: + http_proxy: https://proxy.example.net/ + https_proxy: https://secure.proxy.example.net/ + no_proxy: + - www.example.net +storage: + files: + - path: /etc/hosts + mode: 0644 + contents: + source: https://example.com/etc/hosts +``` From 3c26888991b83574b4f5e908ba2610a373a95e5f Mon Sep 17 00:00:00 2001 From: Benjamin Gilbert Date: Thu, 28 May 2020 13:04:20 -0400 Subject: [PATCH 2/5] docs/migrating-configs: move section --- docs/migrating-configs.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/migrating-configs.md b/docs/migrating-configs.md index 952522b2..5a2829ed 100644 --- a/docs/migrating-configs.md +++ b/docs/migrating-configs.md @@ -8,23 +8,6 @@ There are no breaking changes between versions 1.0.0 and 1.1.0 of the configurat The following is a list of notable new features, deprecations, and changes. -### SHA-256 resource verification - -All `verification.hash` fields now support the `sha256` hash type. - -```yaml fedora-coreos-config -variant: fcos -version: 1.1.0 -storage: - files: - - path: /etc/hosts - mode: 644 - contents: - source: https://example.com/etc/hosts - verification: - hash: sha256-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -``` - ### Compression support for certificate authorities and merged configs The config `merge` and `replace` sections and the `certificate_authorities` section now support gzip-compressed resources via the `compression` field. `gzip` compression is supported for all URL schemes except `s3`. @@ -44,6 +27,23 @@ ignition: compression: gzip ``` +### SHA-256 resource verification + +All `verification.hash` fields now support the `sha256` hash type. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +storage: + files: + - path: /etc/hosts + mode: 644 + contents: + source: https://example.com/etc/hosts + verification: + hash: sha256-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +``` + ### Filesystem mount options The `filesystems` section gained a new `mount_options` field. It is a list of options Ignition should pass to `mount -o` when mounting the specified filesystem. This is useful for mounting btrfs subvolumes. This field only affects mounting performed by Ignition while it is running; it does not affect mounting of the filesystem by the provisioned system. From 3bdb7c8ce1ae7a8e454e9b6a6a5672a0e203e8e0 Mon Sep 17 00:00:00 2001 From: Benjamin Gilbert Date: Thu, 28 May 2020 18:05:43 -0400 Subject: [PATCH 3/5] docs/migrating-configs: add FCC-specific features --- docs/migrating-configs.md | 101 +++++++++++++++++++++++++++++++++++++- test | 2 +- 2 files changed, 101 insertions(+), 2 deletions(-) diff --git a/docs/migrating-configs.md b/docs/migrating-configs.md index 5a2829ed..1e3ec655 100644 --- a/docs/migrating-configs.md +++ b/docs/migrating-configs.md @@ -8,6 +8,89 @@ There are no breaking changes between versions 1.0.0 and 1.1.0 of the configurat The following is a list of notable new features, deprecations, and changes. +### Embedding local files in configs + +The config `merge` and `replace` sections, the `certificate_authorities` section, and the files `contents` and `append` sections gained a new field called `local`, which is mutually exclusive with the `source` and `inline` fields. It causes the contents of a file from the system running FCCT to be embedded in the config. The specified path is relative to a local _files-dir_, specified with the `-d`/`--files-dir` option to FCCT. If no _files-dir_ is specified, this functionality is unavailable. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +ignition: + config: + merge: + - local: config.ign + security: + tls: + certificate_authorities: + - local: ca.pem +storage: + files: + - path: /opt/file + contents: + local: file + append: + - local: file-epilogue + mode: 0644 +``` + +### Embedding directory trees in configs + +The `storage` section gained a new subsection called `trees`. It is a list of directory trees on the system running FCCT whose files, directories, and symlinks will be embedded in the config. By default, the resulting filesystem objects are owned by `root:root`, directory modes are set to 0755, and file modes are set to 0755 if the source file is executable or 0644 otherwise. Attributes of files, directories, and symlinks can be overridden by creating an entry in the `files`, `directories`, or `links` section; such `files` entries must omit `contents` and such `links` entries must omit `target`. + +Tree paths are relative to a local _files-dir_, specified with the `-d`/`--files-dir` option to FCCT. If no _files-dir_ is specified, this functionality is unavailable. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +storage: + trees: + - local: tree + path: /etc/files + files: + - path: /etc/files/overridden-file + mode: 0600 + user: + id: 500 + group: + id: 501 +``` + +### Inline contents on certificate authorities and merged configs + +The `certificate_authorities` section now supports inline contents via the `inline` field. The config `merge` and `replace` sections also now support `inline`, but using this functionality is not recommended. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +ignition: + config: + merge: + - inline: | + {"ignition": {"version": "3.1.0"}} + security: + tls: + certificate_authorities: + - inline: | + -----BEGIN CERTIFICATE----- + MIICzTCCAlKgAwIBAgIJALTP0pfNBMzGMAoGCCqGSM49BAMCMIGZMQswCQYDVQQG + EwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNj + bzETMBEGA1UECgwKQ29yZU9TIEluYzEUMBIGA1UECwwLRW5naW5lZXJpbmcxEzAR + BgNVBAMMCmNvcmVvcy5jb20xHTAbBgkqhkiG9w0BCQEWDm9lbUBjb3Jlb3MuY29t + MB4XDTE4MDEyNTAwMDczOVoXDTI4MDEyMzAwMDczOVowgZkxCzAJBgNVBAYTAlVT + MRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMRMw + EQYDVQQKDApDb3JlT1MgSW5jMRQwEgYDVQQLDAtFbmdpbmVlcmluZzETMBEGA1UE + AwwKY29yZW9zLmNvbTEdMBsGCSqGSIb3DQEJARYOb2VtQGNvcmVvcy5jb20wdjAQ + BgcqhkjOPQIBBgUrgQQAIgNiAAQDEhfHEulYKlANw9eR5l455gwzAIQuraa049Rh + vM7PPywaiD8DobteQmE8wn7cJSzOYw6GLvrL4Q1BO5EFUXknkW50t8lfnUeHveCN + sqvm82F1NVevVoExAUhDYmMREa6jZDBiMA8GA1UdEQQIMAaHBH8AAAEwHQYDVR0O + BBYEFEbFy0SPiF1YXt+9T3Jig2rNmBtpMB8GA1UdIwQYMBaAFEbFy0SPiF1YXt+9 + T3Jig2rNmBtpMA8GA1UdEwEB/wQFMAMBAf8wCgYIKoZIzj0EAwIDaQAwZgIxAOul + t3MhI02IONjTDusl2YuCxMgpy2uy0MPkEGUHnUOsxmPSG0gEBCNHyeKVeTaPUwIx + AKbyaAqbChEy9CvDgyv6qxTYU+eeBImLKS3PH2uW5etc/69V/sDojqpH3hEffsOt + 9g== + -----END CERTIFICATE----- +``` + ### Compression support for certificate authorities and merged configs The config `merge` and `replace` sections and the `certificate_authorities` section now support gzip-compressed resources via the `compression` field. `gzip` compression is supported for all URL schemes except `s3`. @@ -44,9 +127,24 @@ storage: hash: sha256-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 ``` +### Automatic generation of mount units + +The `filesystems` section gained a new `with_mount_unit` field. If `true`, a generic mount unit will be automatically generated for the specified filesystem. + +```yaml fedora-coreos-config +variant: fcos +version: 1.1.0 +storage: + filesystems: + - path: /var/data + device: /dev/vdb1 + format: ext4 + with_mount_unit: true +``` + ### Filesystem mount options -The `filesystems` section gained a new `mount_options` field. It is a list of options Ignition should pass to `mount -o` when mounting the specified filesystem. This is useful for mounting btrfs subvolumes. This field only affects mounting performed by Ignition while it is running; it does not affect mounting of the filesystem by the provisioned system. +The `filesystems` section gained a new `mount_options` field. It is a list of options Ignition should pass to `mount -o` when mounting the specified filesystem. This is useful for mounting btrfs subvolumes. If the `with_mount_unit` field is `true`, this field also affects mount options used by the provisioned system when mounting the filesystem. ```yaml fedora-coreos-config variant: fcos @@ -59,6 +157,7 @@ storage: format: btrfs mount_options: - subvolid=5 + with_mount_unit: true ``` ### Custom HTTP headers diff --git a/test b/test index 9bc535d5..7e8e95f5 100755 --- a/test +++ b/test @@ -19,7 +19,7 @@ mkdir tmpdocs trap 'rm -r tmpdocs' EXIT # Create files-dir contents expected by configs mkdir -p tmpdocs/files-dir/tree -touch tmpdocs/files-dir/local-file3 +touch tmpdocs/files-dir/{config.ign,ca.pem,file,file-epilogue,local-file3} for doc in docs/*md do From 754c316cef11483dcd6bb4ae3e1de696cfcb75f0 Mon Sep 17 00:00:00 2001 From: Benjamin Gilbert Date: Thu, 28 May 2020 18:11:40 -0400 Subject: [PATCH 4/5] docs/development: add spec bump instructions --- docs/development.md | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/docs/development.md b/docs/development.md index f6d74d2f..f9600693 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,5 +1,41 @@ -## Developing FCCT +# Developing FCCT -### Creating a release: +## Creating a release Create a [release checklist](https://github.com/coreos/fcct/issues/new?template=release-checklist.md) and follow those steps. + +## Bumping spec versions + +Up to this point in FCCT development, FCC versions and `base` versions have been 1:1 mapped onto Ignition specs, and we have not had any distro-specific sugar. This checklist therefore describes bumping the Ignition spec version, `base` version, and `config` version, while leaving the distro sugar version unchanged. If your scenario is different, modify to taste. + +### Stabilize Ignition spec version + +- Bump `go.mod` for new Ignition release and update vendor. +- Update imports. Drop `-experimental` from Ignition spec versions in `base/vB_exp/translate_test.go`. + +### Bump base version + +- Rename `base/vB_exp` to `base/vB` and update `package` statements. Update imports. +- Copy `base/vB` to `base/vB+1_exp`. +- Update `package` statements in `base/vB+1_exp`. + +### Bump config version + +- Rename `config/vC_exp` to `config/vC` and update `package` statements. Update imports. +- Drop `-experimental` from `registry` in `config/config.go`. +- Drop `-experimental` from examples in `docs/`. +- Copy `config/vC` to `config/vC+1_exp`. +- Update `package` statements in `config/vC+1_exp`. Bump its base dependency to `base/vB+1_exp`. +- Import `config/vC+1_exp` in `config/config.go` and add `fcos+C+1-experimental` to `registry`. + +### Bump Ignition spec version + +- Add translation function for experimental spec `I+1` to `distro/fcos/vF`. Revendor Ignition. +- Bump Ignition types imports and rename `ToIgnI` and `TestToIgnI` functions in `base/vB+1_exp`. Bump Ignition spec versions in `base/vB+1_exp/translate_test.go`. +- Bump Ignition types imports in `config/vC+1_exp`. Update `Translate` to call `ToIgnI+1` functions. Update versions in `TranslateBytes` comment. + +### Update docs + +- Copy the `C-exp` spec doc to `C+1-exp`. Update the header and the version numbers in the description of the `version` field. +- Rename the `C-exp` spec doc to `C`. Update the header, delete the experimental config warning, and update the version numbers in the description of the `version` field. +- Update `docs/migrating-configs.md` for the new spec version. Copy the relevant section from Ignition's `doc/migrating-configs.md`, convert the configs to FCCs, convert field names to snake case, and update wording as needed. Add subsections for any new FCC-specific features. From c419f62d7d11b906d3008a8391a8bf5d2ca91f23 Mon Sep 17 00:00:00 2001 From: Benjamin Gilbert Date: Thu, 28 May 2020 21:05:04 -0400 Subject: [PATCH 5/5] NEWS: add news for 0.6.0 --- NEWS | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/NEWS b/NEWS index 2429c482..cf9dfa79 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,31 @@ +2020-05-28 FCCT 0.6.0 + + Features: + + - Stabilize FCC spec 1.1.0, targeting Ignition spec 3.1.0 + - Add FCC spec 1.2.0-experimental, targeting Ignition spec + 3.2.0-experimental + - Add inline field to TLS certificate authorities and config merge and + replace (1.1.0) + - Add local field for embedding contents from local file (1.1.0) + - Add storage.trees section for embedding local directory trees (1.1.0) + - Auto-select smallest encoding for inline or local contents (1.1.0) + - Add http_headers field for specifying HTTP headers on fetch (1.1.0) + + Bug Fixes: + + - Include mount options in generated mount units (1.1.0) + - Validate uniqueness constraints within FCC sections + - Omit empty values from output JSON + - Append newline to output + + Docs Changes: + + - Document support for CA bundles in Ignition >= 2.3.0 + - Document support for sha256 resource verification (1.1.0) + - Clarify semantics of overwrite and mode fields + + 2020-03-23 FCCT 0.5.0 Breaking Changes: