Skip to content

Commit

Permalink
Refresh manual examples and local QEMU/KVM docs
Browse files Browse the repository at this point in the history
* Refactor examples to boot provision minimal hosts
with Fedora CoreOS or Flatcar Linux
* Remove the etcd3 cluster example or other specific
kinds of hosts
* Update script get-fedora-coreos
* Remove script get-coreos
  • Loading branch information
dghubble committed Oct 14, 2020
1 parent dadca25 commit ff36cf9
Show file tree
Hide file tree
Showing 54 changed files with 275 additions and 583 deletions.
3 changes: 2 additions & 1 deletion CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Notable changes between releases.
* Publish Matchbox images from internal infra to Quay (`quay.io/poseidon/matchbox`)
* Update Go version from v1.13.4 to v1.13.11
* Update base image from `alpine:3.10` to `alpine:3.12` ([#784](https://github.com/poseidon/matchbox/pull/784))
* Update Kubernetes manifests ([#791](https://github.com/poseidon/matchbox/pull/791))
* Refresh docs and examples for Fedora CoreOS and Flatcar Linux
* Update Kubernetes manifest examples ([#791](https://github.com/poseidon/matchbox/pull/791))
* Include `contrib/k8s` in release tarballs ([#788](https://github.com/poseidon/matchbox/pull/788))
* Remove RPM spec file (Copr publishing stopped in v0.6)

Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# matchbox [![Build Status](https://github.com/poseidon/matchbox/workflows/test/badge.svg)](https://github.com/poseidon/matchbox/actions?query=workflow%3Atest+branch%3Amaster) [![GoDoc](https://godoc.org/github.com/poseidon/matchbox?status.svg)](https://godoc.org/github.com/poseidon/matchbox) [![Quay](https://img.shields.io/badge/container-quay-green)](https://quay.io/repository/poseidon/matchbox)

`matchbox` is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Container Linux or Fedora CoreOS config.
`matchbox` is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Ignition config.

## Features

* Chainload via iPXE and match hardware labels
* Provision Fedora CoreOS and Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Provision Fedora CoreOS or Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Authenticated gRPC API for clients (e.g. Terraform)

## Documentation
Expand All @@ -23,10 +23,11 @@ Matchbox can be installed from a binary or a container image.

## Tutorials

[Getting started](docs/getting-started.md) provisioning machines with Flatcar Linux.
[Getting started](docs/getting-started.md) provisioning machines with Fedora CoreOS or Flatcar Linux.

* Local QEMU/KVM
* [matchbox with Docker](docs/getting-started-docker.md)
* [Local QEMU/KVM](docs/getting-started-docker.md)
* Fedora CoreOS (live PXE or PXE install to disk)
* Flatcar Linux (live PXE or PXE install to disk)
* Clusters
* [etcd3](docs/getting-started-docker.md) - Install a 3-node etcd3 cluster
* [etcd3](https://github.com/poseidon/matchbox/tree/master/examples/terraform/etcd3-install) - Install a 3-node etcd3 cluster (terraform-based)
Expand Down
40 changes: 23 additions & 17 deletions docs/getting-started-docker.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Getting started with Docker

In this tutorial, we'll run `matchbox` on your Linux machine with Docker to network boot and provision a cluster of QEMU/KVM Container Linux machines locally. You'll be able to create Kubernetes clusters, etcd3 clusters, and test network setups.
In this tutorial, we'll run `matchbox` on a Linux machine with Docker to network boot and provision local QEMU/KVM machines as Fedora CoreOS or Flatcar Linux machines. You'll be able to test network setups and Ignition provisioning.

*Note*: To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).
!!! note
To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).

## Requirements

Expand All @@ -25,10 +26,11 @@ $ git clone https://github.com/poseidon/matchbox.git
$ cd matchbox
```

Download CoreOS Container Linux image assets referenced by the `etcd3` [example](../examples) to `examples/assets`.
Download Fedora CoreOS or Flatcar Linux image assets to `examples/assets`.

```sh
$ ./scripts/get-coreos stable 1967.3.0 ./examples/assets
$ ./scripts/get-fedora-coreos stable 32.20200923.3.0 ./examples/assets
$ ./scripts/get-flatcar stable 2605.6.0 ./examples/assets
```

For development convenience, add `/etc/hosts` entries for nodes so they may be referenced by name.
Expand All @@ -45,10 +47,10 @@ For development convenience, add `/etc/hosts` entries for nodes so they may be r

Run the `matchbox` and `dnsmasq` services on the `docker0` bridge. `dnsmasq` will run DHCP, DNS and TFTP services to create a suitable network boot environment. `matchbox` will serve configs to machines as they PXE boot.

The `devnet` convenience script can start these services and accepts the name of any example cluster in [examples](../examples).
The `devnet` convenience script can start these services and accepts the name of any example in [examples](https://github.com/poseidon/matchbox/tree/master/examples).

```sh
$ sudo ./scripts/devnet create etcd3
$ sudo ./scripts/devnet create fedora-coreos
```

Inspect the logs.
Expand All @@ -57,7 +59,7 @@ Inspect the logs.
$ sudo ./scripts/devnet status
```

Take a look at the [etcd3 groups](../examples/groups/etcd3) to get an idea of how machines are mapped to Profiles. Explore some endpoints exposed by the service, say for QEMU/KVM node1.
Inspect the examples and Matchbox endpoints to see how machines (e.g. node1 with MAC `52:54:00:a1:9c:ae`) are mapped to Profiles, and therefore iPXE and Ignition configs.

* iPXE [http://127.0.0.1:8080/ipxe?mac=52:54:00:a1:9c:ae](http://127.0.0.1:8080/ipxe?mac=52:54:00:a1:9c:ae)
* Ignition [http://127.0.0.1:8080/ignition?mac=52:54:00:a1:9c:ae](http://127.0.0.1:8080/ignition?mac=52:54:00:a1:9c:ae)
Expand All @@ -68,7 +70,7 @@ Take a look at the [etcd3 groups](../examples/groups/etcd3) to get an idea of ho
If you prefer to start the containers yourself, instead of using `devnet`,

```sh
$ sudo docker run -p 8080:8080 --rm -v $PWD/examples:/var/lib/matchbox:Z -v $PWD/examples/groups/etcd3:/var/lib/matchbox/groups:Z quay.io/poseidon/matchbox:latest -address=0.0.0.0:8080 -log-level=debug
$ sudo docker run -p 8080:8080 --rm -v $PWD/examples:/var/lib/matchbox:Z -v $PWD/examples/groups/fedora-coreos:/var/lib/matchbox/groups:Z quay.io/poseidon/matchbox:latest -address=0.0.0.0:8080 -log-level=debug
$ sudo docker run --name dnsmasq --cap-add=NET_ADMIN -v $PWD/contrib/dnsmasq/docker0.conf:/etc/dnsmasq.conf:Z quay.io/poseidon/dnsmasq -d
```

Expand Down Expand Up @@ -101,15 +103,12 @@ $ sudo ./scripts/libvirt [start|reboot|shutdown|poweroff|destroy]

## Verify

The VMs should network boot and provision themselves into a three node etcd3 cluster, with other nodes behaving as etcd3 gateways.
The VMs should network boot and provision themselves as declared.

The example profile added autologin so you can verify that etcd3 works between nodes.

```sh
$ systemctl status etcd-member
$ etcdctl set /message hello
$ etcdctl get /message
```
cat /etc/os-release
```

## Clean up

Clean up the containers and VM machines.
Expand All @@ -119,6 +118,13 @@ $ sudo ./scripts/devnet destroy
$ sudo ./scripts/libvirt destroy
```

## Going further
## Going Further

Learn more about [matchbox](matchbox.md) or explore the other [examples](https://github.com/poseidon/matchbox/tree/master/examples).

Try different examples and Ignition declarations:

* Declare an SSH authorized public key (see examples README)
* Declare a systemd unit
* Declare file or directory content

Learn more about [matchbox](matchbox.md) or explore the other [example](../examples) clusters.
6 changes: 3 additions & 3 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Getting started

In this tutorial, we'll show how to use terraform with `matchbox` to provision Container Linux machines.
In this tutorial, we'll use `matchbox` with Terraform to provision Container Linux machines.

You'll install the `matchbox` service, setup a PXE network boot environment, and then use terraform configs to describe your infrastructure and the terraform CLI to create those resources on `matchbox`.
We'll install the `matchbox` service, setup a PXE network boot environment, and use Terraform configs to declare infrastructure and apply resources on `matchbox`.

## matchbox

Install `matchbox` on a dedicated server or Kubernetes cluster. Generate TLS credentials and enable the gRPC API as directed. Save the `ca.crt`, `client.crt`, and `client.key` on your local machine (e.g. `~/.matchbox`).
Install `matchbox` on a host server or Kubernetes cluster. Generate TLS credentials and enable the gRPC API as directed. Save the `ca.crt`, `client.crt`, and `client.key` on your local machine (e.g. `~/.matchbox`).

* Installing on [Container Linux / other distros](deployment.md)
* Installing on [Kubernetes](deployment.md#kubernetes)
Expand Down
11 changes: 6 additions & 5 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Matchbox

Matchbox is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Container Linux or Fedora CoreOS config.
Matchbox is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Ignition config.

## Features

* Chainload via iPXE and match hardware labels
* Provision Container Linux and Fedora CoreOS (powered by [Ignition](https://github.com/coreos/ignition))
* Provision Fedora CoreOS or Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Authenticated gRPC API for clients (e.g. Terraform)

## Installation
Expand All @@ -17,10 +17,11 @@ Matchbox can be installed from a binary or a container image.

## Tutorials

[Getting started](getting-started.md) provisioning machines with Container Linux.
[Getting started](getting-started.md) provisioning machines with Fedora CoreOS or Flatcar Linux.

* Local QEMU/KVM
* [matchbox with Docker](getting-started-docker.md)
* [Local QEMU/KVM](getting-started-docker.md)
* Fedora CoreOS (live PXE or PXE install to disk)
* Flatcar Linux (live PXE or PXE install to disk)
* Clusters
* [etcd3](getting-started-docker.md) - Install a 3-node etcd3 cluster
* [etcd3](https://github.com/poseidon/matchbox/tree/master/examples/terraform/etcd3-install) - Install a 3-node etcd3 cluster (terraform-based)
Expand Down
68 changes: 39 additions & 29 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,41 +13,51 @@ These examples use [Terraform](https://www.terraform.io/intro/) as a client to M

### Customization

You are encouraged to look through the examples and Terraform modules. Implement your own profiles or package them as modules to meet your needs. We've just provided a starting point. Learn more about [matchbox](../docs/matchbox.md) and [Container Linux configs](../docs/container-linux-config.md).
Look through the examples and Terraform modules and use them as a starting point. Learn more about [matchbox](../docs/matchbox.md) and [Container Linux configs](../docs/container-linux-config.md).

## Manual Examples

These examples mount raw Matchbox objects into a Matchbox server's `/var/lib/matchbox/` directory.

| Name | Description | CoreOS Container Linux Version | FS | Docs |
|------------|-------------|----------------|----|-----------|
| simple | CoreOS Container Linux with autologin, using iPXE | stable/1967.3.0 | RAM | [reference](https://coreos.com/os/docs/latest/booting-with-ipxe.html) |
| simple-install | CoreOS Container Linux Install, using iPXE | stable/1967.3.0 | RAM | [reference](https://coreos.com/os/docs/latest/booting-with-ipxe.html) |
| grub | CoreOS Container Linux via GRUB2 Netboot | stable/1967.3.0 | RAM | NA |
| etcd3 | PXE boot a 3-node etcd3 cluster with proxies | stable/1967.3.0 | RAM | None |
| etcd3-install | Install a 3-node etcd3 cluster to disk | stable/1967.3.0 | Disk | None |
| Name | Description | FS | Docs |
|---------------|------------------------------|-----|-------|
| fedora-coreos | Fedora CoreOS live PXE | RAM | [docs](https://docs.fedoraproject.org/en-US/fedora-coreos/live-booting-ipxe/) |
| fedora-coreos-install | Fedora CoreOS install | Disk | [docs](https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/) |
| flatcar | Flatcar Linux live PXE | RAM | [docs](https://docs.flatcar-linux.org/os/booting-with-ipxe/) |
| flatcar-install | Flatcar Linux install | Disk | [docs](https://docs.flatcar-linux.org/os/booting-with-ipxe/) |

### Customization

#### Autologin
For Fedora CoreOS, add an SSH authorized key to Fedora CoreOS Config (`ignition/fedora-coreos.yaml`) and regenerate the Ignition Config.

```
variant: fcos
version: 1.1.0
passwd:
users:
- name: core
ssh_authorized_keys:
- ssh-rsa pub-key-goes-here
```

```
podman run -i --rm quay.io/coreos/fcct:release --pretty --strict < fedora-coreos.yaml > fedora-coreos.ign
```

For Flatcar Linux, add a Matchbox variable to a Group to set the SSH authorized key (or directly update the Container Linux Config).

```
# groups/flatcar-install/flatcar.json
{
"id": "stage-1",
"name": "Flatcar Linux",
"profile": "flatcar",
"selector": {
"os": "installed"
},
"metadata": {
"ssh_authorized_keys": ["ssh-rsa pub-key-goes-here"]
}
}
```

Example profiles pass the `coreos.autologin` kernel argument. This skips the password prompt for development and troubleshooting and should be removed **before production**.

## SSH Keys

Example groups allow `ssh_authorized_keys` to be added for the `core` user as metadata. You might also include this directly in your Ignition.

# /var/lib/matchbox/groups/default.json
{
"name": "Example Machine Group",
"profile": "pxe",
"metadata": {
"ssh_authorized_keys": ["ssh-rsa pub-key-goes-here"]
}
}

#### Conditional Variables

**"pxe"**

Some examples check the `pxe` variable to determine whether to create a `/dev/sda1` filesystem and partition for PXEing with `root=/dev/sda1` ("pxe":"true") or to write files to the existing filesystem on `/dev/disk/by-label/ROOT` ("pxe":"false").
11 changes: 0 additions & 11 deletions examples/groups/bootkube-install/install.json

This file was deleted.

19 changes: 0 additions & 19 deletions examples/groups/bootkube-install/node1.json

This file was deleted.

16 changes: 0 additions & 16 deletions examples/groups/bootkube-install/node2.json

This file was deleted.

16 changes: 0 additions & 16 deletions examples/groups/bootkube-install/node3.json

This file was deleted.

18 changes: 0 additions & 18 deletions examples/groups/bootkube/node1.json

This file was deleted.

16 changes: 0 additions & 16 deletions examples/groups/bootkube/node2.json

This file was deleted.

16 changes: 0 additions & 16 deletions examples/groups/bootkube/node3.json

This file was deleted.

11 changes: 0 additions & 11 deletions examples/groups/etcd3-install/gateway.json

This file was deleted.

Loading

0 comments on commit ff36cf9

Please sign in to comment.