Skip to content

Operator Upgrades

Jump to bottom
Prajakta Belgundi edited this page Apr 27, 2020 · 31 revisions

Supported Upgrade Paths

Till Operator version 0.4.x only minor version upgrades were supported e.g. 0.4.0 -> 0.4.1.

Starting Operator version 0.4.3 we also support major version upgrades for Pravega Operator.

0.4.3 --> 0.5.0

0.4.4 -->0.5.0

Upgrade Guide

Upgrading till 0.4.4

Till version 0.4.4, Pravega operator can be upgraded by modifying the image tag using kubectl edit, patch or apply...

$ kubectl edit <operator deployment name>

The upgrade is handled as a rolling update by Kubernetes and results in a new operator pod being created and he old one being terminated.

Upgrading from 0.4.x to 0.5.0

What changed between 0.4.x and 0.5.0 ?

A lot has changed between operator versions 0.4.x and 0.5.0.

Here is a list of changes and their impact:

  1. The Pravega Cluster CRD

Till Operator 0.4.4, the Pravega CR at version v1alpha1 includes Bookkeeper. Starting version 0.5.0, Pravega CR does not include Bookkeeper. Bookkeeper is moved out and is now a prerequisite for Pravega deployment. It can be installed separately either using Bookkeeper-Operator or some other means.

  1. CR conversion from v1alpha1 to v1beta1

When upgrading to Operator version 0.5.x, it is necessary to migrate the Pravega CR Object from the old version v1alpha1[with Bookkeeper] to the new version v1beta1[without Bookkeeper]. This is done by a Conversion Webhook that gets triggered automatically when Operator is upgraded to 0.5.0. See: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning/

The Conversion Webhook is part of Pravega Operator code (starting 0.5.0-rc1) and it does the following:

a. Converts a Pravega CR v1alpha1 Object to v1beta1 Object by copying all values in the Pravega Spec for Controller, SegmentStore, Tier2 (LongTermStorage) etc...

b. Creates a new Bookkeeper CR object based on the Bookkeeper CRD and populates it with values from the Bookkeeper Spec in Pravega CR Object v1alpha1.

c. Sets owner/controller reference of all Bookkeeper objects [STS, ConfigMap, PVCs, PDB & Headless Service] to the new Bookkeeper CR Object and removes references to the Pravega CR Object. With this, all existing Bookkeeper artifacts are migrated to Bookkeeper CR from Pravega CR. The Bookkeeper CR object so created, should be managed by a Bookkeeper Operator.

d. Sets owner references of all Pravega Cluster objects [Controller Deployment, Services, Config Maps, SegmentStore STS, Services, Config Maps etc...] to APIVersion "pravega.pravega.io/v1beta1" from earlier "pravega.pravega.io/v1alpha1". This makes sure on deletion of Pravega CR Object, all these artifacts are also deleted

  1. OpenAPIV3Schema Validation

Structural schemas are a requirement for Custom Resources [starting apiextensions.k8s.io/v1], and not specifying one disables ConversionWebhook feature in version _apiextensions.k8s.io/v1beta1_. As such, the new Pravega Cluster CRD includes OpenAPIV3Schema Validation for Pravega CRs for both versions v1alpha1 and v1beta1. See: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#specifying-a-structural-schema

  1. Controller Runtime upgrade

Controller Runtime library has been upgraded from v0.1.8 to v0.5.1 as the older one does not support conversion webhooks. The operator uses hub-spokes model in controller-runtime to achieve version conversion.

See: https://book.kubebuilder.io/multiversion-tutorial/tutorial.html

  1. Mutating Webhook replaced with Validating Webhook

The mutating webhook in operator(0.4.x) has been replaced with a validating webhook(0.5.x) that currently only validates pravega version information entered by the user. This can be extended later to add more validations. A mutating webhook was no longer needed as "tag" field in Pravega Image Spec has been removed in v1beta1 and only "version" field is now supported.

  1. Tier2 renamed to LongTermStorage in Pravega Spec

The field "Tier2" in Pravega Spec has been renamed to LongTermStorage in v1beta1. During upgrade the version conversion code takes care of migrating values mentioned inside Tier2 in v1alpha1 to field LongTermStorage in v1beta1 as there is no Tier2 feild in v1beta1.

Pre-requisites

For upgrading Operator to version 0.5.0, the following must be true:

  1. The Kubernetes Server version must be at least 1.15, (WebhookConversion is a beta feature in Kubernetes 1.15) See: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning/#webhook-conversion

  2. Cert-Manager or some other certificate management solution must be deployed for managing webhook service certificates. The upgrade trigger script assume user has cert-manager installed but any other cert management solution can also be used and script would need to be modified accordingly.

Installing cert-manager: https://cert-manager.io/docs/installation/kubernetes/

  1. Bookkeeper Operator must be deployed in the same namespace as Pravega Operator, prior to triggering the upgrade.

Triggering the upgrade

The upgrade to Operator 0.5.0, can be triggered using the script operatorUpgrade.sh under tools folder. This script patches the Pravega Cluster CRD and creates necessary K8s artifacts, needed by 0.5.0 Operator, prior to triggering the upgrade by updating the image tag in Operator deployment.

Migrating out of Pravega CR v1alpha1

Operator 0.5.0 watches and reconciles only v1beta1 Pravega CR and not v1alpha1. After all Pravega Clusters in a given K8s Cluster have migrated to version v1beta1 the served: flag for version v1alpha1 can be set to false to indicate this is no longer a supported Pravega CR version for that K8s Cluster.

Clone this wiki locally