[WIP] add strimzi-kafka-cluster-operator

[Spedoske]

This continued issue is from pull request #229

strimzi-kafka-cluster-operator is an operator write by Java. Therefore, the source code information is not available to Acto.

• add README.md
• add k8s_mapping.py
• add prune_blackbox.py if needed
• lock image versions
• perform a full Acto run

I decide to implement the schema by a monkey patch, because I do not want to pollute the codebase by adding language specific classes.

The current progress of the branch:

1. Acto can detect the resource schema.
2. Acto can apply the schema to the input model
3. Acto can generate test cases for Quantity schema

See #229 (comment)

[Spedoske]

The Kafka operator does not use the official Kubernetes API, resulting in a different schema for it than for other operators that use the official Kubernetes API. An example will show below. Most of fields in the Java schema is also in the go schema, while topologySpreadConstraints, metadata, and enableServiceLinks are not. These difference make Acto cannot recognize some known schema.

Kafka operator:

https://github.com/strimzi/strimzi-kafka-operator/blob/f49182c28229b3d73460ac2b4cf6fbd6bc25e6ed/api/src/main/java/io/strimzi/api/kafka/model/template/PodTemplate.java#L31-L58

/**
 * Representation of a pod template for Strimzi resources.
 */
@Buildable(
        editableEnabled = false,
        builderPackage = Constants.FABRIC8_KUBERNETES_API
)
@JsonInclude(JsonInclude.Include.NON_DEFAULT)
@JsonPropertyOrder({"metadata", "imagePullSecrets", "securityContext", "terminationGracePeriodSeconds", "affinity",
    "tolerations", "topologySpreadConstraint", "priorityClassName", "schedulerName", "hostAliases", "tmpDirSizeLimit"})
@EqualsAndHashCode
@DescriptionFile
public class PodTemplate implements HasMetadataTemplate, Serializable, UnknownPropertyPreserving {
    private static final long serialVersionUID = 1L;

    private MetadataTemplate metadata;
    private List<LocalObjectReference> imagePullSecrets;
    private PodSecurityContext securityContext;
    private int terminationGracePeriodSeconds = 30;
    private Affinity affinity;
    private List<Toleration> tolerations;
    private List<TopologySpreadConstraint> topologySpreadConstraints;
    private String priorityClassName;
    private String schedulerName;
    private List<HostAlias> hostAliases;
    private Boolean enableServiceLinks;
    private String tmpDirSizeLimit;
    private Map<String, Object> additionalProperties = new HashMap<>(0);

Kubernetes Api:

https://github.com/kubernetes-client/go/blob/92040c8d5731095abc7378b48b96f3cec2b4090b/kubernetes/client/v1_pod_spec.go#L12-L95

// PodSpec is a description of a pod.
type V1PodSpec struct {

	// Optional duration in seconds the pod may be active on the node relative to StartTime before the system will actively try to mark it failed and kill associated containers. Value must be a positive integer.
	ActiveDeadlineSeconds int64 `json:"activeDeadlineSeconds,omitempty"`

	// If specified, the pod's scheduling constraints
	Affinity *V1Affinity `json:"affinity,omitempty"`

	// AutomountServiceAccountToken indicates whether a service account token should be automatically mounted.
	AutomountServiceAccountToken bool `json:"automountServiceAccountToken,omitempty"`

	// List of containers belonging to the pod. Containers cannot currently be added or removed. There must be at least one container in a Pod. Cannot be updated.
	Containers []V1Container `json:"containers"`

	// Specifies the DNS parameters of a pod. Parameters specified here will be merged to the generated DNS configuration based on DNSPolicy.
	DnsConfig *V1PodDnsConfig `json:"dnsConfig,omitempty"`

	// Set DNS policy for the pod. Defaults to \"ClusterFirst\". Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'. DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy. To have DNS options set along with hostNetwork, you have to specify DNS policy explicitly to 'ClusterFirstWithHostNet'.
	DnsPolicy string `json:"dnsPolicy,omitempty"`

	// HostAliases is an optional list of hosts and IPs that will be injected into the pod's hosts file if specified. This is only valid for non-hostNetwork pods.
	HostAliases []V1HostAlias `json:"hostAliases,omitempty"`

	// Use the host's ipc namespace. Optional: Default to false.
	HostIPC bool `json:"hostIPC,omitempty"`

	// Host networking requested for this pod. Use the host's network namespace. If this option is set, the ports that will be used must be specified. Default to false.
	HostNetwork bool `json:"hostNetwork,omitempty"`

	// Use the host's pid namespace. Optional: Default to false.
	HostPID bool `json:"hostPID,omitempty"`

	// Specifies the hostname of the Pod If not specified, the pod's hostname will be set to a system-defined value.
	Hostname string `json:"hostname,omitempty"`

	// ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images used by this PodSpec. If specified, these secrets will be passed to individual puller implementations for them to use. For example, in the case of docker, only DockerConfig type secrets are honored. More info: https://kubernetes.io/docs/concepts/containers/images#specifying-imagepullsecrets-on-a-pod
	ImagePullSecrets []V1LocalObjectReference `json:"imagePullSecrets,omitempty"`

	// List of initialization containers belonging to the pod. Init containers are executed in order prior to containers being started. If any init container fails, the pod is considered to have failed and is handled according to its restartPolicy. The name for an init container or normal container must be unique among all containers. Init containers may not have Lifecycle actions, Readiness probes, or Liveness probes. The resourceRequirements of an init container are taken into account during scheduling by finding the highest request/limit for each resource type, and then using the max of of that value or the sum of the normal containers. Limits are applied to init containers in a similar fashion. Init containers cannot currently be added or removed. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
	InitContainers []V1Container `json:"initContainers,omitempty"`

	// NodeName is a request to schedule this pod onto a specific node. If it is non-empty, the scheduler simply schedules this pod onto that node, assuming that it fits resource requirements.
	NodeName string `json:"nodeName,omitempty"`

	// NodeSelector is a selector which must be true for the pod to fit on a node. Selector which must match a node's labels for the pod to be scheduled on that node. More info: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
	NodeSelector map[string]string `json:"nodeSelector,omitempty"`

	// The priority value. Various system components use this field to find the priority of the pod. When Priority Admission Controller is enabled, it prevents users from setting this field. The admission controller populates this field from PriorityClassName. The higher the value, the higher the priority.
	Priority int32 `json:"priority,omitempty"`

	// If specified, indicates the pod's priority. \"system-node-critical\" and \"system-cluster-critical\" are two special keywords which indicate the highest priorities with the former being the highest priority. Any other name must be defined by creating a PriorityClass object with that name. If not specified, the pod priority will be default or zero if there is no default.
	PriorityClassName string `json:"priorityClassName,omitempty"`

	// Restart policy for all containers within the pod. One of Always, OnFailure, Never. Default to Always. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy
	RestartPolicy string `json:"restartPolicy,omitempty"`

	// If specified, the pod will be dispatched by specified scheduler. If not specified, the pod will be dispatched by default scheduler.
	SchedulerName string `json:"schedulerName,omitempty"`

	// SecurityContext holds pod-level security attributes and common container settings. Optional: Defaults to empty.  See type description for default values of each field.
	SecurityContext *V1PodSecurityContext `json:"securityContext,omitempty"`

	// DeprecatedServiceAccount is a depreciated alias for ServiceAccountName. Deprecated: Use serviceAccountName instead.
	ServiceAccount string `json:"serviceAccount,omitempty"`

	// ServiceAccountName is the name of the ServiceAccount to use to run this pod. More info: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
	ServiceAccountName string `json:"serviceAccountName,omitempty"`

	// Share a single process namespace between all of the containers in a pod. When this is set containers will be able to view and signal processes from other containers in the same pod, and the first process in each container will not be assigned PID 1. HostPID and ShareProcessNamespace cannot both be set. Optional: Default to false. This field is alpha-level and is honored only by servers that enable the PodShareProcessNamespace feature.
	ShareProcessNamespace bool `json:"shareProcessNamespace,omitempty"`

	// If specified, the fully qualified Pod hostname will be \"<hostname>.<subdomain>.<pod namespace>.svc.<cluster domain>\". If not specified, the pod will not have a domainname at all.
	Subdomain string `json:"subdomain,omitempty"`

	// Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates delete immediately. If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
	TerminationGracePeriodSeconds int64 `json:"terminationGracePeriodSeconds,omitempty"`

	// If specified, the pod's tolerations.
	Tolerations []V1Toleration `json:"tolerations,omitempty"`

	// List of volumes that can be mounted by containers belonging to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes
	Volumes []V1Volume `json:"volumes,omitempty"`
}

[Spedoske]

Any ideas on making Acto be able to match similar schemas between Java implementation and official Kubernetes implementation? @tianyin @tylergu

[tianyin]

@Spedoske I guess your question is whether there are "automatic" way to establish the mapping?

I don't have a magic way. In fact, I wonder whether the problem is specific to this strimzi operator or to any Java operator.

I would say, for now, let's just put up a manual mapping and later see whether there is a need to do some automation.

Let me know if I misunderstand the problem.

[tianyin]

Close this issue as @Spedoske is no longer working on it.

@MarkintoshZ will take it over and #274 is created.