✨ (WIP) feat: initialize kube-state-metrics plugin #3076
Conversation
k8s-ci-robot
added
the
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Kavinjsir The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
size/L
Kavinjsir
force-pushed
the
feat/ksm-cr-metrics
branch
from
c6123bb to
98f9c0f
Compare
k8s-ci-robot
added
size/XXL
|
At this very beginning, I'm also wondering how we would like such plugin to move on? Rich content for the config fileFundamentally, guess the plugin is expected to extend the config file. We may need to figure out:
Whether it is necessary to install kube-state-metricsIn case the user does not have ksm running in the cluster, is it necessary for us to provide an option to install ksm? (Then the config can be loaded automatically in that case) Does it necessary to work with prometheus-operator/kube-prometheusOnly in case if we have many users using |
| spec: | ||
| resources:{{ range $i, $e := . }} | ||
| - groupVersionKind: | ||
| group: "{{.Group}}.{{.Domain}}" |
There was a problem hiding this comment.
Do we have to scaffold config file for all GVKs in the project, or can we allow users to select the GVK they would like to?
There was a problem hiding this comment.
Yeah, I think it a good idea to give config entry for users to decide which ones to be intrumented. 👍🏼
| metrics: | ||
| - name: "info" | ||
| each: | ||
| type: Info |
There was a problem hiding this comment.
Do we allow this to be user-configurable? I'm fine with the first phase being templated scaffolding, but we should probably think more about how to make this customizable in case we want to?
There was a problem hiding this comment.
Yeah, I guess the target is to make the metrics array 'configurable'.
The _info here is like a sample and practices.
It might can become a default one if considered common. (Well, the content/labels might need improvement)
|
@Kavinjsir adding some thoughts for the questions above:
Does KSM expose some generic metrics (https://github.com/kubernetes/kube-state-metrics/tree/master/docs#default-resources) that are relevant for the user to know? For example, controller-runtime or even client-go by default exposes certain metrics which are useful from an operator authors perspective. Are there any specific metrics out of the lot, which we can create wrappers around and utilize for our benefits?
IMO though kube-prometheus is used widely we should not assume our configurations or scaffoldings to be only based on the same. There are many production use cases wherein the entire monitoring stack can be off-cluster. Also, is there any generic workaround to identify any prometheus on-cluster instance and have an option to allow users to auto-load the configuration if they like to do so? |
| @@ -118,6 +118,11 @@ function scaffold_test_project { | |||
| $kb edit --plugins=grafana.kubebuilder.io/v1-alpha | |||
| fi | |||
|
|
|||
| if [[ $project == "project-v3" || $project == "project-v4" || $project =~ multigroup ]]; then | |||
There was a problem hiding this comment.
WDYT about:
change this sample project-v3-addon-and-grafana name for project-v3-declarative-v1
then, we have a new one as project-v3-metrics were we will add grafana and this one.
Same for v4.
There was a problem hiding this comment.
Yeah, I think maybe we can deal with this in a separate PR? Since this one maybe hold.
There was a problem hiding this comment.
@camilamacedo86 Here is the PR to refact testdata: #3086
|
@camilamacedo86 @varshaprasad96 As discussed in the meeting, it seems like |
|
/hold |
k8s-ci-robot
added
the
do-not-merge/hold
Kavinjsir
changed the title
|
Hi @varshaprasad96 glad to have your ideas with considerations on different perspectives! 👍🏼 The reason why I was wondering if necessary to provide approaches deploying ksm is because our feature is based on that... And about what you've mentioned of the monitoring stacks running outside cluster, which pretty makes sense, and so I can understand maybe it is not necessary for kubebuilder to deal with kube-prometheus... Actually, I'm interested in |
| namespace: | ||
| - metadata | ||
| - namespace |
There was a problem hiding this comment.
Won't this cause issues with cluster-scoped resources?
Kavinjsir
force-pushed
the
feat/ksm-cr-metrics
branch
from
98f9c0f to
0621d60
Compare
k8s-ci-robot
added
the
needs-rebase
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
The Kubernetes project currently lacks enough contributors to adequately respond to all PRs. This bot triages PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
@camilamacedo86 Since the origin issue is closed by the author where kube-state-metrics provides native support for custom resources, I guess we may close this PR at this moment? |
|
@Kavinjsir: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs. This bot triages PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
k8s-ci-robot
added
lifecycle/rotten
|
Close this PR as the origin issue was closed by the author. Feel free to open new issue or leave comment here for relative discussions. |
2022-11-17 Updates
Hold, we may consider the monitoring plugin for integrating this feature, OR phase 2 approach as making this an external plugin.
Motivation
This is a starting PR to resolve #2286
Description
Introduce a new optional plugin
kube-state-metrics.This metrics aims to provide configuration file loaded by KSM to populate state metrics of custom resources.
The state metrics should be generated based on user-defined subresource
status.This PR initializes the plugin to provide the very basic
_infometric of the CRs.A common usage:
"count custom resources by namespace, cluster, etc"
Usage
The plugin is temporary designed to be triggered by
editsubcommand:# Execute the command after APIs are generated kubebuilder edit --plugins=kube-state-metrics.kubebuilder.io/v1-alphaOnce executed, the plugin will look for the apis recorded in
PROJECT. Then, it will generate a config yaml at:kube-state-metrics/cr-config.yamlThe file can be loaded by
kube-state-metricsthrough--custom-resource-state-config-file /path/to/kube-state-metrics/cr-config.yaml.The content refers to https://github.com/kubernetes/kube-state-metrics/blob/v2.6.0/docs/customresourcestate-metrics.md
TODO:
Example
Below is a practice over memcached-operator: