Added tools/slo-rules-generator #77
Conversation
There was a problem hiding this comment.
Great, generally OK from me
There are a plenty of things we can iterate on, including generating of the alerts etc. leaving out all the joining but for now this is GTG
I added some minor nits for the future improvements, but I'm ok with merging it as is and iterating later.
👍
| } | ||
| } | ||
|
|
||
| func (d Domain) IsValid() []error { |
There was a problem hiding this comment.
Future improvement:
Instead of adding everywhere the IsValid function which needs to be explicitly called, It's easier to override the UnmarshallYAML functions which would do the validation right a way
| version: 6 | ||
| alerting: | ||
| team: [email protected] | ||
| escalate: [email protected] |
There was a problem hiding this comment.
Future improvement:
Both these fields feel quite proprietary and bounded to our internal setup.
Maybe we might just add a something line additional_labels? 🤔
| type SloConfiguration map[string]Domain | ||
|
|
||
| func main() { | ||
| if len(os.Args) != 2 { |
There was a problem hiding this comment.
Feature improvement:
Use Kingpin which simply solves all of this
|
Also, I'd consider mentioning it in the CHANGELOG marked as experimental just to let people know what direction we are heading |
Co-authored-by: Martin Chodur <[email protected]>
Co-authored-by: Martin Chodur <[email protected]>
Co-authored-by: Martin Chodur <[email protected]>
Co-authored-by: Martin Chodur <[email protected]>
Co-authored-by: Martin Chodur <[email protected]>
|
Thanks for the review @FUSAKLA, I'm merging it in order to move this forward. I agree that remaining comments should be addressed, however leaving them to a future refactor. |
Adding tool to generate SLO recording rules with metadata for domains.
The original motivation was adding a new recording rule -
slo:burn_rate_thresholdin addition to already existingslo:stable_versionandslo:violation_ratio_threshold. Introduction of SLO domain configuration lets us to automatically generate the metrics mentioned above and allows us further modify burn_rate alerts on per-domain basis.*In future we will probably be generating custom burn-rate alerts for each domain,class,type instead of customizing the alerts via recording rules, but we are not there yet and this PR basically just sync our current internal state of tooling to github :) *