New Relic alert manager

newrelic-alert-manager is a Kubernetes operator which automates the management of New Relic dashboards, alert policies and notification channels.

It allows end users of a Kubernetes cluster to define these resources as Kubernetes Custom Resources

Project status

The project is currently considered stable and suitable for production use.

Supported features

Alerts

The newrelic-alert-manager currently supports the management of the following alerting conditions

• NRQL alerting conditions
• APM alerting conditions
• Infra alerting conditions of type infra_metric

If you are unable to create a particular alerting condition due to lack of support by the operator or the New Relic API, you can try to fall back to defining it as a NRQL alerting condition instead. One such example is given in the FAQ section.

Notification channels

With respect to notification channels, the currently supported types are Email, Slack and Opsgenie channels.

Dashboards

The dashboard API is fully covered by the operator.

Deployment

In order to deploy the operator, execute the following steps:

• Download and extract the Kubernetes manifests (deploy.tar.gz) for the latest release
• Add your secrets to deploy/1-secret.yaml
  • Add the base64 encoded New Relic admin password
  • Optionally, add the default Slack webhook URL for SlackNotificationChannels.
  • Optionally, add the Opsgenie API key for OpsgenieNotificationChannels.
• Deploy the custom resource definitions by running kubectl apply -f deploy/crds/
• Deploy the operator manifests by running kubectl apply -f deploy/

Example Usage

Please check the examples folder to find out how to deploy alert policies together with notification channels.

For more detailed information, you can take a look at the complete API reference.

Debugging resources

If you applied an alert policy but it was not created in New Relic, you can check the status of the policy using kubectl describe alertpolicies <policy-name>. If there was an error while creating the policy, it will be shown in the Status.reason field. Similarly, you can use kubectl describe to debug dashboards and notification channels as well.

FAQ

Where can I find a more information on how each alerting condition parameter affects the alert policy?

The alert condition parameters are best explained by the documentation for the New Relic REST API Some examples include:

• apmConditions.alertThreshold.metric
• apmConditions.alertThreshold.timeFunction
• nrqlConditions.valueFunction

You can review the Alerts conditions API field names page for more information.

How do I create an APM condition of type Web transaction percentiles

Unfortunately, it is not possible to use New Relic's REST API to create these types of conditions. However, you can try to define a NRQL alerting condition instead. The query parameter could be defined as follows:

SELECT percentile(totalTime) FROM Transaction WHERE appName = '<your APM application name>'