Discovery: Determine usefulness of Datadog recommended error alerts

[robrap]

This ticket is about exploring the usefulness of the following types of error monitors in Datadog (DD).

There are two types of error-related monitors that the DD UI helps you create from the Service Catalog => Monitors tab:

1. Error Tracking Monitor

• New issues detected
• APM detects errors on your services so that you can quickly fix them. This monitor alerts you when new errors are detected on your service.
• Important: This type of alert might not be very useful until we get more errors into Error Tracking. See linked ticket.

2. APM Monitor

• Errors: Moving rollup for spiky traffic
• Monitor compares your service's error rate over the last 10 minutes vs the last 4 hours and alerts if the ratio is above 3. This is useful to detect spiky traffic patterns that may be causing increased error rates.

Note: The Service Catalog => Monitors tab can be reached from the Monitors column in the top-level Service Catalog page.

[robrap]

The following ticket is for setting up one of these types of alerts:

• Add "New Issue" (error tracking) monitor for edx-edxapp-lms #706

[robrap]

I'm closing this ticket in favor of #706.