Refactor Email Destination

[qreshi]

Issue #, if available: #3

Description of changes:

This PR is built upon the original one for the Email Destination feature enhancement: #102

The purpose of these changes is to refactor the Email Destination to allow for more flexible use cases and provide ease in scaling out the use of email notifications in Alerting.

Email Accounts

These changes introduce email_accounts which are stored in the opendistro-alerting-config index like other Alerting configurations. This allows for there to be multiple sender emails configured for use in Destinations. A single configured email_account can be used in multiple Destinations and is referred to by its id in the Destination configuration so that changing the email_account reflects the change in all Destinations that reference it.

# Example email account configuration
curl -X GET "localhost:9200/_opendistro/_alerting/email_accounts/YjY7mXMBx015759_IcfW"
{
  "_id" : "YjY7mXMBx015759_IcfW",
  "_seq_no" : 3,
  "_primary_term" : 2,
  "email_account" : {
    "schema_version" : 2,
    "name" : "test_account",
    "email" : "[email protected]",
    "host" : "smtp.mail.com",
    "port" : 465,
    "method" : "ssl"
  }
}

The Elasticsearch keystore will still be used for storing the username and password for an email when authentication is required. Since there can now be multiple sender emails the stored settings will be namespaced based on the name of the email_account. For example, if the email_account had the name "test_account" the settings to be added to the keystore would be as follows:

opendistro.alerting.destination.email.test_account.username
opendistro.alerting.destination.email.test_account.password

Email Groups

This PR also introduces email_groups which are similar to mailing lists. These email_groups can be used to refer to a set group of emails when creating Email Destinations. Similar to email_accounts, these email_groups can be used in multiple Destinations. The recipient list for an Email Destination can also be a combination of email_groups and single emails.

# Example email group configuration
curl -X GET "localhost:9200/_opendistro/_alerting/email_groups/YzY-mXMBx015759_dscs"
{
  "_id" : "YzY-mXMBx015759_dscs",
  "_seq_no" : 4,
  "_primary_term" : 2,
  "email_group" : {
    "schema_version" : 2,
    "name" : "test_email_group",
    "emails" : [
      {
        "email" : "[email protected]"
      },
      {
        "email" : "[email protected]"
      }
    ]
  }
}

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov]

Codecov Report

Merging #244 into master will decrease coverage by 3.48%.
The diff coverage is 65.32%.

@@             Coverage Diff              @@
##             master     #244      +/-   ##
============================================
- Coverage     78.61%   75.13%   -3.49%     
- Complexity      159      191      +32     
============================================
  Files            91      134      +43     
  Lines          3273     4403    +1130     
  Branches        464      581     +117     
============================================
+ Hits           2573     3308     +735     
- Misses          460      786     +326     
- Partials        240      309      +69     

Impacted Files	Coverage Δ	Complexity Δ
...search/alerting/model/destination/CustomWebhook.kt	87.32% <ø> (ø)	0.00 <0.00> (ø)
...erting/transport/TransportGetEmailAccountAction.kt	9.52% <9.52%> (ø)	0.00 <0.00> (?)
...alerting/transport/TransportGetEmailGroupAction.kt	9.52% <9.52%> (ø)	0.00 <0.00> (?)
...endistroforelasticsearch/alerting/MonitorRunner.kt	66.42% <16.00%> (-11.36%)	0.00 <0.00> (ø)
...erting/resthandler/RestSearchEmailAccountAction.kt	19.23% <19.23%> (ø)	0.00 <0.00> (?)
...alerting/resthandler/RestSearchEmailGroupAction.kt	19.23% <19.23%> (ø)	0.00 <0.00> (?)
...ing/transport/TransportDeleteEmailAccountAction.kt	22.22% <22.22%> (ø)	0.00 <0.00> (?)
...rting/transport/TransportDeleteEmailGroupAction.kt	22.22% <22.22%> (ø)	0.00 <0.00> (?)
...ing/transport/TransportSearchEmailAccountAction.kt	28.57% <28.57%> (ø)	0.00 <0.00> (?)
...rting/transport/TransportSearchEmailGroupAction.kt	28.57% <28.57%> (ø)	0.00 <0.00> (?)
... and 100 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 67b5598...1406961. Read the comment docs.

[ftianli-amzn]

We could find a better way to deal with the "patch" coverage check.

[belkirdi]

When do you think this feature will be released?
Thanks

[skkosuri-amzn]

Looks like coverage is going down by 3.57%, that seems to more. Is it possible to increase the coverage?

[lezzago]

Mostly just minor comments, everything else looks good to me

[skkosuri-amzn]

Based on offline discussions we plan to address the coverage reduction in followup PR's.

[fbaligand]

Great!
Happy to see this PR merged!
Thanks all for your work on this!