azure_monitor send fails with 400 - bad request due to more than 10 dimensions in metric

[thecrazymonkey]

Relevant section of values.yaml:

 outputs:
    - azure_monitor:
        timeout: "10s"
        skip_invalid: true
        namespace_prefix: "cp/"
    - health:
        service_address: "http://:8888"
  inputs:
    - prometheus:
        metric_version: 2
        response_timeout: "10s"
        monitor_kubernetes_pods: true
        monitor_kubernetes_pods_namespace: "test"

System info:

Telegraf 1.14
Azure AKS

Steps to reproduce:

1. Configure Prometheus input plugin to scrape from prometheus endpoints
2. Deploy telegraf using helm chart

Expected behavior:

Prometheus metrics delivered to Azure Monitor

Actual behavior:

Send fails with response
Error writing to outputs.azure_monitor: failed to write batch: [400] 400 Bad Request {"error":{"code":"BadRequest","message":"'dimValues' and 'dimNames' lengths should be lesser than 10\r\n"}}

Additional info:

The issue is here:

telegraf/plugins/outputs/azure_monitor/azure_monitor.go

Line 395 in fefd7ff

if len(dimensionNames) > 10 {

The logic above allows 11 items in the array - should be >= 10

[danielnelson]

Thanks for the bug report, we will get this fixed. In the meantime, and perhaps even afterwards, you may want to use the tag_limit processor. This will allow you to limit the number of tags and give some control over which tags to keep when over the limit.

[thecrazymonkey]

Thanks! Did not realize that I can fix it this way. Much appreciated!

[danielnelson]

I added some build links to #7336 with the pull request changes too, would be much appreciated if you could verify the fix.