elb_application_lb_info - Ability to filter results (slow/failed attempts)

[andreiboost]

Summary

elb_application_lb_info doesn't have any option to filter/query specific ELBs beyond lookup by ARN/name. I am trying to filter by VPC ID, but this is not possible with this module. For example amazon.aws.ec2_security_group_info supports this exact case, i.e.

- name: Find all Security Groups
  amazon.aws.ec2_security_group_info:
    filters:
      vpc-id: "vpc-12345"

To achieve the same result with ALBs, I have to resort to the AWS CLI, for example:

- name: Get VPC ALBs
  check_mode: false
  changed_when: false
  ansible.builtin.command:
    cmd: >-
      aws elbv2 describe-load-balancers
      --query 'LoadBalancers[?VpcId==`vpc-12345` && Type==`application`]'
      --output json
  register: __cluster_albs_json

- name: Parse ALB output
  ansible.builtin.set_fact:
    cluster_albs: "{{ __cluster_albs_json.stdout | from_json }}"

For my purposes, this works but the output isn't the same nor as complete as elb_application_lb_info. Please consider adding a filters parameter to this module. elb_application_lb_info often fails due to throttling in accounts with many ALBs, i.e.

Failed to describe listener rules: An error occurred (Throttling) when calling the DescribeRules operation (reached max retries: 4): Rate exceeded

Issue Type

Feature Idea

Component Name

elb_application_lb_info

Additional Information

- name: Find all ALBs in VPC
  amazon.aws.elb_application_lb_info:
    filters:
      vpc-id: "vpc-12345"

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[tremble]

Hi @andreiboost,

Thanks for taking the time to submit this feature request. Unfortunately the AWS ELB APIs doesn't support performing a server-side query like this, as such it's unlikely that this would be implemented any time soon.

The workaround you listed using the "query" parameter from awscli is actually performing a JSONPath lookup on the data from all ELBs.

What should be possible is to perform the lookup in pure Ansible rather than the AWS CLI:

- name: Fetch ELB descriptions
  amazon.aws.elb_application_lb_info:
  register: elb_info

- name: Find ALBs in VPC
  set_fact:
    vpc_albs: "{{ 
        elb_info.load_balancers
        | selectattr('vpc_id', 'equalto', desired_vpc)
        | selectattr('type', 'equalto', 'application')
      }}"

(Whitespace in the filter is for legibility, it can be all on a single line)

[andreiboost]

@tremble interesting, then perhaps the AWS CLI isn't timing out because it doesn't fetch as much stuff? (Listeners, Rules)

[tremble]

perhaps the AWS CLI isn't timing out because it doesn't fetch as much stuff?

Looking at the code, and the error message (Failed to describe listener rules) this is quite possible, automatic retries appear to only be half implemented in the module. Would you be able to see if #1768 at least fixes the error you're seeing (it'll still be slow)?

[andreiboost]

It appears to fix that problem, but it is very slow. Takes about 90-100 seconds to fetch >100 ALBs.

Would you suggest I open another request for skipping rules and/or listeners? That should significantly speed it up when all I want is their ARNs or names.

[tremble]

Would you suggest I open another request for skipping rules and/or listeners? That should significantly speed it up when all I want is their ARNs or names.

A Pull request would be very much appreciated, but please keep in mind that the default behaviour needs to stay as it is today to avoid breaking existing playbooks.

Long term, I think supporting "filter" before we query the remaining information is probably the best option. Because there are other APIs which also don't support server side filtering (KMS springs to mind) this would be best handled as common code in module_utils that we could apply to other modules too. (An RFE to AWS wouldn't be a bad idea either)

[andreiboost]

Understood, I'll see when I get some time to work on a PR. Adding a generic "filter" to utils would be quite a big undertaking for me as I'm not familiar with this codebase but adding options to this module to skip some things is absolutely doable. Later this week or sometime next week should be possible. Thanks for your help (and PR!).