ec2_metadata_facts - fix AttributeError when running on older managed…

… hosts (#1359)

ec2_metadata_facts - fix AttributeError when running on older managed hosts

SUMMARY
Unlike most of our modules, the ec2_metadata_facts is always expected to run on managed hosts, and doesn't depend on the AWS SDKs.
With Python 2.7 support still in place for managed hosts on all supported Ansible versions it makes sense, in the specific case of ec2_metadata_facts, to continue to support Python 2.7 for now.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
ec2_metadata_facts
ADDITIONAL INFORMATION
fixes: #1358

Reviewed-by: Bikouo Aubin
Reviewed-by: Alina Buzachis

changelogs/fragments/1358-ec2_metadata_facts.yml

@@ -0,0 +1,3 @@
+bugfixes:
+- ec2_metadata_facts - fix ``AttributeError`` when running the ec2_metadata_facts module on Python 2 managed nodes
+  (https://github.com/ansible-collections/amazon.aws/issues/1358).

plugins/modules/ec2_metadata_facts.py

@@ -445,6 +445,14 @@
 
 socket.setdefaulttimeout(5)
 
+# The ec2_metadata_facts module is a special case, while we generally dropped support for Python < 3.6
+# this module doesn't depend on the SDK and still has valid use cases for folks working with older
+# OSes.
+try:
+    json_decode_error = json.JSONDecodeError
+except AttributeError:
+    json_decode_error = ValueError
+
 
 class Ec2Metadata:
     ec2_metadata_token_uri = 'http://169.254.169.254/latest/api/token'
@@ -543,7 +551,7 @@ def fetch(self, uri, recurse=True):
                         self._data['%s' % (new_uri)] = content
                         for (key, value) in json_dict.items():
                             self._data['%s:%s' % (new_uri, key.lower())] = value
-                    except (json.JSONDecodeError, AttributeError):
+                    except (json_decode_error, AttributeError):
                         self._data['%s' % (new_uri)] = content  # not a stringified JSON string
 
     def fix_invalid_varnames(self, data):

tests/integration/targets/ec2_metadata_facts/playbooks/setup.yml

@@ -19,6 +19,13 @@
     subnet_cidr: '10.{{ 256 | random(seed=vpc_seed) }}.32.0/24'
 
   tasks:
+  - set_fact:
+      # As lookup plugins don't have access to module_defaults
+      connection_args:
+        region: "{{ aws_region }}"
+        aws_access_key: "{{ aws_access_key }}"
+        aws_secret_key: "{{ aws_secret_key }}"
+        aws_session_token: "{{ security_token | default(omit) }}"
 
   - include_role:
       name: '../setup_sshkey'
@@ -126,13 +133,39 @@
       tags:
         snake_case_key: a_snake_case_value
         camelCaseKey: aCamelCaseValue
-      wait: True
     register: ec2_instance
     vars:
       ansible_python_interpreter: "{{ botocore_virtualenv_interpreter }}"
 
+  - set_fact:
+      ec2_ami_id_py2: "{{ lookup('aws_ssm', '/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2', **connection_args) }}"
+      ec2_ami_ssh_user_py2: "ec2-user"
+
+  - name: Create an instance to test with using Python 2
+    ec2_instance:
+      state: running
+      name: "{{ resource_prefix }}-ec2-metadata-facts-py2"
+      image_id: "{{ ec2_ami_id_py2 }}"
+      vpc_subnet_id: "{{ vpc_subnet_id }}"
+      security_group: "{{ vpc_sg_id }}"
+      instance_type: t2.micro
+      key_name: "{{ ec2_key_name }}"
+      network:
+        assign_public_ip: true
+        delete_on_termination: true
+      metadata_options:
+          instance_metadata_tags: enabled
+      tags:
+        snake_case_key: a_snake_case_value
+        camelCaseKey: aCamelCaseValue
+      wait: True
+    register: ec2_instance_py2
+    vars:
+      ansible_python_interpreter: "{{ botocore_virtualenv_interpreter }}"
+
   - set_fact:
       ec2_instance_id: "{{ ec2_instance.instances[0].instance_id }}"
+      ec2_instance_id_py2: "{{ ec2_instance_py2.instances[0].instance_id }}"
 
   - name: Create inventory file
     template:
@@ -143,3 +176,8 @@
       port: 22
       host: '{{ ec2_instance.instances[0].public_ip_address }}'
       timeout: 1200
+
+  - wait_for:
+      port: 22
+      host: '{{ ec2_instance_py2.instances[0].public_ip_address }}'
+      timeout: 1200

tests/integration/targets/ec2_metadata_facts/playbooks/teardown.yml

@@ -20,6 +20,7 @@
       state: absent
       instance_ids:
         - "{{ ec2_instance_id }}"
+        - "{{ ec2_instance_id_py2 }}"
       wait: True
     ignore_errors: true
     retries: 5

tests/integration/targets/ec2_metadata_facts/templates/inventory.j2

@@ -1,11 +1,25 @@
-[testhost]
+[testhost_py3]
 "{{ ec2_instance.instances[0].public_ip_address }}"
 
+[testhost_py2]
+"{{ ec2_instance_py2.instances[0].public_ip_address }}"
+
+[testhost:children]
+testhost_py3
+testhost_py2
+
 [testhost:vars]
-ansible_user={{ ec2_ami_ssh_user }}
 ansible_ssh_private_key_file="{{ sshkey }}"
 ansible_python_interpreter=/usr/bin/env python
 
+[testhost_py3:vars]
+ansible_user="{{ ec2_ami_ssh_user }}"
+image_id="{{ ec2_ami_id }}"
+
+[testhost_py2:vars]
+ansible_user="{{ ec2_ami_ssh_user_py2 }}"
+image_id="{{ ec2_ami_id_py2 }}"
+
 [all:vars]
 # Template vars that will need to be used in used in tests and teardown
 vpc_id="{{ vpc_id }}"
@@ -16,5 +30,5 @@ vpc_igw="{{ vpc_igw_id }}"
 vpc_route_table_id="{{ vpc_route_table_id }}"
 ec2_key_name="{{ ec2_key_name }}"
 availability_zone="{{ availability_zone }}"
-image_id="{{ ec2_ami_id }}"
 ec2_instance_id="{{ ec2_instance_id }}"
+ec2_instance_id_py2="{{ ec2_instance_id_py2 }}"