Migrate ec2 vpc endpoint modules #392

jillr · 2021-06-21T18:16:38Z

SUMMARY

Moves ec2_vpc_endpoint, ec2_vpc_endpoint_info, and ec2_vpc_endpoint_service_info from community.aws to amazon.aws via git patch.

As discussed at #354 (review), this PR initially demonstrates the result of the git patch process. When that is approved, the git patch process will be repeated directly onto main to preserve git history. This PR will then be rebased on the updated main with the runtime.yml, deprecation, and example documentation changes.

ISSUE TYPE

New Module Pull Request

COMPONENT NAME

ec2_vpc_endpoint
ec2_vpc_endpoint_info
ec2_vpc_endpoint_service_info

ADDITIONAL INFORMATION

Patch process:

git log --pretty=email --patch-with-stat --reverse --full-index --binary -- tests/integration/targets/ec2_vpc_endpoint* plugins/modules/ec2_vpc_endpoint* > ../../amazon/aws/patch

cd ../amazon/aws/

git am < patch

* Rename core collection Rename references to ansible.amazon to amazon.aws. * Rename community.amazon to community.aws Fix pep8 line lengths for rewritten amazon.aws imports * Missed a path in shippable.sh * Dependency repos moved

* Remove ANSIBLE_METADATA entirely, see ansible/ansible/pull/69454. Remove `license` field from galaxy.yml, in favor of `license_file`.

* Update module deprecations Switch version to `removed_at_date` * Don't install amazon.aws from galaxy We've been using galaxy to install amazon.aws in shippable, but that doesn't really work if we aren't publising faster. Get that collection from git so it is most up to date. * We need to declare python test deps now * missed a python dep

Updated module examples with FQCN Signed-off-by: Abhijeet Kasurde <[email protected]>

* Update docs Remove .git from repo url so links in readme will generate correctly Add required ansible version Run latest version of add_docs.py Add version_added string to modules * galaxy.yml was missing authors

Signed-off-by: Abhijeet Kasurde <[email protected]>

* Update comments to reference AnsibleAWSModule rather than AnsibleModule * Bulk re-order imports and split onto one from import per-line. * Add AnsibleAWSModule imports * Migrate boto 2 based modules to AnsibleAWSModule * Move boto3-only modules over to AnsibleAWSModule * Remove extra ec2_argument_spec calls - not needed now we're using AnsibleAWSModule * Remove most HAS_BOTO3 code, it's handled by AnsibleAWSModule * Handle missing Boto 2 consistently (HAS_BOTO) * Remove AnsibleModule imports * Changelog fragment

…r(e) or e.me… (ansible-collections#26) * Py3 compat error handling: use to_native(e) instead of str(e) or e.message * PR comment changes, use fail_json_aws and is_boto3_error_code

…ollections#188) * Migrate from boto3_conn to module.client * Simplify error handling when creating connections * Simplify Region handling * Remove unused imports * Changelog

* Split imports and reorder * Import camel_dict_to_snake_dict and snake_dict_to_camel_dict direct from ansible.module_utils.common.dict_transformations * Remove unused imports * Route53 Info was migrated to Boto3 drop the HAS_BOTO check and import * changelog

* Split imports and sort * Move camel_dict_to_snake_dict imports to ansible.module_utils.common.dict_transformations * Cleanup unused imports * Bulk migration to fail_json_aws * Changelog

* deprecate policy_file * ignore file * changelog

* Ensure ec2_vpc_endpoint returns True when deleting an Endpoint Return not changed when state=absent and endpoint has already been deleted * Add minimal endpoint tests

* More aliases cleanup * Mark ec2_classic_lb tests unstable * Add more comments about why tests aren't enabled

* Added support for 'vpc_endpoint_type'. * Integration test for the 'vpc_endpoint_type' feature. * Added choices in documentation. * Added changelog.

…-collections#465)

* Add no_log=False to mark some more false-positives of the no_log check. * More false-positives confirmed by tremble.

* New module - ec2_vpc_endpoint_service_info * Deprecate querying services through ec2_vpc_endpoint_info * Attempt to cope with some services which have who possible endpoints Co-authored-by: Jill R <[email protected]>

* Stabilize and improve ec2_vpc_endpoint modules - Add tagging support - Make idempotent - Better exception handling - Better check_mode support - Use module_utils for common functions - Enable retries on common AWS failures * Make endpoint deletion idempotent in check_mode * Sanity fixes * Address review feedback

ansibullbot · 2021-06-21T18:22:00Z

cc @s-hertel @tremble @wimnat
click here for bot help

jillr · 2021-06-21T21:09:37Z

recheck

tremble

with #298 boto3 became the default requirement for modules, this isn't the case for community.aws yet

tremble · 2021-06-22T08:09:20Z

plugins/modules/ec2_vpc_endpoint_service_info.py

+version_added: 1.5.0
+description:
+  - Gets details related to AWS VPC Endpoint Services.
+requirements: [ boto3 ]


Suggested change

requirements: [ boto3 ]

post #298 this isn't needed in amazon.aws (ansible-collections/community.aws#569 is still pending)

Shoot, 569 is totally on me - thanks! :)

569 has now been merged, not sure how you'd like to handle the updates here.

tremble · 2021-06-22T08:09:57Z

plugins/modules/ec2_vpc_endpoint_info.py

+description:
+  - Gets various details related to AWS VPC endpoints.
+  - This module was called C(ec2_vpc_endpoint_facts) before Ansible 2.9. The usage did not change.
+requirements: [ boto3 ]


Suggested change

requirements: [ boto3 ]

tremble · 2021-06-22T08:10:16Z

plugins/modules/ec2_vpc_endpoint.py

+  - Creates AWS VPC endpoints.
+  - Deletes AWS VPC endpoints.
+  - This module supports check mode.
+requirements: [ boto3 ]


Suggested change

requirements: [ boto3 ]

Disable the `squash-merge` merge strategy just the time we land ansible-collections/amazon.aws#392

goneri · 2021-06-25T14:46:54Z

I add a Depends-On: ansible/project-config#862 to be sure we don't squash the commits during the PR landing :-).

goneri · 2021-06-25T19:54:03Z

@jillr, I've pushed https://github.com/ansible-collections/amazon.aws/pull/393/commits which is based on your branch.

The difference is that I add a footer in the commit message. It includes a URL pointing on the original commit.

I use the following script to build this up: https://github.com/goneri/promote_aws_modules

jillr · 2021-07-07T20:26:05Z

@goneri Should we close this and do all of the migrations with the automation? Or do you think it's worth landing this one since the PR is open?

goneri · 2021-07-12T07:18:46Z

@jillr, we can actually close both and reopen a clean PR. I did the script as a proof of concept but anyone can actually continue from that.

jillr · 2021-07-28T22:11:37Z

We have a new way of doing these; closing and will open a new PR

* Remove cloudformation (amazon.aws) placebo recordings * Move module unit tests to the right location so ansible-test finds them

* Remove cloudformation (amazon.aws) placebo recordings * Move module unit tests to the right location so ansible-test finds them This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections/community.aws@1cf83b1

jillr and others added 24 commits June 21, 2021 10:40

Initial commit

92d53c9

migration test cleanup

0e3509f

Rename collection (ansible-collections#12)

fb9ad47

* Rename core collection Rename references to ansible.amazon to amazon.aws. * Rename community.amazon to community.aws Fix pep8 line lengths for rewritten amazon.aws imports * Missed a path in shippable.sh * Dependency repos moved

Remove METADATA and cleanup galaxy.yml (ansible-collections#70)

f4d7b63

* Remove ANSIBLE_METADATA entirely, see ansible/ansible/pull/69454. Remove `license` field from galaxy.yml, in favor of `license_file`.

Update Examples with FQCN (ansible-collections#67)

b1e3252

Updated module examples with FQCN Signed-off-by: Abhijeet Kasurde <[email protected]>

Update docs (ansible-collections#99)

b053393

* Update docs Remove .git from repo url so links in readme will generate correctly Add required ansible version Run latest version of add_docs.py Add version_added string to modules * galaxy.yml was missing authors

Docs: sanity fixes (ansible-collections#133)

d72cedd

Signed-off-by: Abhijeet Kasurde <[email protected]>

Python 3 compatibility error handling: use to_native(e) instead of st…

0397789

…r(e) or e.me… (ansible-collections#26) * Py3 compat error handling: use to_native(e) instead of str(e) or e.message * PR comment changes, use fail_json_aws and is_boto3_error_code

Cleanup: Bulk Migration from boto3_conn to module.client() (ansible-c…

7c6a87f

…ollections#188) * Migrate from boto3_conn to module.client * Simplify error handling when creating connections * Simplify Region handling * Remove unused imports * Changelog

Bulk migration to fail_json_aws (ansible-collections#361)

599dddd

* Split imports and sort * Move camel_dict_to_snake_dict imports to ansible.module_utils.common.dict_transformations * Cleanup unused imports * Bulk migration to fail_json_aws * Changelog

ec2_vpc_endpoint - deprecate policy_file (ansible-collections#366)

cde6a4c

* deprecate policy_file * ignore file * changelog

ec2_vpc_endpoint - fixup deletion 'changed' (ansible-collections#362)

5f48c85

* Ensure ec2_vpc_endpoint returns True when deleting an Endpoint Return not changed when state=absent and endpoint has already been deleted * Add minimal endpoint tests

Yet more integration test aliases file cleanup (ansible-collections#431)

bb5462e

* More aliases cleanup * Mark ec2_classic_lb tests unstable * Add more comments about why tests aren't enabled

Added support for 'vpc_endpoint_type'. (ansible-collections#460)

4dd5e48

* Added support for 'vpc_endpoint_type'. * Integration test for the 'vpc_endpoint_type' feature. * Added choices in documentation. * Added changelog.

Fix version_added and changelog from ansible-collections#460 (ansible…

59319ad

…-collections#465)

More no_log=False to fix sanity tests (ansible-collections#474)

33afe8a

* Add no_log=False to mark some more false-positives of the no_log check. * More false-positives confirmed by tremble.

Update ec2_vpc_endpoint_info AWS retries

8348213

Add a unique tag to test against

c6a8724

Migrate ec2_vpc_endpoint* modules to amazon.aws

23defe8

ansibullbot added community_review integration tests/integration module module needs_triage new_module New module labels Jun 21, 2021

ansibullbot added new_plugin New plugin plugins plugin (any type) tests tests labels Jun 21, 2021

Add a changelog and sanity ignores

fbd7cdc

jillr changed the title ~~Migrate ec2 endpoint~~ Migrate ec2 vpc endpoint modules Jun 21, 2021

Do symlinks correctly

36b8a47

tremble requested changes Jun 22, 2021

View reviewed changes

jillr requested a review from goneri June 22, 2021 23:48

goneri mentioned this pull request Jun 25, 2021

amazon.aws: temp turn on the merge strategy ansible/project-config#862

Merged

goneri added a commit to goneri/project-config that referenced this pull request Jun 25, 2021

amazon.aws: temp turn on the merge strategy

76fe702

Disable the `squash-merge` merge strategy just the time we land ansible-collections/amazon.aws#392

goneri mentioned this pull request Jun 28, 2021

[WIP] Remove ec2_vpc_endpoint* modules and tests ansible-collections/community.aws#606

Closed

gravesm removed the needs_triage label Jun 29, 2021

goneri assigned alinabuzachis Jul 12, 2021

jillr closed this Jul 28, 2021

jillr deleted the migrate_ec2_endpoint branch July 28, 2021 22:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate ec2 vpc endpoint modules #392

Migrate ec2 vpc endpoint modules #392

jillr commented Jun 21, 2021 •

edited by goneri

Loading

ansibullbot commented Jun 21, 2021

jillr commented Jun 21, 2021

tremble left a comment

tremble Jun 22, 2021

jillr Jun 22, 2021

tremble Jul 11, 2021

tremble Jun 22, 2021

tremble Jun 22, 2021

goneri commented Jun 25, 2021

goneri commented Jun 25, 2021

jillr commented Jul 7, 2021

goneri commented Jul 12, 2021

jillr commented Jul 28, 2021

Migrate ec2 vpc endpoint modules #392

Migrate ec2 vpc endpoint modules #392

Conversation

jillr commented Jun 21, 2021 • edited by goneri Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

ansibullbot commented Jun 21, 2021

jillr commented Jun 21, 2021

tremble left a comment

Choose a reason for hiding this comment

tremble Jun 22, 2021

Choose a reason for hiding this comment

jillr Jun 22, 2021

Choose a reason for hiding this comment

tremble Jul 11, 2021

Choose a reason for hiding this comment

tremble Jun 22, 2021

Choose a reason for hiding this comment

tremble Jun 22, 2021

Choose a reason for hiding this comment

goneri commented Jun 25, 2021

goneri commented Jun 25, 2021

jillr commented Jul 7, 2021

goneri commented Jul 12, 2021

jillr commented Jul 28, 2021

jillr commented Jun 21, 2021 •

edited by goneri

Loading