Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3_bucket: Refactor s3 bucket module #2057

Conversation

mandar242
Copy link
Contributor

SUMMARY

Refactor s3_bucket module

  • Break down functions into smaller scoped functions.
  • Add documentation to each function to clarify what it does.
  • Add type hints to functions.
ISSUE TYPE
  • Bugfix Pull Request
  • Docs Pull Request
COMPONENT NAME

s3_bucket

ADDITIONAL INFORMATION

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/ef19dc32528d45e1b5ffd09c69e9f0ed

✔️ ansible-galaxy-importer SUCCESS in 4m 49s
✔️ build-ansible-collection SUCCESS in 16m 33s
✔️ ansible-test-splitter SUCCESS in 7m 16s
✔️ integration-amazon.aws-1 SUCCESS in 7m 30s
Skipped 43 jobs

@mandar242 mandar242 force-pushed the refactor_s3_bucket_module branch from d146ce9 to 002ea20 Compare April 15, 2024 21:24
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/1019177791204b4c999388b7d012fbf3

✔️ ansible-galaxy-importer SUCCESS in 5m 11s
✔️ build-ansible-collection SUCCESS in 14m 50s
✔️ ansible-test-splitter SUCCESS in 6m 30s
✔️ integration-amazon.aws-1 SUCCESS in 6m 02s
Skipped 43 jobs

Copy link
Contributor

@tremble tremble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looking good, I'd actually started looking into refactoring s3_object with an eye towards using @AWSErrorHandler (see also #1998). Which would probably be a nice follow up to this one.

The use of hinting seems inconsistent, and some of the return documentation looks dubious (eg. get_bucket_versioning returning None).

plugins/modules/s3_bucket.py Outdated Show resolved Hide resolved
plugins/modules/s3_bucket.py Outdated Show resolved Hide resolved
plugins/modules/s3_bucket.py Outdated Show resolved Hide resolved
plugins/modules/s3_bucket.py Outdated Show resolved Hide resolved
@mandar242 mandar242 force-pushed the refactor_s3_bucket_module branch from 1da1311 to b9320b5 Compare April 17, 2024 03:08
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/f47b8e79e997406db852d90660df3404

✔️ ansible-galaxy-importer SUCCESS in 5m 29s
✔️ build-ansible-collection SUCCESS in 14m 55s
✔️ ansible-test-splitter SUCCESS in 6m 09s
✔️ integration-amazon.aws-1 SUCCESS in 7m 51s
Skipped 43 jobs

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/51290ea220e64461a144d64a04262e00

✔️ ansible-galaxy-importer SUCCESS in 5m 18s
✔️ build-ansible-collection SUCCESS in 15m 29s
✔️ ansible-test-splitter SUCCESS in 6m 14s
✔️ integration-amazon.aws-1 SUCCESS in 7m 35s
Skipped 43 jobs

@@ -352,6 +352,7 @@

import json
import time
from typing import Iterator, List, Tuple
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from typing import Iterator, List, Tuple
from typing import Iterator
from typing import List
from typing import Tuple

Copy link
Contributor

@abikouo abikouo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall

requester_pays = module.params.get("requester_pays")
tags = module.params.get("tags")
purge_tags = module.params.get("purge_tags")
def handle_bucket_versioning(s3_client, module: AnsibleAWSModule, name: str) -> tuple[bool, dict]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not set the attribute type boto3.client for the s3_client as described in the function documentation?

Copy link
Contributor Author

@mandar242 mandar242 Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when added s3_client: boto3.client its throwing sanity failures and integration test failures (import)
ERROR: Found 47 pylint issue(s) which need to be resolved:
ERROR: plugins/modules/s3_bucket.py:378:40: undefined-variable: Undefined variable 'boto3'
ERROR: plugins/modules/s3_bucket.py:432:44: undefined-variable: Undefined variable 'boto3'
ERROR: plugins/modules/s3_bucket.py:476:50: undefined-variable: Undefined variable 'boto3'

one thing we can do is add import boto3 but I am not sure if doing that would cause boto version issues


# Requester pays

def handle_bucket_requester_pays(s3_client, module: AnsibleAWSModule, name: str) -> tuple[bool, dict]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same remark here for the s3_client parameter

@mandar242 mandar242 force-pushed the refactor_s3_bucket_module branch from 391c25d to 692ee36 Compare April 19, 2024 21:09
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/e0952fc92d0a44dbb7dd676a563c5b9b

✔️ ansible-galaxy-importer SUCCESS in 4m 06s
✔️ build-ansible-collection SUCCESS in 14m 57s
✔️ ansible-test-splitter SUCCESS in 5m 30s
✔️ integration-amazon.aws-1 SUCCESS in 5m 58s
Skipped 43 jobs

@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label Apr 20, 2024
Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/8d300c7093fb409e988e644cbb35a232

✔️ ansible-galaxy-importer SUCCESS in 5m 11s
✔️ build-ansible-collection SUCCESS in 15m 46s
✔️ ansible-test-splitter SUCCESS in 6m 24s
✔️ integration-amazon.aws-1 SUCCESS in 7m 53s
Skipped 43 jobs

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit 99e13ae into ansible-collections:main Apr 20, 2024
40 checks passed
@mandar242 mandar242 deleted the refactor_s3_bucket_module branch April 26, 2024 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergeit Merge the PR (SoftwareFactory)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants