Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core 3.70 compatibility #3876

Merged
merged 2 commits into from
Feb 11, 2025
Merged

Conversation

pedro-psb
Copy link
Member

@pedro-psb pedro-psb commented Feb 4, 2025

#3865 declares compatibility with pulpcore 3.70, but the PR is getting a lot of errors not related to the changes.

For example, I can see a lot of binding validation errors which were somehow expected because of our bump of binding generator image version (see here).

I'm creating this PR so other errors related to the compatibility can be addressed independently.

@github-actions github-actions bot added multi-commit Added when a PR consists of more than one commit no-changelog no-issue labels Feb 4, 2025
@pedro-psb pedro-psb force-pushed the core-3.70-compatibility branch from 94104a1 to f0bf4ba Compare February 5, 2025 20:54
@@ -88,7 +88,7 @@ class ModulemdDefaultsSerializer(NoArtifactContentSerializer):

module = serializers.CharField(help_text=_("Modulemd name."))
stream = serializers.CharField(help_text=_("Modulemd default stream."))
profiles = CustomJSONField(help_text=_("Default profiles for modulemd streams."))
profiles = serializers.JSONField(help_text=_("Default profiles for modulemd streams."))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the distinction between serializers.JSONField, JSONDictField, and JSONListField ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. serializers.JSONField (drf) accept any json type (bool, str, object, array, ...) and has openapi type any.
  2. JSONDictField (custom) validates it's a json object (aka dict) and has openapi type object.
  3. JSONListField (custom) validates it's a json array (aka list) and has openapi type array.

2 and 3 subclasses from 1 and then adds the layer of validation.

But actually I found that my implementation of 2 and 3 is wrong.
I should either:

a) add the correct version of these fields here to get the types and validation right
b) use the serializers.JSONField which will change the types we use in openapi to any (which will probably not break ruby bindings this time because the new bindindg generator image probably supports any type)

Copy link
Contributor

@dralley dralley Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

B) is what the python plugin just did I believe

pulp/pulp_python#791

Copy link
Member Author

@pedro-psb pedro-psb Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I'll do that here as there is already a lot going on

Comment on lines 165 to 187
@pytest.fixture
def pulp_requests(bindings_cfg):
"""Uses requests lib to issue an http request to pulp server using pulp_href.

Example:
>>> response = pulp_requests("get", "/pulp/api/v3/.../?repository_version=...")
>>> type(response)
requests.Response
"""
ALLOWED_METHODS = ("get", "update", "delete", "post")
auth = (bindings_cfg.username, bindings_cfg.password)
host = bindings_cfg.host

def _pulp_requests(method: str, pulp_href: str, body=None):
if method not in ALLOWED_METHODS:
raise ValueError(f"Method should be in: {ALLOWED_METHODS}")
url = urljoin(host, pulp_href)
request_fn = getattr(requests, method)
return request_fn(url, auth=auth)

return _pulp_requests


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I have incidentally created a similar thing here:
https://github.com/pulp/pulp_ansible/blob/84a32718cbe18216b4067a50eea222d713c818ed/pulp_ansible/tests/functional/api/collection/v3/test_collection.py#L29

In any case, you can use requests.request(method, *) instead of the getattr dance.

Copy link
Member Author

@pedro-psb pedro-psb Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I guess I'll addopt the session approach.

@pedro-psb
Copy link
Member Author

@pedro-psb pedro-psb marked this pull request as ready for review February 11, 2025 18:40
required=False,
help_text=_("A JSON document describing config.repo file"),
)

def to_representation(self, instance):
data = super().to_representation(instance)
# Import workflow may cause these fields to be stored as "" in the database
Copy link
Contributor

@dralley dralley Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a separate codepath that ought to be adjusted here? Anything to fix with a data migration? Ideally we want total consistency w/ how values are stored in a field.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that we can fix import-export workflow to ensure it stores null values and add migration to fix empty strings. I can open an issue for that.

Copy link
Contributor

@dralley dralley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of this looks reasonable and most of it looks like a cleanup over what came before. Nice job!

Please squash before merging

@pedro-psb pedro-psb force-pushed the core-3.70-compatibility branch from 6621b87 to 1e9e9ec Compare February 11, 2025 21:30
This does:
* Bumps lower and upper bounds for pulpcore: >=3.70, <3.85
* Removes pulp_smash and replaces it with similar fixtures
* Adapt to the new bindings changes caused by openapi-generator bump
* Revert the usage of CustomJSONField (to serializer.JSONField) due to
  bindings changes.
@pedro-psb pedro-psb force-pushed the core-3.70-compatibility branch from 1e9e9ec to 08a5100 Compare February 11, 2025 21:31
@pedro-psb pedro-psb enabled auto-merge (rebase) February 11, 2025 21:33
@pedro-psb pedro-psb merged commit e9d4b24 into pulp:main Feb 11, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multi-commit Added when a PR consists of more than one commit no-changelog no-issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants