-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AVRO-3649: Fix for union type to match default values for any innertype #1922
Conversation
* @param jsonValue a value to check against the schema | ||
* @return true if the value is valid according to this schema | ||
*/ | ||
public boolean isValidDefault(JsonNode jsonValue) { |
Check notice
Code scanning / CodeQL
Missing Override annotation Note
Schema.isValidDefault
It does. This would have to be documented and implemented for other languages too. If there is a union |
For bytes, even this code doesn't work Schema schemaBytes = Schema.create(Type.BYTES);
BinaryNode validBinary = new BinaryNode("Hello".getBytes(StandardCharsets.UTF_8));
assertTrue(schemaBytes.isValidValue(validBinary)); It it may be another issue. For docs, indeed, text is now reviewed For other languages, i will made other PR (otherwise, this one will be too big) |
BinaryNode is documented as "Base64 encoded binary value", but Avro does not use Base64 in field default values or in JSON encoding, so I don't think Schema.isValidValue needs to support the BinaryNode type. |
for other languages too |
boolean acceptNull = fieldSchema.getTypes().stream().map(Schema::getType).anyMatch((t) -> t == Schema.Type.NULL); | ||
if (acceptNull) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One can also call fieldSchema.isNullable()
here
case UNION: // union default: first branch | ||
return isValidDefault(schema.getTypes().get(0), defaultValue); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks to me that these changes make any value for the null schema or a union an invalid value. What am I overlooking?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
static method isValidDefault (line 1650 on new version) is private, called by new virtual function "public boolean isValidValue(final Object value) {", line 1645.
This function is redefine for NULL Schema (line 1387) and for union schema (line 1248).
May be the main flaw of this PR is to keep the static isValidDefault method and not build function isValidValue for each schema type, but it also would duplicate code for STRING, BYTES, ENUM for example (See here)).
34a76ea
to
343e343
Compare
343e343
to
0d5d9b9
Compare
0d5d9b9
to
7af5cc8
Compare
What is the purpose of the change
Change the logic on Union type to validate a value (default value in particular) for AVRO-3649. Instead of check value with first union inner type, try with inner type until find a match.
Verifying this change
Some test were updated.
Documentation