JSON Schema
new proposed output format #195
Replies: 6 comments 27 replies
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
-
|
Can you post the schema/instance that generated that output? I'll run it through my implementation, which now has a branch for the proposal at #63. I'd like to see what that shows for comparison. |
Beta Was this translation helpful? Give feedback.
-
|
Here is the document. |
Beta Was this translation helpful? Give feedback.
-
|
Okay... after a bit of fixing some things, I landed on this, which is basically covered by the first three lines of your condensed output: {
"valid": false,
"evaluationPath": "",
"schemaLocation": "https://json-schema.org/draft/2020-12/schema#",
"instanceLocation": "",
"nested": [
{
"valid": false,
"evaluationPath": "/allOf/1/$ref/properties/properties/additionalProperties/$dynamicRef",
"schemaLocation": "https://json-schema.org/draft/2020-12/schema#",
"instanceLocation": "/properties/allOf",
"errors": {
"type": "Value is \"array\" but should be \"object, boolean\""
}
},
{
"valid": false,
"evaluationPath": "/allOf/1/$ref/properties/properties/additionalProperties/$dynamicRef/allOf/0/$ref",
"schemaLocation": "https://json-schema.org/draft/2020-12/meta/core#",
"instanceLocation": "/properties/allOf",
"errors": {
"type": "Value is \"array\" but should be \"object, boolean\""
}
},
{
"valid": false,
"evaluationPath": "/allOf/1/$ref/properties/properties/additionalProperties/$dynamicRef/allOf/2/$ref",
"schemaLocation": "https://json-schema.org/draft/2020-12/meta/validation#",
"instanceLocation": "/properties/allOf",
"errors": {
"type": "Value is \"array\" but should be \"object, boolean\""
}
},
{
"valid": false,
"evaluationPath": "/allOf/1/$ref/properties/properties/additionalProperties/$dynamicRef/allOf/3/$ref",
"schemaLocation": "https://json-schema.org/draft/2020-12/meta/meta-data#",
"instanceLocation": "/properties/allOf",
"errors": {
"type": "Value is \"array\" but should be \"object, boolean\""
}
},
{
"valid": false,
"evaluationPath": "/allOf/1/$ref/properties/properties/additionalProperties/$dynamicRef/allOf/4/$ref",
"schemaLocation": "https://json-schema.org/draft/2020-12/meta/format-annotation#",
"instanceLocation": "/properties/allOf",
"errors": {
"type": "Value is \"array\" but should be \"object, boolean\""
}
},
{
"valid": false,
"evaluationPath": "/allOf/1/$ref/properties/properties/additionalProperties/$dynamicRef/allOf/5/$ref",
"schemaLocation": "https://json-schema.org/draft/2020-12/meta/content#",
"instanceLocation": "/properties/allOf",
"errors": {
"type": "Value is \"array\" but should be \"object, boolean\""
}
}
]
}It is a bit scroll-y, yeah. I'm not sure how to condense it more, though. It's pretty trivial to take the above and extract the bits you need. The problem I see here, though is that these errors (although in this case, they're all the same error) were all produced by different locations. Lastly, it looks like you're doing some extra processing to collect and combine similar messages to make it more human-consumable. While I agree that it's probably beneficial for your users, I don't know that prescribing such extra processing is necessary for the spec. |
Beta Was this translation helpful? Give feedback.
-
|
What would annotations look like in this format? For those you need to know the keyword that generated the annotation. You can discern this with only data location and annotation value. {
"properties": {
"foo": {
"default": "this might be the description",
"description": "or this could be"
}
}
} |
Beta Was this translation helpful? Give feedback.
-
This phrasing raises a question for me: Are you thinking of output as the literal return value of the To me, "output" is a thing that can be accessed after evaluation. It could be a static data structure, but it could also be an object of some sort that you can query for any of the output formats using one or more methods. I'm not trying to argue for anything specific here, I just want to make sure we are not making a decision based on an assumed structure of programmatic interfaces. |
Beta Was this translation helpful? Give feedback.
-
Yes, to a degree. The function call returns some sort of results.
I don't understand the mechanics of this. What would you use to "access" the output if it's not (at least contained in) the return value?
This is precisely what I view the output to be: a data structure that can be queried. For example, using JSON Path, you can get all of the titles with |
Beta Was this translation helpful? Give feedback.
-
I meant an object that could be queried for various things including but not limited to the standard machine-readable output formats. For example, it could be queried for a human-friendly format instead, or some sort of tree-walking interface like Julian's (pre-standard-output) implementation offers. At one point you said:
However, this would be a totally 2020-12-specification-compliant interface: result = schema.evaluate(instance)
if result.valid:
return result.basic()
else:
raise Exception(result.verbose())While I'm open to other arguments for consistency of format, the argument that they need to be the same structure to some degree because they are all being returned from the same function is not an absolute constraint even in strongly-typed languages. |
Beta Was this translation helpful? Give feedback.
-
|
I don't see that block of code as being in scope of the spec. It's just choosing between two spec-defined output formats.
I'm arguing for a consistent model. That model is the output unit, and the various trees/lists that it can produce. |
Beta Was this translation helpful? Give feedback.
-
That's my point. The spec doesn't control it, therefore the whole "there's only one function return so everything has to look the same" is not a valid argument.
Sure, that's a reasonable position, I was just responding to your statement "But there's only one validation call which produces a result." |
Beta Was this translation helpful? Give feedback.
-
|
So that it's not buried in Slack, copy pasting my brief opinion (brief in the sense that I have not thought hard here so if I'm way off on the key topic being discussed feel free to say so):
|
Beta Was this translation helpful? Give feedback.
-
|
I think it might be worth adding (if not there already) some text that says the output is intended to be machine-consumable and that implementations which cater to humans are free to define output as they see fit for their users. Specifically interoperability is not a concern for these cases. |
Beta Was this translation helpful? Give feedback.
-
In a (rather heated and lengthy) discussion in Slack, I expressed that I would be okay with a format like this being in supplementary material as mere guidance for applications that consume JSON Schema, but it does not belong in the spec. The spec cannot be expected to prescribe what applications show to their users. The needs of an application's users are something that only the designers & developers of that application can fully understand. Historically, the spec has been firmly grounded in the validation of JSON data. This is why we have shied away from alternate uses like forms or code generation. The language in the spec is already validation-centric rather than application-centric. Applications using JSON Schema are related but separate. As such the spec can and should define an output format implementations can use in order to communicate the results of a validation to a host application. This interaction is solely between machines, namely the implementation, which is the representative of the specification, and the application that is consuming it. The specification cannot define output for:
The prescribed output need only apply to implementations of the specification which are made available for general use (i.e. for unknown consumers). These unknown consumers would need to be designed to accept the prescribed output and manipulate it to suit their consumers' (whether machine or human) needs, which is outside the purview of the spec. |
Beta Was this translation helpful? Give feedback.
-
|
Here's a use in the wild of an output format very similar to this: OAI/OpenAPI-Specification#2983 I am going ahead and adding this format to my own implementation, and will make more refinements going forward after it sees some use in the wild. I see no way of closing this discussion. |
Beta Was this translation helpful? Give feedback.
-
|
The 3rd path recommendations I mentioned would presumably be things easy to build on top of the machine-readable formats. |
Beta Was this translation helpful? Give feedback.
-
Not at all. What I'm saying is that the output, as it is, is intended for machine readability and interoperability. If there is a human-readable output that is specified separately from what's there, I'm for it. Again, this is a good reason to extract the output into it's own spec. Then we could have multiple output specs: one for people and one for machines. |
Beta Was this translation helpful? Give feedback.
-
|
Additionally, I think what's presented here can be extracted from what's in the spec. |
Beta Was this translation helpful? Give feedback.
-
Of course it could, was that in dispute at some point?
I'm pretty sure that's the proposal here. Can we stop treating this discussion as something that diminishes or changes the machine-readable output? It's a different solution for a different problem. Which is why I want to stop acting like there's only one "output" that we're all, I dunno, battling for control of somehow. |
Beta Was this translation helpful? Give feedback.
-
|
This discussion originated at the same time I was proposing to refine the existing output structure. At the time, we didn't have a concept of separation between human-readable and machine-readable output; it was just "output." My initial responses were taking this as a direct challenge to what I was proposing. Now that we have worked out that the distinction is there, I fully support deeming what's in the spec as "intended for machine consumption" and treating this not as an alternative, but as a separate thing entirely, intended for a human consumer. That said, I'm not sure that human readable output needs to be strictly specified. Loosely, perhaps. For instance, just specifying what information needs to be there or how grouping different results can happen. But largely, implementations are going to have different groups of users, and those users tend to have different needs that the implementation should absolutely cater to. But... as my implementation doesn't have human consumers, I'll leave this to those implementations which do. |
Beta Was this translation helpful? Give feedback.
-
I've been using this output format in my implementation for quite some time -- it only shows the data location of an error, not the keyword location, and is suitable for situations where the schema is not published and therefore keyword locations are not helpful, or as a quick way to show the source of all errors (for example when the schema itself has an error and cannot be validated).
Here's an example from one of my schemas where I had an
allOfindented at the wrong level in the openapi yaml file, and therefore loading it into an openapi tool throws an exception:Here is the same result in 'basic' format -- as you can see it is much more verbose, and more difficult for a poor user to figure out what went wrong:
The format is essentially
sprintf("at '%s': %s", instanceLocation, error), in the standard evaluation order, with any duplicate lines removed (sometimes several errors are issued at the same instance location (the first error is actually repeated 7 times in the original because it comes from the top level of every vocabulary -- "I expected to see a schema here, but instead got an array").I would like to formalize this into the spec as a format called something like 'compact' or 'abridged', as I find it quite useful in some applications - e.g. I use it in the exception message thrown when a schema is not considered valid by my implementation. Note that this is a string, not a json object (although it can be used as a json array simply enough, where each line is an array item).
Beta Was this translation helpful? Give feedback.
All reactions