Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for http:// URLs when using $ref #87

Closed
joelittlejohn opened this issue Jun 23, 2013 · 23 comments
Closed

Add support for http:// URLs when using $ref #87

joelittlejohn opened this issue Jun 23, 2013 · 23 comments

Comments

@joelittlejohn
Copy link
Owner

Original author: [email protected] (February 01, 2013 22:51:08)

I have a json-schema valid schema (validated using JSV) and I setup my maven project to generate POJO out of it.

I get the following error:
Execution default of goal com.googlecode.jsonschema2pojo:jsonschema2pojo-maven-plugin:0.3.4:generate failed: URI scheme is not "file" (com.googlecode.jsonschema2pojo:jsonschema2pojo-maven-plugin:0.3.4:generate:default:generate-sources)

Using 0.3.4 version.

My schema ref$ to another schema, which is in same src/main/resources/schema directory. I refer to it e.g. "$ref": "./myotherschema.json#"

What am I doing wrong?

Original issue: http://code.google.com/p/jsonschema2pojo/issues/detail?id=87

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 01, 2013 23:04:25
Philippe, I can't seem to replicate this problem. Would you be able to attach an example schema? Feel free to cut it down to just the single $ref that causes the problem.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 02, 2013 13:34:44
Attached are the json-schemas. "root" links to "version" and "common-types", and "version" links to "common-types" as well.

I stored these schemas to the src/main/resources/schema.

Error is regarding URI scheme, it seems to expect "file". Since the URI is relative (and valid as long the file is in same path than referrer) I would not expect this error.

I tried to replace the relative path with absolute URI (to my server) but tool then failed with an error indicating the URI scheme as an authority component... which I do not understand either (tried http:// and https://).

Any cue?

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 02, 2013 13:59:02
Below the stacktrace, when running from command line:

Exception in thread "main" java.lang.IllegalArgumentException: URI scheme is not "file"
at java.io.File.(Unknown Source)
at com.googlecode.jsonschema2pojo.SchemaStore.create(SchemaStore.java:53)
at com.googlecode.jsonschema2pojo.SchemaStore.create(SchemaStore.java:96)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:59)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:83)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:38)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:98)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:73)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:39)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:60)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:31)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:106)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:55)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:95)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:66)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:83)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:38)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:98)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:73)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:39)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:60)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:31)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:106)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:55)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:95)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:66)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.SchemaMapper.generate(SchemaMapper.java:83)
at com.googlecode.jsonschema2pojo.cli.Jsonschema2Pojo.generate(Jsonschema2Pojo.java:96)
at com.googlecode.jsonschema2pojo.cli.Jsonschema2Pojo.main(Jsonschema2Pojo.java:64)

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 02, 2013 16:22:09
Thanks for the extra information. What value are you using for the source argument when invoking the tool?

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 02, 2013 16:25:14
I passed in the path where all these schema files were located.

Envoy� de mon iPhone

Le 2013-02-02 � 11:22 AM, "[email protected]"
[email protected] a �crit :

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 02, 2013 17:17:37
What would be useful is to include the $ref value that fails to load as a File (URI value).

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 03, 2013 16:01:00
For your info, the schemas are valid using the online validator http://json-schema-validator.herokuapp.com/

Note that I have to enable the 2 options: "Use draft v3" and "Use id for addressing". I believe this last one might be the cause of the issue in your tool.

Attached is a sample json instance that validates successfully against the schema.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 03, 2013 20:12:55
Philippe, I've tried generating pojos based on your example schemas. The problem is not related to your $ref links between files like "./myotherschema.json#", it's related to these:

"$ref": "http://json-schema.org/draft-03/links#"

At present this tool only supports file links so the tool complains that 'URI schema is not "file"' because it has found 'http://' and it can't deal with this ref. It would be nice if the error message included the text of the problematic URI so I'll fix this.

You need to download the links schema and use a local reference. Be warned though that the links schema also references both the hyper-schema and core schema. You're going to end up generating Java beans for the whole of JSON Schema, which is probably not a good thing to attempt since many of structural definitions used in the core and hyper schema do not map well to Java.

To solve this, I'd personally download the links schema and remove the "schema" and "targetSchema" properties from it.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 03, 2013 22:51:09
What is the technical difficulties to support http:// URIs instead of plain file:// URIs?

This sounds to me more of an issue, since usually, the schemas the be implemented cannot be modified (provided by the API provider which the implementation wishes to implement).

I see the workaround but would love to this this fixed in a future release. Why would not the structural definitions of the official meta schemas get properly generated? Any specific keywords you do not handle right for now?

At least I get the Java files generated with the workaround provided, but I really would like to avoid editing the schemas, which validates fine with JSV.

Philippe

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 03, 2013 23:22:13
There's no technical difficulty in resolving http://, it's simply not implemented. At present a Java File object is used to read the schema, and this can't be backed by an http:// URI. I think supporting http URIs would be really useful and I agree we should look to include this.

Regarding the meta schemas, the difficulties I'm referring to are not really related to keyword support. The biggest problem with mapping the full, official schemas in a meaningful way is that they make extensive use of union types. There are many instances in where these simply can't create a Java type hierarchy that is easily mappable by a JSON binding library like Jackson. The solution here is typically adding library-specific type hints to the JSON being mapped, which something that I think should be avoided.

Imagine a simply schema that defines a property that can be either integer or string. How would you elegantly represent this property in Java? (use Object type?). If you need to represent such dynamic data, what benefit are you getting in mapping it to a statically typed API of Java beans? The core schema and hyper-schema are good examples of documents that have been designed with the extensibility and dynamic nature of JSON in mind, by definition they are not considerate of the limitations of Java and static typing.

In practice, generating Java types and using data-binding is not a one-size-fits-all solution. If you have very dynamic data (e.g. the structure can vary wildly in ways that repel attempts to define rigid interfaces), sometimes data-binding is the wrong way to go.

Putting that aside for now, I understand that it's sometimes not desirable to change a provided schema. Thinking for a moment about the users of your generated types though, is it really desirable to create Link beans whose interface depends on a large hyper-schema type, which in turn depends on a much larger set of types representing the core schema? These seem largely irrelevant to your Java-based representation of Links.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 05, 2013 01:40:54
Thank you for the insightful comments.

I agree with you that referring to complete link description types
from metadata is overkill.

Do you have ETA about supporting the

  • http:// $ref
  • resource:// $ref
    ?

Any chance to see this coming -even experimental- in 0.3.5?

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 05, 2013 16:03:02
Yes, I'm happy to get these into 0.3.5. If I can find some time then I'll try to get the release out this week.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 05, 2013 17:13:20
Will it include both http:// and resource:// ??

Thanks a bunch!

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 05, 2013 19:40:53
Yes, hopefully it'll be simple to add both. See comments here:

http://code.google.com/p/jsonschema2pojo/issues/detail?id=85

And the Milestone labels here:

http://code.google.com/p/jsonschema2pojo/issues/list

Cheers

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 09, 2013 18:22:04
Philippe, this and issue # 85 are implemented in master now. Feel free to build a snapshot and try it out. It would be good if you could let me know that things are working with your examples before I cut the 0.3.5 release.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 13, 2013 14:47:15
Hi Joel.
I did test the feature using version 0.3.6-SNAPSHOT I built on my own using eclipse 3.7 and maven 3.0.
However I did uncover two unrelated issues.
If you have a space in the jason schema filename, jacson-core-2.1.3.jar don’t decode the Uri properly and throw an exception from JsonFactory.java:1404 ie:
... 20 more
Caused by: java.io.FileNotFoundException: D:\dev\jsonschemaPOC\jsonschema-versions\src\main\resources\schema\version%20-%20Copie.json (Le fichier spécifié est introuvable)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:120)
at java.io.FileInputStream.(FileInputStream.java:79)
at com.fasterxml.jackson.core.JsonFactory._optimizedStreamFromURL(JsonFactory.java:1404)
at com.fasterxml.jackson.core.JsonFactory.createJsonParser(JsonFactory.java:797)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:1701)
at com.googlecode.jsonschema2pojo.ContentResolver.resolve(ContentResolver.java:63)

Looking at the code of jackson, it looks like a bug since the code is doing the exact same thing if there is a space (%20).

Second, My eclipse had problems parsing the poms in jacsonschema2pojo because the plugins section is not enclosed by a pluginManagement section. It was throwing a cryptic error :
"maven-dependency-plugin (goals "copy-dependencies", "unpack") is not supported by m2e. pom.xml "

Appart from theses 2 issues, The new feature looks to be working in the maven plugin.
Do you think its also supported in the ant task ?

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 13, 2013 21:05:14
Hi LuoisRene. Just in case you're not aware, these changes are now released in 0.3.5 so you can use them there.

Regarding the space in the name, could you give me a demonstration of exactly what input you are using (example pom, example schemas). We may simply have to live with this as a known limitation until it's fixed upstream in Jackson, but I'll take a look and see if there's anything I can do.

Regarding the m2e error:

"maven-dependency-plugin (goals "copy-dependencies", "unpack") is not supported by m2e."

this garbage produced by m2e 1.0 and later. I'm loathe to introduce IDE-specific extras into the poms but I believe you can fix this by adding an Eclipse-specific 'lifecycle-mapping' plugin configuration into the pom. Google the error and you will find a load of links and explanation. Try the top accepted answer here for a solution:

http://stackoverflow.com/questions/8706017/maven-dependency-plugin-goals-copy-dependencies-unpack-is-not-supported-b

(I don't intend to add this m2e cruft into the project though and I still hold out hope that Sonatype will address this themselves).

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 13, 2013 22:24:26
Louis-Ren�, re: eclipse issue, please check in the project I shared
yesterday with you and confirm if the workaround I used was the same
as the one Joe is proposing above (it worked at least with 0.3.4
within Eclipse).

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 13, 2013 23:29:34
@joe: what is the usual way to download snapshot of 0.3.5 (we're not
much used to Git). I would prefer Louis-Ren� (who works in my team)
actually test the 0.3.5 you would ship. By the way, any expected ETA
when you would release that as an official build?

Thanks!

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 13, 2013 23:36:58
Philippe, this is already released as an official build - 0.3.5 :)

Check the downloads area or update your plugin version if using maven.

@joelittlejohn
Copy link
Owner Author

From [email protected] on February 15, 2013 21:10:34
Joe: to reproduce the bug, simply create a json schema file and name it
with a space in it e.g. "My Test.json". Then run jsonschema2pojo on it.
This will generate the error (in 0.3.6-snapshot).

@joelittlejohn
Copy link
Owner Author

From [email protected] on June 14, 2013 07:28:27
Not working on linux with a space in the path...

Failed to execute goal com.googlecode.jsonschema2pojo:jsonschema2pojo-maven-plugin:0.3.6:generate (default) on project json-tool: Execution default of goal com.googlecode.jsonschema2pojo:jsonschema2pojo-maven-plugin:0.3.6:generate failed: Unrecognised URI, can't resolve this: file:/files/06%20Workspaces/ws_android/json-tool/src/main/resources/schema/ticket.schema

Correct file name on linux is: /files/06\ Workspaces/ws_android/json-tool/src/main/resources/schema/ticket.schema

@joelittlejohn
Copy link
Owner Author

From [email protected] on June 19, 2013 22:04:34
lorenz, see issue #94 for more details on this. It is caused by a bug in Jackson that is now fixed, jsonschema2pojo 0.3.7 will use the fixed Jackson library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant