Cleaner API

[parkan]

Right now extracting the relevant data object from a query response is kind of ugly:

remote.queryWithData('SELECT * FROM images.dpla LIMIT 1').then(r => r[0].simple.stmt.body.simple.object.data)

or

א > remote.query('SELECT * FROM images.dpla LIMIT 1').then(r => remote.data(r.map(_ => _.value.simple.stmt.body.simple.object)))
[ { key: 'QmeFJSTPKSEiNqebxZvYcduWH8UBmxqNq724gHEQnxV5D1',
    data: <Buffer a2 66 73 63 68 65 6d 61 a1 61 2f 78 2e 51 6d 59 47 52 51 59 6d 57 43 33 42 41 74 54 41 69 38 38 6d 46 62 37 47 56 65 46 73 55 4b 47 4d 34 6e 6d 32 35 ... > } ]
// decode...

This would only get more problematic with other statement types.

From a user perspective, the contract is more like "I'm asking for ID(s) in [a] namespace(s), give me a merged set of of what that namespace knows about it, respecting my trust settings". Given that #17 isn't merged yet (and lots of thinking about merges hasn't happened yet), we can just make the merge function be naive set add.

So this API looks more like

א > remote.magicQuery('SELECT * FROM images.dpla LIMIT 1')

[ { schema: { '/': 'QmYGRQYmWC3BAtTAi88mFb7GVeFsUKGM4nm25SBUB9vfc9' }, // ??
    data:
     { artist_names: [Object], // FIXME: expand this also?
       aspect_ratio: 0.7166666666666667,
       attribution: [Object],
       camera_exif: {},
       date_captured: null,
       date_created_at_source: null,
       date_created_original: null,
       date_source_version: null,
       dedupe_hsh: '3fdfefe0f8381403',
       derived_qualities: [Object],
       description: 'Clausen/Donnelly House',
       keywords: [],
       licenses: [Object],
       location: [Object],
       native_id: 'dpla_http://dp.la/api/items/1ff6b36174426026847c8f8ca216ffa9',
       orientation: null,
       providers_list: [Object],
       sizes: [Object],
       source: [Object],
       source_dataset: 'dpla',
       source_tags: [Object],
       title: [Object],
       transient_info: [Object],
       url_direct: [Object],
       url_shown_at: [Object] } } ]

Questions:

• Should we include schema or just object itself? What if all schemas in a result set are the same/different?
• What about packing this into a "results" object that has "data_rows" but also some metadata fields?
• What about refs? Are we sufficiently "application level" to find and (recursively?) reattach all ref leaves?

[parkan]

@yusefnapora @vyzo thoughts welcome here

[yusefnapora]

So the idea is that you issue the query for a given WKI in a namespace, and concat returns everything it knows. Then we filter by publishers that you trust, and give you the set of objects attached to those statements?

Is it important to preserve who published which objects and when for this API?

I'm not sure about the refs... unless you mean fetching the objects in the deps field by hash and "inflating" them in the result? Like the schema reference in the example, which should be in deps.

[parkan]

So the idea is that you issue the query for a given WKI in a namespace, and concat returns everything it knows. Then we filter by publishers that you trust, and give you the set of objects attached to those statements?

yeah

Is it important to preserve who published which objects and when for this API?

good question! I don't really know

I'm not sure about the refs... unless you mean fetching the objects in the deps field by hash and "inflating" them in the result? Like the schema reference in the example, which should be in deps.

yeah, like assembling the full graph if it has artist refs or w/e -- this is probably not a v1 feature, but may inform the underlying query a bit

[parkan]

Ok, are we kicking this out of this sprint? The oracle we're shipping will be relatively simple (permissionless write only) so won't need this change

[yusefnapora]

I'd be into that... it would be nice to think it through a bit more. I haven't come up with something I love yet.

[parkan]

Related to #177