Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fragments #72

Merged
merged 11 commits into from
Dec 4, 2020
16 changes: 16 additions & 0 deletions fragments/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,17 @@
# STAC API - Fragments

This directory contains reusable OpenAPI fragments that are used by other capabilities. They tend to
be parameters that get mixed in to various endpoints, and keeping them in this separate fragment directory
cholmes marked this conversation as resolved.
Show resolved Hide resolved

For example '[sort](sort/)' introduces a parameter (`sortby`) that can be used by both [item-search](../item-search)
at the `/search` endpoint, and by [features](../ogcapi-features) in any of its `items` endpoints. To keep
things clean we specify a conformance class for each, so clients can know exactly what to expect. Each
conformance class is specified in the relevant folder as an 'extension' to the main capability. But their
semantics are exactly the same, so we put the shared openapi definition in this `fragments` directory.

| Fragment Name | Description |
|----------------------------------------|----------------------------------------------------------------------------|
| [Fields](fields/README.md) | Adds parameter to control which fields are returned in the response. |
| [Query](query/README.md) | Adds parameter to compare properties and only return the items that match |
| [Context](context/README.md) | Adds search related metadata (context) to GeoJSON Responses. |
| [Sort](sort/README.md) | Adds Parameter to control sorting of returns results. |
2 changes: 1 addition & 1 deletion fragments/context/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
This extension is intended to augment the core ItemCollection
object when the ItemCollection is the result of a search, for example, from calling the `/search` API endpoint.

**Note**: *This extension is currently scoped to just the [STAC Item Search](../../item-search) functionality.
**Note**: *This extension is currently only used by [STAC Item Search](../../item-search) functionality.
OGC API has their own way returning `numberMatched` and `numberReturned` at the top level, instead of in a context
object. We are hoping to [align](https://github.com/opengeospatial/ogcapi-common/issues/82), but until then it
is recommended to use STAC Context in the cross-collection `search` endpoint, and follow the OGC API way when
Expand Down
17 changes: 4 additions & 13 deletions fragments/fields/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,15 @@
# STAC API - Fields Fragment
# STAC API - Fields Fragment
cholmes marked this conversation as resolved.
Show resolved Hide resolved

- **OpenAPI specification:** [openapi.yaml](openapi.yaml)
- **Fragment [Maturity Classification](../../extensions.md#extension-maturity):** Pilot
- **Dependents:**
- [Item Search](../../item-search)

m-mohr marked this conversation as resolved.
Show resolved Hide resolved
By default, the STAC search endpoint `/search` returns all attributes of each Item, as there is no way to specify
exactly those attributes that should be returned. The Fields API Extension adds new functionality that allows the
client to suggest to the server which Item attributes should be included or excluded in the response.

When calling `/search` using POST with`Content-Type: application/json`, this extension adds an attribute `fields` with
When used in a POST request with `Content-Type: application/json`, this adds an attribute `fields` with
an object value to the core JSON search request body. The `fields` object contains two attributes with string array
values, `include` and `exclude`.

When calling `/search` using GET or POST with `Content-Type: application/x-www-form-urlencoded` or
When used with GET or POST with `Content-Type: application/x-www-form-urlencoded` or
`Content-Type: multipart/form-data`, the semantics are the same, except the syntax is a single parameter `fields` with
a comma-separated list of attribute names, where `exclude` values are those prefixed by a `-` and `include` values are
those with no prefix, e.g., `-geometry`, or `id,-geometry,properties`.
Expand All @@ -29,7 +25,7 @@ polygons can be very large when reprojected to EPSG:4326, as in the case of a hi
Implementations are also not required to implement semantics for nested values whereby one can include an object, but
exclude attributes of that object, e.g., include `properties` but exclude `properties.datetime`.

No error should be returned if a specified field has no value for it in the catalog. For example, if the attribute
No error should be returned if a specified field has no value for it in the catalog. For example, if the attribute
"properties.eo:cloud_cover" is specified but there is no cloud cover value for an Item or the API does not even
support the EO Extension, a successful HTTP response should be returned and the Item entities will not contain that
attribute.
Expand All @@ -42,11 +38,6 @@ will not be a valid STAC Item.
Implementations may return attributes not specified, e.g., id, but should avoid anything other than a minimal entity
representation.

**NOTE**: *This extension is built to be compatible
[OAFeat](https://github.com/opengeospatial/ogcapi-features) endpoints, but does not yet have a conformance
class to use in that way. Our goal is to align with OGC API functionality, which looks to be prioritized currently as
[future work](https://github.com/opengeospatial/ogcapi-features/issues/451)*

## Include/Exclude Semantics

1. If `fields` attribute is specified with an empty object, or with both `include` and `exclude` set to null or an
Expand Down
7 changes: 3 additions & 4 deletions fragments/query/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,15 @@

- **OpenAPI specification:** [openapi.yaml](openapi.yaml)
- **Extension [Maturity Classification](../../extensions.md#extension-maturity):** Pilot
Likely to get deprecated in the future in favor of CQL.
Likely to get deprecated in the future in favor of [CQL](http://docs.opengeospatial.org/DRAFTS/19-079.html).
- **Dependents:**
- [Item Search](../../item-search)

The STAC search endpoint, `/search`, by default only accepts a limited set of core filter parameters.
The Query API extension adds additional filters for searching on the properties of Items.
The `query` parameter adds additional filters for searching on the properties of Items.

The syntax for the `query` filter is:

```js
```json
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend to revert this change as the example is invalid JSON and JS renders this a bit better (i.e. I chose JS specifically)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gotcha - didn't realize it was deliberate.

{
"query": {
"<property_name>": {
Expand Down
16 changes: 6 additions & 10 deletions fragments/sort/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,8 @@
- **Fragment [Maturity Classification](../../extensions.md#extension-maturity):** Pilot
- **Dependents:**
- [Item Search](../../item-search)

By default, the STAC search endpoint `/search` returns results in no specified order. Whatever order the results are in
is up to the implementor, and will typically default to an arbitrary order that is fastest for the underlying data store
to retrieve results.

The Sort API Extension adds a new parameter, `sortby`, that allows the user to define fields by which to sort results.
This defines a new parameter, `sortby`, that allows the user to define fields by which to sort results.
Only string, numeric, and datetime attributes of Item (`id` and `collection` only) or Item Properties (any attributes)
may be used to sort results. It is not required that implementations support sorting over all attributes, but
implementations should return an error when attempting to sort over a field that does not support sorting.
Expand All @@ -18,13 +14,13 @@ Fields may be sorted in ascending or descending order. The syntax between GET r
body vary. The `sortby` value is an array, so multiple sort fields can be defined which will be used to sort
the data in the order provided (e.g., first by `datetime`, then by `eo:cloud_cover`).

**NOTE**: *This extension is built to be compatible [OAFeat](../../ogcapi-features/README.md) endpoints,
but does not yet have a conformance class to use in that way.
Our goal is to align with OGC API functionality, which is currently being worked on as part of OGC API - Records.*
**NOTE**: *This fragment may change, as our goal is to align with OGC API functionality, and sorting is currently being
worked on as part of OGC API - Records, see [this issue](https://github.com/opengeospatial/ogcapi-records/issues/22)
for the latest discussion.*

## HTTP GET (or POST Form)

When calling `/search` using GET (or POST with `Content-Type: application/x-www-form-urlencoded` or
When calling a relevant endpoint using GET (or POST with `Content-Type: application/x-www-form-urlencoded` or
`Content-Type: multipart/form-data)`, a single parameter `sortby` with a comma-separated list of item field names should
be provided. The field names may be prefixed with either "+" for ascending, or "-" for descending. If no sign is
provided before the field name, it will be assumed to be "+".
Expand All @@ -41,7 +37,7 @@ Note that examples 1 and 2 are symantically equivalent, as well as examples 3 an

## HTTP POST JSON Entity

When calling `/search` using POST with`Content-Type: application/json`, this extension adds an attribute `sortby` with
When calling the relevant endpoint using POST with`Content-Type: application/json`, this adds an attribute `sortby` with
an object value to the core JSON search request body.

The syntax for the `sortby` attribute is:
Expand Down
197 changes: 62 additions & 135 deletions item-search/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,32 +28,6 @@ This link should look like:
}
```

## Extensions

### Context

- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#context>**
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Dependencies**: [STAC API - Context Fragment](../fragments/context/)

### Fields

- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#fields>
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Dependencies**: [STAC API - Fields Fragment](../fragments/fields/)

### Query

- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#query>
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Dependencies**: [STAC API - Query Fragment](../fragments/query/)

### Sort

- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#sort>
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Dependencies**: [STAC API - Sort Fragment](../fragments/sort/)

## Query Parameters and Fields

The following list of parameters is used to narrow search queries. They can all be represented as query string parameters
Expand All @@ -75,6 +49,8 @@ GET /search?collections=landsat8,sentinel&bbox=-10.415,36.066,3.779,44.213&limit
}
```

For more examples see [examples.md](examples.md).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also put the link into the "intro" list, here people may miss it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the 'intro' list?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list with maturity, conformance uri etc.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line 3-6 in the file


### Query Parameter Table

The core parameters for STAC search are defined by OAFeat, and STAC adds a few parameters for convenience.
Expand All @@ -89,7 +65,7 @@ The core parameters for STAC search are defined by OAFeat, and STAC adds a few p
| collections | \[string] | STAC | Array of one or more Collection IDs to include in the search for items. Only Items in one of the provided Collections will be searched |

Only one of either **intersects** or **bbox** should be specified. If both are specified, a 400 Bad Request response
should be returned. See [examples](#examples) to see sample requests.
should be returned. See [examples](examples.md) to see sample requests.

## Response

Expand Down Expand Up @@ -180,124 +156,75 @@ searching on specific Item properties.
The other HTTP verbs are not supported in STAC Item Search. The [Transaction Extension](../ogcapi-features/extensions/transaction/README.md)
does implement them, for STAC and OAFeat implementations that want to enable writing and deleting items.

#### Examples
## Extensions

##### Simple GET based search
Request:
```http
HTTP GET /search?bbox=-110,39.5,-105,40.5
```
These extensions provide additional functionality that enhances the core item search. All are specified as
[fragments](../fragments), as they are re-used by other extensions. The below conformance classes are used
to indicate that the `search` endpoint can make use of them.

Response with `200 OK`:
```json
{
"type": "FeatureCollection",
"features": [],
"links": [
{
"rel": "next",
"href": "http://api.cool-sat.com/search?page=2"
}
]
}
```
Following the link `http://api.cool-sat.com/search?page=2` will send the user to the next page of results.
### Fields

##### POST search with body and merge fields
Request to `HTTP POST /search`:
```json
{
"bbox": [-110, 39.5, -105, 40.5]
}
```
- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#fields>
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Definition**: [STAC API - Fields Fragment](../fragments/fields/)

Response with `200 OK`:
```json
{
"type": "FeatureCollection",
"features": [],
"links": [
{
"rel": "next",
"href": "http://api.cool-sat.com/search",
"method": "POST",
"body": {
"page": 2,
"limit": 10
},
"merge": true
}
]
}
```
By default, the STAC search endpoint `/search` returns all attributes of each Item, as there is no way to specify
exactly those attributes that should be returned. The Fields extension to Item Search adds new functionality that
allows the client to suggest to the server which Item attributes should be included or excluded in the response,
through the use of a `fields` parameter. The full description of how this extension works can be found in the
[fields fragment](../fragments/fields/).

This tells the client to POST to the search endpoint using the original request with the `page` and `limit` fields
merged in to obtain the next set of results:
STAC API's that support the fields functionality must include the conformance class
<http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#fields> in the `conformsTo` response at
the root (`/`) landing page, to indicate to clients that they will respond properly to requests that use
the `fields` parameter on the endpoint specified by the `search` rel.

Request to `POST /search`:
```json
{
"bbox": [-110, 39.5, -105, 40.5],
"page": 2,
"limit": 10
}
```
### Query

This can be even more effective when using continuation tokens on the server, as the entire request body need not be
repeated in the subsequent request:
- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#query>
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Definition**: [STAC API - Query Fragment](../fragments/query/)

Response with `200 OK`:
```json
{
"rel": "next",
"href": "http://api.cool-sat.com/search",
"method": "POST",
"body": {
"next": "a9f3kfbc98e29a0da23"
}
}
```
The above link tells the client not to merge (default of false) so it is only required to pass the next token in the body.
The STAC search endpoint, `/search`, by default only accepts a limited set of parameters to limit the results
by properties. The Query extension adds a new parameter, `query`, that can take a number of comparison operators to
match predicates between the fields requested and the values of Items. It can be used with both GET and POST, though
GET includes the exact same JSON. The full details on the JSON structure are specified in the [query
fragment](../fragments/query/).

Request to `POST /search`:
```json
{
"next": "a9f3kfbc98e29a0da23"
}
```
STAC API's that support the query functionality must include the conformance class
<http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#query> in the `conformsTo` response at
the root (`/`) landing page, to indicate to clients that they will respond properly to requests that use
the `query` parameter on the endpoint linked to by the `search` rel.

##### POST search using headers
Request to `HTTP POST /search`:
```json
{
"bbox": [-110, 39.5, -105, 40.5],
"page": 2,
"limit": 10
}
```
### Sort

Response with `200 OK`:
```json
{
"type": "FeatureCollection",
"features": [],
"links": [
{
"rel": "next",
"href": "http://api.cool-sat.com/search",
"method": "POST",
"headers": {
"Search-After": "LC81530752019135LGN00"
}
}
]
}
```
- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#sort>
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Definition**: [STAC API - Sort Fragment](../fragments/sort/)

This tells the client to POST to the search endpoint with the header `Search-After` to obtain the next set of results:
By default, the STAC search endpoint `/search` returns results in no specified order. Whatever order the results are in
is up to the implementor, and will typically default to an arbitrary order that is fastest for the underlying data store
to retrieve results. This extension adds a new parameter, `sortby`, that lets a user specify a comma separated list of
field names to sort by, with an indication of direction. It can be used with both GET and POST, the former using '+' and
'-' to indicate sort order, and the latter including a 'direction' field in JSON. The full description of the semantics
of this extension can be found in the [sort fragment](../fragments/sort).

Request:
```http
POST /search
Search-After: LC81530752019135LGN00
```
STAC API's that support the query functionality must include the conformance class
Copy link
Collaborator

@m-mohr m-mohr Dec 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conformsTo details are repeated many times. Maybe there a more clever way without writing so much? The shorter the docs, the more likely people read it. Is it enough to specify Conformance URI at the top and have somewhere centrally defined that you need conformance classes? At a place you can't miss it. A bold note in the core or so...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's fair. I think for these I was going more towards the 'requirements' style, to make clear exactly what you have to do. But my plan is to do actual requirements in a next release, in their own document, and to keep this one as the narrative. So I think it's ok to just remove some of the detail/repetition.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to document conformsTo on the landing page anyway, we have not done that yet: #27 (comment)

<http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#sort> in the `conformsTo` response at
the root (`/`) landing page, to indicate to clients that they will respond properly to requests that use
the `sortby` parameter on the endpoint specified by the `search` rel.

### Context

- **Conformance URI:** <http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#context>**
- **Extension [Maturity Classification](../extensions.md#extension-maturity):** Pilot
- **Definition**: [STAC API - Context Fragment](../fragments/context/)

This extension is intended to augment the core ItemCollection responses from the `search` API endpoint with a
JSON object called `context` that includes the number of items `matched`, `returned` and the `limit` requested.
The full description and examples of this are found in the [context fragment](../fragments/context)

STAC API's that support the context functionality must include the conformance class
<http://stacspec.org/spec/api/1.0.0-beta.1/extensions/item-search#context> in the `conformsTo` response at
the root (`/`) landing page, to indicate to clients that they will to respond with a `context` JSON object in all
responses from the `search` endpoint.
Loading