Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add data loader and new OpenAPI 3 loader #2484

Merged
merged 4 commits into from
Jun 27, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assets/images/modules/data_loader/intro.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
75 changes: 75 additions & 0 deletions docs/en/latest/modules/data_loader.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: Data Loader
keywords:
- APISIX
- APISIX Dashboard
- Data Loader
- OpenAPI
description: This document contains information about the Apache APISIX Dashboard data loader framework.
---

<!--
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
-->

## What is Data Loader

The data loader is an abstraction of the data import and export functionality under different specification definitions.

Based on it, developers can easily implement the ability to export APISIX data entities to generate different data files, and parse the data files and convert them into APISIX data entities for import into APISIX.

## Definition

```go
type Loader interface {
// Import accepts data and converts it into entity data sets
Import(input interface{}) (*DataSets, error)

// Export accepts entity data sets and converts it into a specific format
Export(data DataSets) (interface{}, error)
}
```

Implement the above functions to complete the data conversion between Raw data format and DataSets data set, APISIX Dashboard will use DataSets data format as the intermediate format for importing and exporting according to it.

Developers can look at the code in [api/internal/handler/data_loader/loader/loader.go](https://github.com/apache/apisix-dashboard/blob/master/api/internal/handler/data_loader/loader/loader.go) for the definition of the DataSets structure and the Data Loader interface.

## Supported data loader

- [OpenAPI 3](data_loader/openapi3.md): Currently only data import is supported

## How to support other data loader

Extending the data loader is simple and requires some development in the backend and frontend.

### Implement data loader conversion logic (back-end)

Create a structure that implements the above interface, which contains two parts.

- Import function: complete the parsing and conversion from raw format `[]byte` uploaded by the user to DataSets structure [api/internal/handler/data_loader/loader/openapi3/import.go](https://github.com/apache/apisix-dashboard/blob/master/api/internal/handler/data_loader/loader/openapi3/import.go)
- Export function: complete the generation of raw format for DataSets structure data inputted from APISIX Dashboard

Adds a new item to the data loader list of the import and export handler.

### Add front UI support for new data loader (front-end)

Adds the previously created data loader to the frontend selector. Refer to [this](https://github.com/apache/apisix-dashboard/blob/master/web/src/pages/Route/components/DataLoader/Import.tsx#L167-L172) for more details.

:::note
When you implement a data loader that requires partial input of custom parameters, you can create a form for it to enter data. Refer to [this](https://github.com/apache/apisix-dashboard/blob/master/web/src/pages/Route/components/DataLoader/loader/OpenAPI3.tsx) for more details.
:::
119 changes: 119 additions & 0 deletions docs/en/latest/modules/data_loader/openapi3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
title: OpenAPI 3
keywords:
- APISIX
- APISIX Dashboard
- Data Loader
- OpenAPI
description: This document contains information about the OpenAPI 3 data loader.
---

<!--
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
-->

## Overview

OpenAPI 3 data loader currently supports importing standard OpenAPI 3 documentation to generate individual paths as routes and upstreams in Apache APISIX.

## Configuration

| Name | Type | Default | Description |
|--------------|---------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| merge_method | boolean | true | HTTP method merge. When an OpenAPI 3 Path has multiple HTTP methods, this parameter controls whether they are merged or not. When multiple HTTP methods have different detailed configurations, such as `securitySchema` etc., this parameter can be turned off to prevent the APISIX Dashboard importer from merging them into a single APISIX route, which will generate a route for each HTTP method for each Path. |

## Usage

### Import

1. Open the import drawer menu

![Open menu](../../../../assets/images/modules/data_loader/intro.png)

2. Configure import parameters

Select `OpenAPI 3` in the data loader type and set a task name for this import, which will determine the name of the route and upstream generated by this import.

:::note

The current `OpenAPI 3` generates routes and upstream names according to this rule.

- When the task name is `demo` and the HTTP method merge is enabled.

| OpenAPI 3 Path & Method | APISIX route name | APISIX route uri | APISIX route methods |
|-------------------------|--------------------|------------------|----------------------|
| GET /hello1 | demo_hello1 | /hello1 | GET |
| GET & POST /hello2 | demo_hello2 | /hello2 | GET, POST |
| PUT /hello3/{name} | demo_hello3/{name} | /hello3 | PUT |

- When the task name is `demo` and HTTP method merging is disabled.

| OpenAPI 3 Path & Method | APISIX route name | APISIX route uri | APISIX route methods |
|-------------------------|--------------------------|------------------|----------------------|
| GET /hello1 | demo_hello1_GET | /hello1 | GET |
| PUT & DELETE /hello2 | demo_hello2_PUT | /hello2 | PUT |
| | demo_hello2_DELETE | /hello2 | DELETE |
| PATCH /hello3/{name} | demo_hello3/{name}_PATCH | /hello3 | PATCH |

Generate an empty upstream named `demo`, which has no node data configured for fields such as `nodes`, so users can modify it manually to meet their needs.

:::

The following shows the parameters for the `OpenAPI 3` data loader, which currently has the Merge HTTP Methods configuration.

Finally, select an OpenAPI 3 documentation file in Upload section and submit the form.

![Set import parameters](../../../../assets/images/modules/data_loader/openapi3-1.png)

:::note
Only one OpenAPI 3 documentation file can be selected at this time.
:::

3. After submitting and viewing the import results

When the import successful, the generated routes and the number of upstreams will be displayed.

![Imported successfully](../../../../assets/images/modules/data_loader/openapi3-2.png)

When the import fails, the number of errors and the reason for the error are displayed.

![Import failure](../../../../assets/images/modules/data_loader/openapi3-3.png)

4. View the generated routes

![Route List](../../../../assets/images/modules/data_loader/openapi3-4.png)

5. Modify upstream configuration

The import process generates an upstream named `demo` with the same name as the import task.

![Upstream List](../../../../assets/images/modules/data_loader/openapi3-5.png)

It does not have an upstream node configured and cannot be requested properly yet, so you need to modify its node or service discovery configuration. Remove this default node and add node information according to your own service.

![Upstream configuration](../../../../assets/images/modules/data_loader/openapi3-6.png)

Save the upstream configuration.

6. Test API

Use the test tool to call the API to determine if it is configured correctly.

### Export

Not supported yet.