feat: refactor Plugin Orchestration #1813

juzhiyuan · 2021-04-21T10:24:57Z

Please answer these questions before submitting a pull request, or your PR will get closed.

Why submit this pull request?

Bugfix
New feature provided
Improve performance
Backport patches

What changes will this PR take into?

Hi, this PR aims to refactor then improve the Plugin Orchestration feature :)

use X6 to replace the core of FlowChart;
support Plugin Card (Category);
support Start & End & Condition card;
support card limitation: Start card with bottom Connection Port; End card with top Connection Port;
support Plugin Editor (In Progress)
Data transformer;
i18n of course;

Thanks to https://github.com/antvis/x6 .

Related issues

resolve #1247

Project

https://github.com/apache/apisix-dashboard/projects/5

Checklist:

Did you explain what problem does this PR solve? Or what new features have been added?
Have you added corresponding test cases?
Have you modified the corresponding document?
Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

netlify · 2021-04-21T10:25:03Z

Deploy preview for apisix-dashboard ready!

Built with commit 5d6f3f7

https://deploy-preview-1813--apisix-dashboard.netlify.app

codecov-commenter · 2021-04-21T11:02:09Z

Codecov Report

Merging #1813 (e27bcd6) into master (1770596) will decrease coverage by 18.80%.
The diff coverage is n/a.

❗ Current head e27bcd6 differs from pull request most recent head 5d6f3f7. Consider uploading reports for the commit 5d6f3f7 to get more accurate results

@@             Coverage Diff             @@
##           master    #1813       +/-   ##
===========================================
- Coverage   71.22%   52.42%   -18.81%     
===========================================
  Files         173       39      -134     
  Lines        6221     2745     -3476     
  Branches      712        0      -712     
===========================================
- Hits         4431     1439     -2992     
+ Misses       1534     1112      -422     
+ Partials      256      194       -62

Flag	Coverage Δ
backend-e2e-test	`?`
backend-e2e-test-ginkgo	`?`
backend-unit-test	`52.42% <ø> (-0.04%)`	⬇️
frontend-e2e-test	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
api/internal/utils/version.go	`0.00% <0.00%> (-100.00%)`	⬇️
api/internal/filter/request_id.go	`0.00% <0.00%> (-100.00%)`	⬇️
api/internal/core/entity/entity.go	`0.00% <0.00%> (-100.00%)`	⬇️
api/internal/core/store/storehub.go	`0.00% <0.00%> (-71.03%)`	⬇️
api/internal/filter/cors.go	`0.00% <0.00%> (-66.67%)`	⬇️
api/internal/filter/schema.go	`0.00% <0.00%> (-55.47%)`	⬇️
api/internal/utils/consts/api_error.go	`0.00% <0.00%> (-50.00%)`	⬇️
api/internal/handler/data_loader/route_import.go	`27.41% <0.00%> (-37.50%)`	⬇️
api/internal/handler/handler.go	`42.59% <0.00%> (-35.19%)`	⬇️
api/internal/handler/schema/schema.go	`66.66% <0.00%> (-33.34%)`	⬇️
... and 143 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1770596...5d6f3f7. Read the comment docs.

juzhiyuan · 2021-04-24T12:51:33Z

fixing CI

juzhiyuan · 2021-04-24T12:59:02Z

Test building failed due to ESLint error, it doesn't affect the Netlify Preview. I will continue fixing them.

web/src/components/PluginFlow/PluginFlow.tsx

web/src/components/PluginFlow/components/ConfigPanel/ConfigNode/index.tsx

LiteSun · 2021-04-26T07:03:45Z

I pulled the code into a local run and tried it out, and the new plugin orchestration works great.
But I found two small problems:

how do I know which one is the true exit and which one is the false exit for each conditional link?
the problem of automatic resize, see the video below for details
https://user-images.githubusercontent.com/31329157/116041986-7aca7980-a6a0-11eb-9ed5-e31e8f89395a.mov
how to enable the plug-in for users in orchestration mode? Drag the plug-in to the drawing board and still need enable to enable the plug-in?

juzhiyuan · 2021-05-05T13:36:42Z

I will add E2E test cases for this PR

nic-chen · 2021-05-06T01:06:46Z

As for key is required issue, @nic-chen please take a look.

curl 'http://localhost:8000/apisix/admin/routes' \
  -H 'Connection: keep-alive' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"' \
  -H 'Accept: application/json' \
  -H 'Authorization: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAyMjIzNzgsImlhdCI6MTYyMDIxODc3OCwic3ViIjoiYWRtaW4ifQ.n-kqe-O125cu1p0T01FhMKFRyg6MsRE2sqpTmMeEQO8' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36' \
  -H 'Content-Type: application/json;charset=UTF-8' \
  -H 'Origin: http://localhost:8000' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Referer: http://localhost:8000/routes/create' \
  -H 'Accept-Language: en,zh-CN;q=0.9,zh;q=0.8' \
  -H 'Cookie: _ga=GA1.1.1414702999.1618817327; _ga_WQLBQL6GY3=GS1.1.1618817327.1.1.1618817540.0; next-i18next=en-US; _gid=GA1.1.619947394.1620143250' \
  --data-raw '{"uris":["/*"],"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"priority":0,"name":"test222","desc":"","status":1,"labels":{},"script":{"chart":{"cells":[{"shape":"edge","attrs":{"line":{"stroke":"#5F95FF","strokeWidth":1,"targetMarker":{"name":"classic","size":8}}},"id":"25e20dd5-ca57-4b60-ba05-405b0a078ac0","router":{"name":"manhattan"},"zIndex":0,"source":{"cell":"69ca1277-fd6c-4486-b7df-4de4547f30d9","port":"fa3f2c5b-8fd5-482b-8544-4148d0f0bb00"},"target":{"cell":"be7c5c72-00be-4678-92c1-cda07ecfe735","port":"b5ddd0eb-4ecb-462b-9723-019087933544"}},{"position":{"x":160,"y":90},"size":{"width":80,"height":42},"attrs":{"text":{"textWrap":{"text":"开始"}},"body":{"rx":24,"ry":24}},"shape":"flow-chart-start-rect","ports":{"groups":{"top":{"position":"top","attrs":{"circle":{"r":3,"magnet":true,"stroke":"#5F95FF","strokeWidth":1,"fill":"#fff","style":{"visibility":"hidden"}}}},"right":{"position":"right","attrs":{"circle":{"r":3,"magnet":true,"stroke":"#5F95FF","strokeWidth":1,"fill":"#fff","style":{"visibility":"hidden"}}}},"bottom":{"position":"bottom","attrs":{"circle":{"r":3,"magnet":true,"stroke":"#5F95FF","strokeWidth":1,"fill":"#fff","style":{"visibility":"hidden"}}}},"left":{"position":"left","attrs":{"circle":{"r":3,"magnet":true,"stroke":"#5F95FF","strokeWidth":1,"fill":"#fff","style":{"visibility":"hidden"}}}}},"items":[{"group":"bottom","id":"fa3f2c5b-8fd5-482b-8544-4148d0f0bb00"}]},"id":"69ca1277-fd6c-4486-b7df-4de4547f30d9","zIndex":1},{"position":{"x":100,"y":187},"size":{"width":200,"height":60},"attrs":{"text":{"text":"key-auth"},"title":{"text":"key-auth"}},"shape":"flow-chart-plugin-rect","ports":{"groups":{"top":{"position":"top","attrs":{"circle":{"r":3,"magnet":true,"stroke":"#5F95FF","strokeWidth":1,"fill":"#fff","style":{"visibility":"hidden"}}}},"bottom":{"position":"bottom","attrs":{"circle":{"r":3,"magnet":true,"stroke":"#5F95FF","strokeWidth":1,"fill":"#fff","style":{"visibility":"hidden"}}}}},"items":[{"group":"top","id":"b5ddd0eb-4ecb-462b-9723-019087933544"},{"group":"bottom","id":"de0bae68-787d-4977-b75b-b8f030d0636e"}]},"id":"be7c5c72-00be-4678-92c1-cda07ecfe735","zIndex":2,"data":{"disable":false}}]},"conf":{"be7c5c72-00be-4678-92c1-cda07ecfe735":{"disable":false}},"rule":{"root":"be7c5c72-00be-4678-92c1-cda07ecfe735"}},"vars":[],"upstream_id":"352139465977758504"}' \
  --compressed

ok

juzhiyuan · 2021-05-06T03:26:26Z

https://github.com/apache/apisix-dashboard/pull/1813/checks?check_run_id=2514882778

@nic-chen @starsz Please check this CI

LiteSun · 2021-05-10T00:27:38Z

web/cypress/integration/plugin/create-route-with-plugin-orchestration.spec.js

+    groupButton: '.ant-radio-group',
+    canvas: '.x6-graph-svg',
+    startNode:
+      '#stencil > div > div.x6-widget-stencil-content > div:nth-child(1) > div > div > svg > g > g.x6-graph-svg-stage > g:nth-child(1) > g > circle',


it would be better to use data-* (https://docs.cypress.io/guides/references/best-practices#Selecting-Elements) to reduce selector length.

web/cypress/integration/plugin/create-route-with-plugin-orchestration.spec.js

web/package.json

starsz · 2021-05-10T03:35:56Z

https://github.com/apache/apisix-dashboard/pull/1813/checks?check_run_id=2514882778

@nic-chen @starsz Please check this CI

It looks good now.

…ard into feat-plugin-flow

juzhiyuan · 2021-05-11T01:08:49Z

Hi, I just fixed the dag-to-lua issue in CI, and retriggered the Test to make sure it's stable as expected.

juzhiyuan · 2021-05-11T01:59:46Z

CI passes twice, cc @liuxiran @imjoey @LiteSun @bzp2010 to review, thanks!

imjoey

/lgtm
Thanks.

feat: added PluginFlow

3bd1ece

juzhiyuan force-pushed the feat-plugin-flow branch 2 times, most recently from 549b064 to dcebacd Compare April 23, 2021 11:39

feat: support transformer

bdf6a7b

juzhiyuan force-pushed the feat-plugin-flow branch from dcebacd to bdf6a7b Compare April 23, 2021 11:45

juzhiyuan added 6 commits April 23, 2021 19:52

chore: use English

d70f25c

chore: added convertToData

7e13934

chore: remove debug data

001e449

feat: use notification

2f694e3

chore: refactor Create Route

94189cb

chore: update typings

8e2ccf2

juzhiyuan marked this pull request as ready for review April 24, 2021 05:32

juzhiyuan added 5 commits April 24, 2021 15:23

feat: support i18n

824f228

feat: support i18n

05ace2b

fix: save plugin data

4c23d3b

feat: support i18n

1cda61e

feat: support i18n

3134474

juzhiyuan requested review from LiteSun and liuxiran and removed request for LiteSun April 24, 2021 08:51

juzhiyuan added 3 commits April 24, 2021 20:35

fix: create route

bd06de5

chore: update i18n

1033189

chore: update i18n

73c56f2

juzhiyuan requested a review from LiteSun April 24, 2021 12:45

LiteSun reviewed Apr 25, 2021

View reviewed changes

web/src/components/PluginFlow/PluginFlow.tsx Outdated Show resolved Hide resolved

web/src/components/PluginFlow/components/ConfigPanel/ConfigNode/index.tsx Outdated Show resolved Hide resolved

juzhiyuan requested review from liuxiran and LiteSun May 5, 2021 13:32

juzhiyuan added 2 commits May 6, 2021 11:17

disable eslint tip

b514ae3

Merge branch 'master' into feat-plugin-flow

f2f885d

test: added testcase

bc0f43b

LiteSun reviewed May 10, 2021

View reviewed changes

juzhiyuan added 10 commits May 10, 2021 21:59

fix: ci

243b2ae

Merge branch 'master' into feat-plugin-flow

d72e538

Update package.json

1cf880b

fix: support running dag-lib

a2074ad

Merge branch 'feat-plugin-flow' of github.com:juzhiyuan/apisix-dashbo…

27048b0

…ard into feat-plugin-flow

fix: support running dag-lib

827fde6

fix: support running dag-lib

d6c2619

fix: support running dag-lib

f712fc9

fix: support running dag-lib

f3c9985

fix: support running dag-lib

5d6f3f7

bzp2010 approved these changes May 11, 2021

View reviewed changes

liuxiran approved these changes May 11, 2021

View reviewed changes

LiteSun approved these changes May 11, 2021

View reviewed changes

juzhiyuan merged commit 423c9e8 into apache:master May 11, 2021

juzhiyuan deleted the feat-plugin-flow branch May 11, 2021 06:36

imjoey approved these changes May 11, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: refactor Plugin Orchestration #1813

feat: refactor Plugin Orchestration #1813

juzhiyuan commented Apr 21, 2021 •

edited

Loading

netlify bot commented Apr 21, 2021 •

edited

Loading

codecov-commenter commented Apr 21, 2021 •

edited

Loading

juzhiyuan commented Apr 24, 2021

juzhiyuan commented Apr 24, 2021 •

edited

Loading

LiteSun commented Apr 26, 2021 •

edited

Loading

juzhiyuan commented May 5, 2021

nic-chen commented May 6, 2021

juzhiyuan commented May 6, 2021

LiteSun May 10, 2021

starsz commented May 10, 2021

juzhiyuan commented May 11, 2021

juzhiyuan commented May 11, 2021

imjoey left a comment

feat: refactor Plugin Orchestration #1813

feat: refactor Plugin Orchestration #1813

Conversation

juzhiyuan commented Apr 21, 2021 • edited Loading

netlify bot commented Apr 21, 2021 • edited Loading

codecov-commenter commented Apr 21, 2021 • edited Loading

Codecov Report

juzhiyuan commented Apr 24, 2021

juzhiyuan commented Apr 24, 2021 • edited Loading

LiteSun commented Apr 26, 2021 • edited Loading

juzhiyuan commented May 5, 2021

nic-chen commented May 6, 2021

juzhiyuan commented May 6, 2021

LiteSun May 10, 2021

Choose a reason for hiding this comment

starsz commented May 10, 2021

juzhiyuan commented May 11, 2021

juzhiyuan commented May 11, 2021

imjoey left a comment

Choose a reason for hiding this comment

juzhiyuan commented Apr 21, 2021 •

edited

Loading

netlify bot commented Apr 21, 2021 •

edited

Loading

codecov-commenter commented Apr 21, 2021 •

edited

Loading

juzhiyuan commented Apr 24, 2021 •

edited

Loading

LiteSun commented Apr 26, 2021 •

edited

Loading