Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conformance additions for 2.2 (second attempt) #1426

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
* Added reference materials and supported platforms information for FDC3 in .NET via the [finos/fdc3-dotnet](https://github.com/finos/fdc3-dotnet) project. ([#1108](https://github.com/finos/FDC3/pull/1108))
* Specifications for getAgent() and Browser-Resident Desktop Agents. ([#1191](https://github.com/finos/FDC3/pull/1191))
* Specification for Preload Desktop Agents. This content was previously in the supported platforms section. It had been revised and amended to include recommended behavior related to the new validateAppIdentity() function. ([#1191](https://github.com/finos/FDC3/pull/1191))
* Added optional validateAppIdentity() function to DesktopAgent interface. ([#1191](https://github.com/finos/FDC3/pull/1191))
* Typescript definitions for getAgent() and related types. ([#1191](https://github.com/finos/FDC3/pull/1191))
* Typescript definitions for Desktop Agent Communication Protocol (DACP). These constitute the internal "wire protocol" that the "@finos/fdc3" library uses to communicate with Browser-Resident DAs. ([#1191](https://github.com/finos/FDC3/pull/1191))
* Typescript definitions for Web Connection Protocol (WCP). These constitute the messages used to establish connectivity between "@finos/fdc3" and a Browser-Resident DA. ([#1191](https://github.com/finos/FDC3/pull/1191))
* Added support for broadcast actions to the `fdc3.action` context type, allowing an Action to represent the broadcast of a specified context to an app or user channel. ([#1368](https://github.com/finos/FDC3/pull/1368))
* Added utility functions `isStandardContextType(contextType: string)`, `isStandardIntent(intent: string)`,`getPossibleContextsForIntent(intent: StandardIntent)`. ([#1139](https://github.com/finos/FDC3/pull/1139))
* Added support for event listening outside of intent or context listnener. Added new function `addEventListener`, type `EventHandler`, enum `FDC3EventType` and interfaces `FDC3Event` and `FDC3ChannelChangedEvent`. ([#1207](https://github.com/finos/FDC3/pull/1207))
* Added new `CreateOrUpdateProfile` intent. ([#1359](https://github.com/finos/FDC3/pull/1359))
* Added conformance tests into the FDC3 API documentation in the current version and backported into 2.0 and 2.1. Removed outdated 1.2 conformance tests (which are preserved in the older 2.0 and 2.1 versions). ([#1417](https://github.com/finos/FDC3/pull/1417)).
* Added conformance tests into the FDC3 API documentation in the current version and back-ported into 2.0 and 2.1. Removed outdated 1.2 conformance tests (which are preserved in the older 2.0 and 2.1 versions). ([#1417](https://github.com/finos/FDC3/pull/1417)).
* Added conformance tests to documentation for features introduced in FDC3 2.2 (`fdc3.addEventListener`, `PrivateChannel.addEventListener` and `getAgent`). ([#1425](https://github.com/finos/FDC3/pull/1425))

### Changed

Expand Down
14 changes: 14 additions & 0 deletions docs/api/conformance/Basic-Tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,20 @@ hide_title: true

_These are some basic sanity tests implemented in the FDC3 Conformance Framework. It is expected that Desktop Agent testers will run these first before commencing the much more thorough tests in section 2 onwards._

## Connection

![2.2+](https://img.shields.io/badge/FDC3-2.2+-purple) In FDC3 2.2, a new interface was introduced to support Browser-based FDC3 Desktop Agents, known as a 'Desktop Agent Proxy', and a new [`getAgent`](../ref/GetAgent) API call was introduced to all apps to retrieve the Desktop Agent API via that interface or the existing 'Desktop Agent Preload' interface. This test pack checks that the a connection is made correctly via `getAgent`.

| App | Step | Description |
|-----|-----------------|----------------------------------------------------------|
| A | `getAgent` | A calls `getAgent` and waits for the promise to resolve to a `DesktopAgent` instance. |
kriswest marked this conversation as resolved.
Show resolved Hide resolved
kriswest marked this conversation as resolved.
Show resolved Hide resolved
| A | `getInfo` | A can call the `getInfo()` method on the `DesktopAgent` instance to get the `ImplementationMetadata` object. <br /> Check that fdc3Version is set to 2.2. <br />Check that provider and providerVersion are populated. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are specifically requesting the provider and its version and in the basic test below (BasicGI1) we are just calling out "provider details", should we either update the former or the latter just to be either more specific or more vague? (I would prefer to be more specific, but not sure if the test actually checks for the version in the latter case and if that is intentional.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also similarly just like a general comment, a lot of these rest definitions are written in different styles across all of them, like here it is "A can" in other files it was "should", and sometimes it is just described in present tense. That is a scope creep for this PR but maybe others agree that it should be more standardized, we can create a followup issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the definitions are less than consistent and do need a clean-up. @robmoffat is keen to re-write all these cases and their implementation in gherkin syntax (although it's not clear that he will have time to do so next year - so that may be on the maintainers to do), which would be a good time to standardize them.

I would happily support and issue to fix the language (without changing meaning) and maybe able to collaborate on implementing that fix. It is out of scope for this PR however - but could be a good thing to do within 2.2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in the basic tests we say all of 'You can', 'An application can', 'The application should' with no variance in meaning - that is indeed horribly inconsistent!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kemerava I've tried to make the language in the basic tests more consistent (haven't touched the others). I've also made BasicGI1 more specific about the FDC3 version (it should match the version being tested against for conformance).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(those changes were made in #1417

kriswest marked this conversation as resolved.
Show resolved Hide resolved
| A | `getUserChannels`| A can call the `getUserChannels()` method on the `DesktopAgent` instance to get the `Channel` objects representing the system channels.<br />Check **user** channels are returned.|
kriswest marked this conversation as resolved.
Show resolved Hide resolved

- `GetAgentAPI`: ![2.2+](https://img.shields.io/badge/FDC3-2.2+-purple) Perform the above test.

## Basic API Usage

- `BasicCL1`: You can create a context listener by calling `fdc3.addContextListener('fdc3.contact',<handler>)`. A `Listener` object is returned and can be used to remove the listener again by calling its `unsubscribe` function.
- `BasicCL2`: You can create an **unfiltered** context listener by calling `fdc3.addContextListener(null,<handler>)`. A `Listener` object is returned and can be used to remove the listener again by calling its `unsubscribe` function.
- `BasicIL1`: You can create an intent listener by calling `fdc3.addIntentListener(<intent name>,<handler>)`. A `Listener` object is returned and can be used to remove the listener again by calling its `unsubscribe` function.
Expand Down
44 changes: 23 additions & 21 deletions docs/api/conformance/Intents-Tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ You will need to pre-populate the AppDirectory with the following items (some of

| App | Usage | ListensFor `(pattern: intent([context-types…]) (=> result-type)`) | On Startup |
|-----|-------------------------------------------------------|-----------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|
| A | Raise Intent tests without results | `aTestingIntent(testContextX,testContextZ)`<br/>`sharedTestingIntent1(testContextX)` | addIntentListener() for given intents |
| B | Raise Intent tests with Context results | `bTestingIntent(testContextY)`<br/>`sharedTestingIntent1(testContextX, testContextY) => testContextY` | addIntentListener() for given intents |
| A | Raise Intent tests without results | `aTestingIntent(testContextX,testContextZ)`<br />`sharedTestingIntent1(testContextX)` | addIntentListener() for given intents |
| B | Raise Intent tests with Context results | `bTestingIntent(testContextY)`<br />`sharedTestingIntent1(testContextX, testContextY) => testContextY` | addIntentListener() for given intents |
| C | Find Intent tests (never started) | `cTestingIntent(testContextX) => testContextZ` | addIntentListener() for given intents |
| D | Find Intent tests (never started) | `sharedTestingIntent2(testContextX) => testContextZ` | addIntentListener() for given intents |
| E | Find Intent & Raise Intent with Channel result | `sharedTestingIntent2(testContextY) => channel` | addIntentListener() for given intents |
Expand Down Expand Up @@ -186,30 +186,32 @@ Finally, please note that this is a larger set of apps than were required for 1.

## PrivateChannel Lifecycle Events

![2.2+](https://img.shields.io/badge/FDC3-2.2+-purple) In FDC3 2.2, the existing [`PrivateChannel`](../ref/PrivateChannel) lifecycle event functions (`onAddContextListener`, `onUnsubscribe` and `onDisconnect`) functions were deprecated in favour of a new [`addEventListener`](https://fdc3.finos.org/docs/next/api/ref/PrivateChannel#addeventlistener) function which supports events of all three types. Hence, the tests in this section have been updated to use [`addEventListener`](https://fdc3.finos.org/docs/next/api/ref/PrivateChannel#addeventlistener).

| App | Step | Details |
|-------|-----------------|---------------------------------------------------------------------------------------------------|
| Test | 1. Raise intent | Test raises an intent with `fdc3.raiseIntent(‘"kTestingIntent", testContextX, {appId: "<K's appId>"})`<br />starts app K. |
| K | 2. Receive Intent & Context | After starting up, K runs `fdc3.addIntentListener("kTestingIntent")` to register its listener.<br />It them receives `testContextX`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App K's `appId` and `instanceId` set. |
| Test | 4. await results | Test should `await resolution.getResult()` on the `IntentResolution` object returned in the previous step. A promise should be returned quickly. |
| K | 5. Create PrivateChannel and setup event listeners | K should create a `PrivateChannel` object via `const privChan = await fdc3.createPrivateChannel()`,<br />it should then add listeners for the 3 events offered + a context listener via:<br />- `const listener1 = await privChan.onAddContextListener(handler1);`<br />- `const listener2 = await privChan.onUnsubscribe(handler2);`<br />- `const listener3 = await privChan.onDisconnect(handler3);`<br />- `const listener4 = await privChan.addContextListener("testContextX", handler4)`<br />it should then return the `PrivateChannel`. |
| Test | 6. receive PrivateChannel | The promise received by Test from `resolution.getResult()` should resolve to a `PrivateChannel` object. Confirm that the `type` of the Channel object is "private". |
| Test | 7. addContextListener | Test should add a context listener to the PrivateChannel object via `const listener1 = privChan.addContextListener("testContextZ", handler)` |
| K | 8. Receive event & broadcast context | The `onAddContextListener` handler (`listener1`) added in step 5 should fire after Test adds its listener. Once it has, K should broadcast a short stream of `testContextZ` objects, with consecutive integer values in them (e.g. 1-5). |
| Test | 9. Unsubscribe listener | Test should confirm receipt of the expected context objects, in the expected order, broadcast by K. It should then remove its context listener with `listener1.unsubscribe().` |
| K | 10. Receive unsubscribe event | The event handler registered by K via `onUnsubscribe` should fire. If it does not and the test moves to a subsequent step, K should indicate this to the test runner (failing the test).|
| Test | 11. Broadcast context | Test should broadcast at least one `testContextX` object via the PrivateChannel (back to K). |
| K | 12. Receive context | K should confirm receipt of the expected context. If it does not and the test moves to a subsequent step K should indicate this to the test runner (failing the test).|
| Test | 13. re-run addContextListener | Test should (again) add a context listener to the PrivateChannel object via `const listener2 = privChan.addContextListener("testContextZ", handler)` |
| K | 14. Receive event & broadcast context | The `onAddContextListener` handler added in step 5 should (again) fire after Test adds its listener. Once it has, K should again broadcast a short stream of `testContextZ` objects, with consecutive integer values in them (e.g. 6-10). |
| Test | 15. Disconnect | Test should (again) confirm receipt of the expected context objects, in the expected order, broadcast by K. It should then disconnect from the channel with [`privChan.disconnect().`](https://fdc3.finos.org/docs/api/ref/PrivateChannel#disconnect) |
| K | 16. Receive events & cleanup | The `onUnsubscribe` handler added in step 5 should (again) fire after Test calls `privChan.disconnect()`. Subsequently, the `onDisconnect` handler also added in step 5 should fire. Once it has, K can unsubscribe its listeners, indicate to the test runner that all steps were completed and close. |

- `2.0-PrivateChannelsLifecycleEvents`: Perform the above test.
| Test | 1. Raise intent | Test raises an intent with `fdc3.raiseIntent(‘"kTestingIntent", testContextX, {appId: "<K's appId>"})`<br />starts app K. |
| K | 2. Receive Intent & Context | After starting up, K runs `fdc3.addIntentListener("kTestingIntent")` to register its listener.<br />It them receives `testContextX`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App K's `appId` and `instanceId` set. |
| Test | 4. await results | Test should `await resolution.getResult()` on the `IntentResolution` object returned in the previous step. A promise should be returned quickly. |
| K | 5. Create PrivateChannel and setup event listeners | K should create a `PrivateChannel` object via `const privChan = await fdc3.createPrivateChannel()`,<br />it should then add listeners for the 3 events offered + a context listener via:<br />- `const listener1 = await privChan.addEventListener("addContextListener", handler1);`<br />- `const listener2 = await privChan.addEventListener("unsubscribe", handler2);`<br />- `const listener3 = await privChan.addEventListener("disconnect", handler3);`<br />- `const listener4 = await privChan.addEventListener(null, handler4);`<br />- `const listener5 = await privChan.addContextListener("testContextX", handler5)`<br />it should then return the `PrivateChannel`. |
| Test | 6. receive PrivateChannel | The promise received by Test from `resolution.getResult()` should resolve to a `PrivateChannel` object. Confirm that the `type` of the Channel object is "private". |
| Test | 7. addContextListener | Test should add a context listener to the PrivateChannel object via `const testListener1 = privChan.addContextListener("testContextZ", contextHandler)` |
| K | 8. Receive event & broadcast context | The handlers for both the "addContextListener" event (`handler1`) and all events (`handler4`) added in step 5 should fire after Test adds its listener. Once it has, K should broadcast a short stream of `testContextZ` objects, with consecutive integer values in them (e.g. 1-5). |
| Test | 9. Unsubscribe listener | Test should confirm receipt of the expected context objects, in the expected order, broadcast by K. It should then remove its context listener with `testListener1.unsubscribe().` |
| K | 10. Receive unsubscribe event | The handlers for both the "unsubscribe" event (`handler2`)and all events registered by K (`handler4`) should fire. If either does not and the test moves to a subsequent step, K should indicate this to the test runner (failing the test).|
kriswest marked this conversation as resolved.
Show resolved Hide resolved
| Test | 11. Broadcast context | Test should broadcast at least one `testContextX` object via the PrivateChannel (back to K). |
| K | 12. Receive context | K should confirm receipt of the expected context. If it does not and the test moves to a subsequent step K should indicate this to the test runner (failing the test).|
| Test | 13. re-run addContextListener | Test should (again) add a context listener to the PrivateChannel object via `const testListener2 = privChan.addContextListener("testContextZ", contextHandler)` |
| K | 14. Receive event & broadcast context | Both the "addContextListener" event handler (`handler1`) and handler for all events (`handler4`) added in step 5 should (again) fire after Test adds its listener. Once it has, K should again broadcast a short stream of `testContextZ` objects, with consecutive integer values in them (e.g. 6-10). |
| Test | 15. Disconnect | Test should (again) confirm receipt of the expected context objects, in the expected order, broadcast by K. It should then disconnect from the channel with [`privChan.disconnect().`](https://fdc3.finos.org/docs/api/ref/PrivateChannel#disconnect) |
| K | 16. Receive events & cleanup | Both the "unsubscribe" event handler (`handler2`) and handler for all events (`handler4`) added in step 5 should (again) fire after Test calls `privChan.disconnect()`. Subsequently, both the "disconnect" handler (`handler3`) and handler for all events (`handler4`) also added in step 5 should fire. Once they have, K can unsubscribe its listeners, indicate to the test runner that all steps were completed and close. |

- `2.2-PrivateChannelsLifecycleEvents`: ![2.2+](https://img.shields.io/badge/FDC3-2.2+-purple) Perform the above test.

## Resolving Ambiguous Intents

FDC3 Desktop Agent MUST provide a method of resolving ambiguous intents (i.e. those that might be resolved by multiple applications) or unspecified intents (calls to raiseIntentForContext that return multiple options). This is often accomplished by providing a user interface allowing the user to select the desired target application or intent and application.
An FDC3 Desktop Agent MUST provide a method of resolving ambiguous intents (i.e. those that might be resolved by multiple applications) or unspecified intents (calls to raiseIntentForContext that return multiple options). This is often accomplished by providing a user interface allowing the user to select the desired target application or intent and application.

As the methods of resolving ambiguous intents are often user interactive, it is either difficult or impossible to implement an automated test for this. Hence, manual tests should be performed as a final step in a conformance test. These tests are based on the same applications defined for and used in other intent tests - however a separate manual test app should be provided to enable the test.

Expand Down
Loading