Testable (Phase 1)

[MauricioUyaguari]

Summary

Related to: #1077
Version V0 of testable manager.

With this PR, Users can define testable tests in text mode. This includes ones with relational data.
Run the tests via the testable manager on the side bar. View failing tests (expected <-> actual) data.
Yet to implement: Testable Editors
Depended on release finos/legend-engine#658 to work but is backward compatible.

How did you test this change?

• Test(s) added
• Manual testing (please provide screenshots/recordings)
• No testing (please provide an explanation)

testable.mp4

TESTDATA FOR VIDEO

###Data
Data data::RelationalData
{
  Relational
  #{
    PersonTable:
      {
        columns:
        [
          'id',
          'firm_id',
          'firstName',
          'lastName'
        ];
        rows:
        [
          '1,1,John,Doe',
          '2,1,Nicole,Smith',
          '3,2,Time,Smith'
        ];
      }
    FirmTable:
      {
        columns:
        [
          'id',
          'legal_name'
        ];
        rows:
        [
          '1,Finos',
          '2,Apple'
        ];
      }
  }#
}


###Service
Service execution::service::MyServiceWithTDSRetrurn2
{
  pattern: '/d2c48a9c-70fa-46e3-8173-c355e774004f';
  documentation: '';
  autoActivateUpdates: true;
  execution: Single
  {
    query: |model::Firm.all()->project([x|$x.employees.firstName, x|$x.employees.lastName, x|$x.legalName], ['Employees/First Name', 'Employees/Last Name', 'Legal Name']);
    mapping: execution::RelationalMapping;
    runtime: execution::Runtime;
  }
  testSuites:
  [
    testSuite1:
    {
      data:
      [
        connections:
        [
          connection_1:
            Reference
            #{
              data::RelationalData
            }#
        ]
      ]
      tests:
      [
        test1:
        {
          asserts:
          [
            shouldPass:
              EqualToJson
              #{
                expected : 
                  ExternalFormat
                  #{
                    contentType: 'application/json';
                    data: '[{"Employees/First Name":"John","Employees/Last Name":"Doe","Legal Name":"Finos"},{"Employees/First Name":"Nicole","Employees/Last Name":"Smith","Legal Name":"Finos"},{"Employees/First Name":"Time","Employees/Last Name":"Smith","Legal Name":"Apple"}]\n';
                  }#;
              }#,
            shouldFail:
              EqualToJson
              #{
                expected : 
                  ExternalFormat
                  #{
                    contentType: 'application/json';
                    data: '[{"Employees/First Name":"JohnWW","Employees/Last Name":"Doe","Legal Name":"Finos"},{"Employees/First Name":"Nicole","Employees/Last Name":"Smith","Legal Name":"Finos"},{"Employees/First Name":"Time","Employees/Last Name":"Smith","Legal Name":"Apple"}]\n';
                  }#;
              }#
          ]
        }
      ]
    }
  ]
}

Service execution::service::MyServiceWithTDSRetrurn
{
  pattern: '/d2c48a9c-70fa-46e3-8173-c355e774004f';
  documentation: '';
  autoActivateUpdates: true;
  execution: Single
  {
    query: |model::Firm.all()->project([x|$x.employees.firstName, x|$x.employees.lastName, x|$x.legalName], ['Employees/First Name', 'Employees/Last Name', 'Legal Name']);
    mapping: execution::RelationalMapping;
    runtime: execution::Runtime;
  }
  testSuites:
  [
    testSuite1:
    {
      data:
      [
        connections:
        [
          connection_1:
            Reference
            #{
              data::RelationalData
            }#
        ]
      ]
      tests:
      [
        test1:
        {
          asserts:
          [
            shouldPass:
              EqualToJson
              #{
                expected : 
                  ExternalFormat
                  #{
                    contentType: 'application/json';
                    data: '[{"Employees/First Name":"John","Employees/Last Name":"Doe","Legal Name":"Finos"},{"Employees/First Name":"Nicole","Employees/Last Name":"Smith","Legal Name":"Finos"},{"Employees/First Name":"Time","Employees/Last Name":"Smith","Legal Name":"Apple"}]\n';
                  }#;
              }#,
            shouldFail:
              EqualToJson
              #{
                expected : 
                  ExternalFormat
                  #{
                    contentType: 'application/json';
                    data: '[{"Employees/First Name":"John","Employees/Last Name":"Doe","Legal Name":"Finos"},{"Employees/First Name":"Nicole","Employees/Last Name":"Smith","Legal Name":"Finos"},{"Employees/First Name":"Time","Employees/Last Name":"Smith","Legal Name":"Apple"}]\n';
                  }#;
              }#
          ]
        }
      ]
    }
  ]
}

Service execution::service::MyServiceWithTDSRetrurnError
{
  pattern: '/d2c48a9c-70fa-46e3-8173-c355e774004f';
  documentation: '';
  autoActivateUpdates: true;
  execution: Single
  {
    query: |model::Firm.all()->graphFetch(#{model::Firm{employees{firstName,lastName},legalName}}#)->serialize(#{model::Firm{employees{firstName,lastName},legalName}}#);
    mapping: execution::RelationalMapping;
    runtime: execution::Runtime;
  }
  testSuites:
  [
    testSuite1:
    {
      data:
      [
        connections:
        [
          connection_1:
            Reference
            #{
              data::RelationalData
            }#
        ]
      ]
      tests:
      [
        test1:
        {
          asserts:
          [
            shouldPass:
              EqualToJson
              #{
                expected : 
                  ExternalFormat
                  #{
                    contentType: 'application/json';
                    data: '[{"Employees/First Name":"John","Employees/Last Name":"Doe","Legal Name":"Finos"},{"Employees/First Name":"Nicole","Employees/Last Name":"Smith","Legal Name":"Finos"},{"Employees/First Name":"Time","Employees/Last Name":"Smith","Legal Name":"Apple"}]\n';
                  }#;
              }#,
            shouldFail:
              EqualToJson
              #{
                expected : 
                  ExternalFormat
                  #{
                    contentType: 'application/json';
                    data: '[{"Employees/First Name":"JohnWW","Employees/Last Name":"Doe","Legal Name":"Finos"},{"Employees/First Name":"Nicole","Employees/Last Name":"Smith","Legal Name":"Finos"},{"Employees/First Name":"Time","Employees/Last Name":"Smith","Legal Name":"Apple"}]\n';
                  }#;
              }#
          ]
        }
      ]
    }
  ]
}


###Relational
Database store::TestDB
(
  Table FirmTable
  (
    id INTEGER PRIMARY KEY,
    legal_name VARCHAR(200)
  )
  Table PersonTable
  (
    id INTEGER PRIMARY KEY,
    firm_id INTEGER,
    firstName VARCHAR(200),
    lastName VARCHAR(200)
  )

  Join FirmPerson(PersonTable.firm_id = FirmTable.id)
)


###Pure
Class model::Person
{
  firstName: String[1];
  lastName: String[1];
}

Class model::Firm
{
  legalName: String[1];
  employees: model::Person[1];
}


###Mapping
Mapping execution::RelationalMapping
(
  *model::Person: Relational
  {
    ~primaryKey
    (
      [store::TestDB]PersonTable.id
    )
    ~mainTable [store::TestDB]PersonTable
    firstName: [store::TestDB]PersonTable.firstName,
    lastName: [store::TestDB]PersonTable.lastName
  }
  *model::Firm: Relational
  {
    ~primaryKey
    (
      [store::TestDB]FirmTable.id
    )
    ~mainTable [store::TestDB]FirmTable
    legalName: [store::TestDB]FirmTable.legal_name,
    employees[model_Person]: [store::TestDB]@FirmPerson
  }

  MappingTests
  [
    test_1
    (
      query: |model::Firm.all()->project([x|$x.employees.firstName, x|$x.employees.lastName, x|$x.legalName], ['Employees/First Name', 'Employees/Last Name', 'Legal Name']);
      data:
      [
        <Relational, CSV, store::TestDB, 
          'default\n'+
          'PersonTable\n'+
          'id,firm_id,firstName,lastName\n'+
          '1,1,John,Doe\n'+
          '2,1,Nicole,Smith\n'+
          '3,2,Tim,Smith\n'+
          '----\n'+
          'default\n'+
          'FirmTable\n'+
          'id,legal_Name\n'+
          '1,Finos\n'+
          '2,Apple\n'+
          '\n\n\n'
        >
      ];
      assert: '[{"values":["John","Doe","Finos"]},{"values":["Nicole","Smith","Finos"]},{"values":["Tim","Smith","Apple"]}]';
    )
  ]
)


###Connection
RelationalDatabaseConnection model::MyConnection
{
  store: store::TestDB;
  type: H2;
  specification: LocalH2
  {
    testDataSetupSqls: [
      'Drop table if exists FirmTable;\nDrop table if exists PersonTable;\nCreate Table FirmTable(id INT, Legal_Name VARCHAR(200));\nCreate Table PersonTable(id INT, firm_id INT, lastName VARCHAR(200), firstName VARCHAR(200));\nInsert into FirmTable (id, Legal_Name) values (1, \'FirmA\');\nInsert into FirmTable (id, Legal_Name) values (2, \'Apple\');\nInsert into PersonTable (id, firm_id, lastName, firstName) values (1, 1, \'John\', \'Doe\');\nInsert into PersonTable (id, firm_id, lastName, firstName) values (2, 2, \'Tim\', \'Smith\');\nInsert into PersonTable (id, firm_id, lastName, firstName) values (3, 3, \'Nicole\', \'Doe\');\n\n'
      ];
  };
  auth: DefaultH2;
}


###Runtime
Runtime execution::Runtime
{
  mappings:
  [
    execution::RelationalMapping
  ];
  connections:
  [
    store::TestDB:
    [
      connection_1: model::MyConnection
    ]
  ];
}

[changeset-bot]

🦋 Changeset detected

Latest commit: 547e238

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 24 packages

Name	Type
@finos/legend-studio	Minor
@finos/legend-graph	Minor
@finos/legend-manual-tests	Patch
@finos/legend-art	Patch
@finos/legend-extension-dsl-data-space	Patch
@finos/legend-extension-dsl-diagram	Patch
@finos/legend-extension-dsl-persistence	Patch
@finos/legend-extension-dsl-text	Patch
@finos/legend-extension-external-language-morphir	Patch
@finos/legend-extension-external-store-service	Patch
@finos/legend-extension-mapping-generation	Patch
@finos/legend-studio-app	Patch
@finos/legend-studio-extension-management-toolkit	Patch
@finos/legend-studio-extension-query-builder	Patch
@finos/legend-application	Patch
@finos/legend-extension-external-format-json-schema	Patch
@finos/legend-graph-extension-collection	Patch
@finos/legend-query	Patch
@finos/legend-taxonomy	Patch
@finos/legend-query-app	Patch
@finos/legend-taxonomy-app	Patch
@finos/legend-studio-deployment	Patch
@finos/legend-query-deployment	Patch
@finos/legend-taxonomy-deployment	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[codecov]

Codecov Report

Merging #1114 (547e238) into master (27df808) will decrease coverage by 0.19%.
The diff coverage is 21.75%.

@@            Coverage Diff             @@
##           master    #1114      +/-   ##
==========================================
- Coverage   42.82%   42.62%   -0.20%     
==========================================
  Files        1125     1128       +3     
  Lines       49360    49804     +444     
  Branches    11274    11386     +112     
==========================================
+ Hits        21137    21231      +94     
- Misses      28153    28503     +350     
  Partials       70       70              

Impacted Files	Coverage Δ
packages/legend-graph/src/MetaModelConst.ts	100.00% <ø> (ø)
packages/legend-graph/src/graph/BasicModel.ts	68.18% <0.00%> (-0.37%)	⬇️
packages/legend-graph/src/graph/PureGraphPlugin.ts	100.00% <ø> (ø)
...graph/src/graphManager/AbstractPureGraphManager.ts	100.00% <ø> (ø)
...ction/changeDetection/DSLService_ObserverHelper.ts	22.10% <0.00%> (+0.23%)	⬆️
...ph/src/models/metamodels/pure/data/EmbeddedData.ts	38.46% <ø> (ø)
...models/pure/packageableElements/service/Service.ts	83.33% <ø> (ø)
...gend-graph/src/models/metamodels/pure/test/Test.ts	100.00% <ø> (ø)
...c/models/metamodels/pure/test/assertion/EqualTo.ts	0.00% <0.00%> (ø)
...dels/metamodels/pure/test/assertion/EqualToJson.ts	50.00% <0.00%> (-50.00%)	⬇️
... and 44 more

[akphi]

Left reviews to clean this up since we need to merge this to unblock development. Comments are mostly about cosmetics. The only really major thing is I don't fully grasp why we complicate the extension mechanism, so we should dumb it down.