Finished Tap Refactor, upgraded all streams to v2#77

Merged

dmosorast merged 221 commits intosinger-io:masterfrom DownstreamDataTeam:release/41

May 17, 2022

+8,887-5,771

Commits on Feb 14, 2022

Merge branch 'master' into 'local-master'
Radu Marinoiu
committed

Commits on Feb 16, 2022

release/37 changes, squashed
Radu Marinoiu
committed

Commits on Feb 18, 2022

Added Snyk dependency scanning
Alexandru Rosca
committed

Commits on Feb 22, 2022

Merge branch 'release-37-squashed' into 'release/37'
Radu Marinoiu
committed
Finished centres refactor
Radu Marinoiu
committed

Commits on Feb 23, 2022

Added unit tests for TapGenerator
Radu Marinoiu
committed
Added unit tests for TapGenerator
Radu Marinoiu
committed
Added unit tests for centres
Radu Marinoiu
committed
Refactored Clients stream and created unit tests for it
Radu Marinoiu
committed
Updated Snyk CI tests
Alexandru Rosca
committed
Updated Snyk code test job
Alexandru Rosca
committed

Commits on Feb 24, 2022

Merge branch 'feature/ECDDC-575_Refactor-clients-stream' into 'release/38'
Mihai Antonio Caciuc
committed
Merge branch 'release/38' into 'feature/ECDDC-555_Refactor-centres-stream'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-555_Refactor-centres-stream' into 'release/38'
Radu Marinoiu
committed
Updated Snyk code test job
Alexandru Rosca
committed
Allowed dev job to fail
Alexandru Rosca
committed
Added field "assigned_branch_key" to clients stream
Radu Marinoiu
committed
Refactored the branches stream
Alexandru Rosca
committed
Added unit tests
Alexandru Rosca
committed
Renamed unit test functions for centres and clients
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-604_Add-assigned-branch-key-to-clients-stream' into 'release/38'
Radu Marinoiu
committed
Added "branches" in sync.py
Alexandru Rosca
committed

Commits on Feb 28, 2022

Changed readme file to correct tap_config.json into config.json
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-607_Change-readme-to-correct-config-name' into 'release/38'
Radu Marinoiu
committed

Commits on Mar 1, 2022

[ECDDC-601] Add code scanning with sonar
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-601_Add-code-scanning-with-Sonar' into 'release/38'
Radu Marinoiu
committed
Moved unit tests to outside of the TapTester tests folder
Radu Marinoiu
committed
Fixed some paths, included catalogs
Radu Marinoiu
committed
Fixed Branches and Centres inherited processor class
Alexandru Rosca
committed
Merge branch 'hotfix/ECDDC-625_move-unit-tests' into hotfix/ECDDC-625_move-unit-tests_release-38
Radu Marinoiu
committed
Moved new unit tests (from release/38) to the new location
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-554_refactor_branches_stream' into hotfix/ECDDC-625_move-unit-tests_release-38
Radu Marinoiu
committed
Also moved branches tests to new folder
Radu Marinoiu
committed
Merge branch 'release/38' into 'feature/ECDDC-572_snyk_code_scanning'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-554_refactor_branches_stream' into 'release/38'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-572_snyk_code_scanning' into 'release/38'
Alexandru Rosca
committed
Merge branch 'hotfix/ECDDC-625_move-unit-tests_release-38' into 'release/38'
Radu Marinoiu
committed

Commits on Mar 2, 2022

Merge branch 'master' into release-38-sync-to-master
Radu Marinoiu
committed
Merge branch 'release-38-sync-to-master' into 'release/38'
Radu Marinoiu
committed

Commits on Mar 3, 2022

Refactored deposit transactions stream
Radu Marinoiu
committed
Refactored index rate sources stream
Radu Marinoiu
committed
Added unit tests for index rate sources stream refactor
Radu Marinoiu
committed
Refactored communications stream
Alexandru Rosca
committed
Moved endpoint_body var into its own function
Alexandru Rosca
committed

Commits on Mar 8, 2022

FInished installments
Radu Marinoiu
committed
Fixed strftime, added unit tests
Radu Marinoiu
committed
Refactored groups stream
Alexandru Rosca
committed
Added test files
Alexandru Rosca
committed

Commits on Mar 9, 2022

Fix for records missing bookmark_field
Radu Marinoiu
committed
Removed redundant setting of deduplication key (same as default value)
Radu Marinoiu
committed
Removed redundant setting of deduplication key (same as default value)
Radu Marinoiu
committed
Fixed installments stream, now uses start_date instead of bookmark for dueFrom param
Radu Marinoiu
committed
Merge branch 'master' into 'release/39'
Radu Marinoiu
committed

Commits on Mar 11, 2022

Refactored loan_transactions stream
Alexandru Rosca
committed
Added unit tests
Alexandru Rosca
committed

Commits on Mar 15, 2022

Fixed unit tests for installments
Radu Marinoiu
committed
Replaced installments processor with tap processor
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-584_refactor_communications_stream' into 'release/39'
Radu Marinoiu
committed
Merge branch 'release/39' into 'feature/ECDDC-589_refactor_loan_transactions_stream'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-589_refactor_loan_transactions_stream' into 'release/39'
Radu Marinoiu
committed
Merge branch 'release/39' into feature/ECDDC-585_refactor_groups_stream
Alexandru Rosca
committed
Merge branch 'release/39' into feature/ECDDC-586_refactor-index-rate-sources-stream
Radu Marinoiu
committed
Merge branch 'release/39' into feature/ECDDC-584_refactor-deposit-transactions
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-584_refactor-deposit-transactions' into 'release/39'
Radu Marinoiu
committed
Merge branch 'release/39' into 'feature/ECDDC-586_refactor-index-rate-sources-stream'
Radu Marinoiu
committed
Merge branch 'release/39' into 'feature/ECDDC-585_refactor_groups_stream'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-586_refactor-index-rate-sources-stream' into 'release/39'
Radu Marinoiu
committed
Merge branch 'release/39' into 'feature/ECDDC-585_refactor_groups_stream'
Radu Marinoiu
committed
Fixed import
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-585_refactor_groups_stream' into 'release/39'
Radu Marinoiu
committed
Merge branch 'release/39' into feature/ECDDC-587_refactor-installments-stream
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-587_refactor-installments-stream' into 'release/39'
Radu Marinoiu
committed
Refactored tasks stream
Alexandru Rosca
committed
Merge branch 'release/39' into feature/ECDDC-590_refactor_tasks_stream
Alexandru Rosca
committed
Bugfix for streams ignoring pagination (index_rate_sources), but more generic so we can reuse it for any stream exhibiting this behaviour
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-590_refactor_tasks_stream' into 'release/39'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-586_refactor-index-rate-sources-stream' into 'release/39'
Mihai Antonio Caciuc
committed

Commits on Mar 17, 2022

Refactored users stream
Alexandru Rosca
committed
Removed duplicated list element
Alexandru Rosca
committed
Added a comma after the last list element
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-591_refactor_users_stream' into 'release/40'
Alexandru Rosca
committed

Commits on Mar 21, 2022

Solved redundancy of requirements
Radu Marinoiu
committed
[ECDDC-646] Integrate sonar into pipeline
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-646_Integrate-sonar-into-pipeline' into 'release/40'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-647_remove-redundance-of-requirements' into 'release/40'
Radu Marinoiu
committed
Added a severity threshold for Snyk dev test
Alexandru Rosca
committed
Increased severity threshold for Snyk dev test
Alexandru Rosca
committed

Commits on Mar 22, 2022

Refactored credit_arrangements stream, implemented unit tests
Radu Marinoiu
committed

Commits on Mar 23, 2022

Merge branch 'master-merge' into release/40
Radu Marinoiu
committed
Fixed typos
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-592_Refactor-credit-arrangements-stream' into 'release/40'
Radu Marinoiu
committed
Merge branch 'master-merge' into 'release/40'
Radu Marinoiu
committed
Refactored main.py to move all imports and Generator/Processor selection logic to a helper module
Radu Marinoiu
committed
Added a new stream: interest_accrual_breakdown
Alexandru Rosca
committed
Removed unused import temporarily
Radu Marinoiu
committed
Added the tests for the new stream
Alexandru Rosca
committed
Added task_link_key field to tasks stream
Alexandru Rosca
committed
New Sonar config version
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-653_implement-tests-for-fields' into 'release/40'
Radu Marinoiu
committed
Merge branch 'release/40' into feature/ECDDC-603_add_interest_accrual_breakdown_stream
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-646_Integrate-sonar-into-pipeline-fix' into 'release/40'
Radu Marinoiu
committed
Solved a merge conflict issue
Alexandru Rosca
committed
Finished implementing catalog automatic fields checker test
Radu Marinoiu
committed
Fixed the tests after merging the release branch
Alexandru Rosca
committed
Fixed issue with users deduplication key
Alexandru Rosca
committed

Commits on Mar 24, 2022

Merge branch 'feature/ECDDC-591_refactor_users_stream' into 'release/40'
Radu Marinoiu
committed
Merge branch 'release/40' into feature/ECDDC-653_implement-tests-for-fields
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-603_add_interest_accrual_breakdown_stream' into 'release/40'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-649_add_missing_tasks_fields' into 'release/40'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-652_adjust_snyk_dev_test' into 'release/40'
Alexandru Rosca
committed

Commits on Mar 25, 2022

Merge branch 'feature/ECDDC-653_implement-tests-for-fields' into 'release/40'
Radu Marinoiu
committed
Added tap tester job to ci pipeline
Alexandru Rosca
committed

Commits on Mar 28, 2022

Refactored activities stream
Radu Marinoiu
committed
Added unit tests for activities generator
Radu Marinoiu
committed
Updated tap tester job
Alexandru Rosca
committed
squash! Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Finished audit trail refactor, needs polishing and unit tests
Radu Marinoiu
committed
Finished audit trail refactor, needs polishing and unit tests
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-657_Refactor-audit-trail-stream' of gitlab.com:mambucom/product/ecosystem/mambu-marketplace/connectors/singer/tap-mambu into feature/ECDDC-657_Refactor-audit-trail-stream
Radu Marinoiu
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed
Finished refactoring of Audit Trail stream
Radu Marinoiu
committed

Commits on Mar 29, 2022

Implemented unit tests for audit trail stream
Radu Marinoiu
committed
Update test_processors.py
Radu Marinoiu
committed
Added missing fields to loan_transaction's 'affected_amounts' object
Radu Marinoiu
committed
Added all the other missing fields
Radu Marinoiu
committed
Fixed field name
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-667_Add-affected-amount-missing-fields' into 'release/40'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-658_Refactor-activities-stream' into 'release/40'
Alexandru Rosca
committed
Merge branch 'release/40' into 'feature/ECDDC-657_Refactor-audit-trail-stream'
Radu Marinoiu
committed
Updated tap tester job
Alexandru Rosca
committed
Updated tap tester job
Alexandru Rosca
committed

Commits on Mar 30, 2022

Merge branch 'feature/ECDDC-657_Refactor-audit-trail-stream' into 'release/40'
Radu Marinoiu
committed
Merge branch 'master' into 'release/40'
Radu Marinoiu
committed

Commits on Mar 31, 2022

Commits on Apr 1, 2022

Added old field back, so that the change is backwards compatible
Radu Marinoiu
committed
Finished refactoring of deposit_products stream, waiting on schema
Radu Marinoiu
committed
Merge branch 'hotfix/ECDDC-674_add-tax-on-interest-from-arrears-amount' into 'release/40'
Radu Marinoiu
committed
Removed sorting from deposit products as it's not present in the v2
Radu Marinoiu
committed
Added loan_products stream to sync.py
Radu Marinoiu
committed
Added deposit products to sync.py
Radu Marinoiu
committed
Finished refactor of custom_field_sets, waiting for schema script
Radu Marinoiu
committed

Commits on Apr 4, 2022

Added some custom exceptions to the Tap, and throwing/catching them instead of throwing generic "RuntimeException" with a custom text that we check after
Radu Marinoiu
committed

Commits on Apr 5, 2022

Merge branch 'feature/ECDDC-672_Rethink-choose-next-record' into 'release/41'
Radu Marinoiu
committed
Added work so far, need to fetch changes from release/41
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-543_Refactor-gl-accounts-stream
Radu Marinoiu
committed
Added the script that generates streams schema
Alexandru Rosca
committed

Commits on Apr 6, 2022

Final changes to gl_accounts stream
Radu Marinoiu
committed
Fixed some edge cases
Alexandru Rosca
committed
Added custom exceptions and missing stream
Alexandru Rosca
committed
Updated loan_products schema
Radu Marinoiu
committed
Updated deposit_products schema
Radu Marinoiu
committed
Removed problematic child stream
Alexandru Rosca
committed

Commits on Apr 7, 2022

Updated custom_field_sets schema
Radu Marinoiu
committed
Updated gl_accounts schema
Radu Marinoiu
committed
Ordered alphabetically the fields and added custom_fields field to specific streams
Alexandru Rosca
committed
Reorganised the script into multiple files
Alexandru Rosca
committed
Refactored gl_journal_entries, including unit tests and schema update
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-588_Refactor-loan-products-stream
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-583_refactor-deposit-products-stream
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-582_refactor-custom-field-sets-stream
Radu Marinoiu
committed
Fixed unit test
Radu Marinoiu
committed
Fixed unit tests
Radu Marinoiu
committed

Commits on Apr 8, 2022

Finished refactor for release/41 streams, need to merge with other branches to continue
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-588_Refactor-loan-products-stream' into feature/ECDDC-656_Refactor-Tap-core-code
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-583_refactor-deposit-products-stream' into feature/ECDDC-656_Refactor-Tap-core-code
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-582_refactor-custom-field-sets-stream' into feature/ECDDC-656_Refactor-Tap-core-code
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-543_Refactor-gl-accounts-stream' into feature/ECDDC-656_Refactor-Tap-core-code
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-430_Refactor-gl-journal-entries-stream' into feature/ECDDC-656_Refactor-Tap-core-code
Radu Marinoiu
committed
Refactored imports
Radu Marinoiu
committed
Merge remote-tracking branch 'origin/feature/ECDDC-673_schema_fetcher' into feature/ECDDC-677_schema_checker
Alexandru Rosca
committed
Fixed sync isinstance (replaced with issubclass)
Radu Marinoiu
committed

Commits on Apr 11, 2022

Fixed issue with generator values
Radu Marinoiu
committed
moved custom_fields field to the properties
Alexandru Rosca
committed
Merge remote-tracking branch 'origin/feature/ECDDC-673_schema_fetcher' into feature/ECDDC-677_schema_checker
Alexandru Rosca
committed
pascal to snake convertor - added small hack for ID word
Alexandru Rosca
committed
Merge remote-tracking branch 'origin/feature/ECDDC-673_schema_fetcher' into feature/ECDDC-677_schema_checker
Alexandru Rosca
committed

Commits on Apr 12, 2022

added custom_fields field to all streams
Alexandru Rosca
committed
Merge remote-tracking branch 'origin/feature/ECDDC-673_schema_fetcher' into feature/ECDDC-677_schema_checker
Alexandru Rosca
committed
updated the schemas
Alexandru Rosca
committed

Commits on Apr 13, 2022

Merge branch 'hotfix/ECDDC-678_Fix-processor-choose-next-record' into 'release/40'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-430_Refactor-gl-journal-entries-stream' into 'release/41'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-582_refactor-custom-field-sets-stream' into 'release/41'
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-588_Refactor-loan-products-stream' into 'release/41'
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-543_Refactor-gl-accounts-stream
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-583_refactor-deposit-products-stream
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-673_schema_fetcher' into 'release/41'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-543_Refactor-gl-accounts-stream' into 'release/41'
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-583_refactor-deposit-products-stream
Radu Marinoiu
committed
Merge branch 'feature/ECDDC-583_refactor-deposit-products-stream' into 'release/41'
Radu Marinoiu
committed
Merge branch 'release/41' into feature/ECDDC-656_Refactor-Tap-core-code
Radu Marinoiu
committed
Merge branch 'release/41' into 'feature/ECDDC-677_schema_checker'
Alexandru Rosca
committed
fixed merge conflicts issue
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-677_schema_checker' into 'release/41'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-656_Refactor-Tap-core-code' into 'release/41'
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-654_tap_tester_to_ci' into 'release/41'
Alexandru Rosca
committed

Commits on Apr 14, 2022

adjusted an import to match the new structure
Alexandru Rosca
committed
renamed mambu unittests folder
Alexandru Rosca
committed
updated ci file to use the new tests dir name
Alexandru Rosca
committed
added fixture jsons
Alexandru Rosca
committed

Commits on Apr 15, 2022

Merge branch 'feature/ECDDC-656_Refactor-Tap-core-code' into 'release/41'
Alexandru Rosca
committed

Commits on Apr 18, 2022

Merge branch 'master' into master-into-release-40
Radu Marinoiu
committed
Merge branch 'master-into-release-40' into release-40-into-release-41
Radu Marinoiu
committed

Commits on Apr 19, 2022

Merge branch 'release-40-into-release-41' into 'release/41'
Radu Marinoiu
committed

Commits on Apr 20, 2022

Added new fields to loan_repayments schema
Alexandru Rosca
committed
Updated loan_repayments schema
Alexandru Rosca
committed
Added new fields to activities schema + sorted the fields
Alexandru Rosca
committed

Commits on Apr 26, 2022

Updated the schemas and fixed typos
Alexandru Rosca
committed
Added format attribute to date fields
Alexandru Rosca
committed
Merge branch 'feature/ECDDC-685_check_v1_stream_schemas' into 'release/41'
Alexandru Rosca
committed

Commits on Apr 27, 2022

Merge branch 'master' into merge_master_to_release_41
Alexandru Rosca
committed
Merge branch 'merge_master_to_release_41' into 'release/41'
Alexandru Rosca
committed

Commits on May 13, 2022

Added format date-time to interest accrual breakdown
Radu Marinoiu
committed
Merge branch 'hotfix/ECDDC-719_interest-accrual-missing-format-date-time' into 'release/41'
Radu Marinoiu
committed