Refactored 7 more streams

[DownstreamDataTeam]

Description of change

Refactored the following streams:

• Communications
• Deposit Transactions
• Groups
• Index Rate Sources
• Installments
• Loan Transactions
• Tasks

Manual QA steps

• Compared output before & after for each stream when extracting from a sandbox environment
• Ran unit tests
• Ran tap tester

Risks

Rollback steps

• revert this branch

[cmerrick]

Hi @DownstreamDataTeam, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

[dmosorast]

Hi @DownstreamDataTeam! The automatic fields test is failing for tasks when trying to get encoded_key from the record in the processor.

The rest of the tests passed so should this field be marked as "inclusion": "automatic"? Looks like the pk is listed as ID from discovery, so I'm not quite sure what the issue is here: whether the PK should be encoded_key, if the processor is looking for the wrong field, or if encoded_key should be marked as automatic in the tasks stream.

2022-03-18 15:05:21,005Z    tap - INFO replicated 1 records from "tasks" endpoint
2022-03-18 15:05:21,005Z    tap - CRITICAL 'encoded_key'
2022-03-18 15:05:21,005Z    tap - Traceback (most recent call last):
2022-03-18 15:05:21,005Z    tap -   File "/usr/local/share/virtualenvs/tap-mambu/bin/tap-mambu", line 33, in <module>
2022-03-18 15:05:21,006Z    tap -     sys.exit(load_entry_point('tap-mambu', 'console_scripts', 'tap-mambu')())
2022-03-18 15:05:21,006Z    tap -   File "/usr/local/share/virtualenvs/tap-mambu/lib/python3.8/site-packages/singer/utils.py", line 229, in wrapped
2022-03-18 15:05:21,006Z    tap -     return fnc(*args, **kwargs)
2022-03-18 15:05:21,006Z    tap -   File "/opt/code/tap-mambu/tap_mambu/__init__.py", line 50, in main
2022-03-18 15:05:21,006Z    tap -     sync(client=client,
2022-03-18 15:05:21,006Z    tap -   File "/opt/code/tap-mambu/tap_mambu/sync.py", line 922, in sync
2022-03-18 15:05:21,006Z    tap -     total_records = sync_endpoint_refactor(
2022-03-18 15:05:21,006Z    tap -   File "/opt/code/tap-mambu/tap_mambu/tap_mambu_refactor/main.py", line 71, in sync_endpoint_refactor
2022-03-18 15:05:21,007Z    tap -     return processor.process_streams_from_generators()
2022-03-18 15:05:21,007Z    tap -   File "/opt/code/tap-mambu/tap_mambu/tap_mambu_refactor/tap_processors/processor.py", line 111, in process_streams_from_generators
2022-03-18 15:05:21,007Z    tap -     if min_record_value == self.generator_values[iterator][self.endpoint_deduplication_key]:
2022-03-18 15:05:21,007Z    tap - KeyError: 'encoded_key'

Otherwise I just have one question on this PR, seems good.

[singer-bot]

You did it @DownstreamDataTeam!

Thank you for signing the Singer Contribution License Agreement.