Skip to content
This repository has been archived by the owner on Apr 9, 2021. It is now read-only.

Fixed reading incomplete event JSON strings from the stream #97

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ihor
Copy link

@ihor ihor commented Jan 29, 2021

Overview

The event handler code that processes the stream input expects chunks to be full event strings (from the beginning of the string until the end of line character), which is not always true.

outputStream._write = async (chunk, encoding, done) => {
    chunk.toString().split("\n")
        .filter(str => str.startsWith('data'))

Because of that, sometimes incomplete JSON strings are read from the stream, which results in the following errors:

Error while processing an event.
Event: data:{"DeployProcessed":{"deploy_hash":"af4f25e8236afecacf24ec45751e6de5af06682280cbf3d3bc6adea9749fb185","account":"018267d68f8d249b1430551ecc7b4c176d66f2ba2bf98d5547e7c3accc99375e53","timestamp":"2021-01-28T12:49:18.335Z","ttl":"1h","dependencies":[],"block_hash":"f0462bfeec91c520b7f139311037d703119053a935d88cb969027bfde22c72e0","execution_result":{"Success":{"effect":{"operations":[{"key":"deploy-af4f25e8236afecacf24ec45751e6de5af06682280cbf3d3bc6adea9749fb185","kind":"Write"},{"key":"hash-16f7f0d0899fc928adc13b1f5e4f857a03729e09ce9b651ce24d175c6d06ad99","kind":"Read"},{"key":"uref-36db58a395b29be22b6760ad56a1e3c7834988c9c60a5e74677dd65c21fceeaf-000","kind":"Write"},{"key":"account-hash-f5c8e69a44360b63a0d2f4bbe46648c05cb935417faa8b2e67deae29f6f2eae4","kind":"Write"},{"key":"hash-1bf3224baf001c6410f5cd6820924114d40be93cbb912432001cac9f62c0d0ce","kind":"Read"},{"key":"hash-ac387ec1e25673f8dd79e221a79e4d9e2265d2fc2f52d8552614f559df0810b9","kind":"Read"},{"key":"uref-650196897547fe641f16f9e1bcd9d497c96d6b203e4b9a587b62fbf2f4587706-000","kind":"Write"},{"key":"transfer-7b3ec69713c3ec591a6aaca27bc7e901d3a8aa3474eef47a57500422fd5db34d","kind":"Write"},{"key":"uref-3bf3ef523f2b252abcd4cefcc9d96bd92c140c6b8901d7336884801585d3372d-000","kind":"Add"},{"key":"hash-be8d196e566c19405a948944b2063ee575f363d6c0525ce65715d9a07bd73446","kind":"Read"},{"key":"uref-4d5c3ab2d09c09e76130d70cb9cd940eb6a37c16b2ac3100c6850914533c02b9-000","
Error: SyntaxError: Unexpected end of JSON input

Error while processing an event.
Event: data:{"FinalitySignature":{"block_hash":"5d586c6985567f897598148639c8413ea2f37b8173bcccd264629fb5757f669e","era_id":22,"signature":"01fc98d03c28d1d2c8bc849cd7c0822485050ed91ab88242eebe26797746cdedef3349923b6d53890ec9751163fb34de3c48ece283bc80062399cfc52991c1600a","public_key":"018b15761be0c5271
Error: SyntaxError: Unexpected end of JSON input

In this PR, I wrapped the original stream into the readline interface to ensure only full lines are passed to the consumer.

Which JIRA ticket does this PR relate to?

Add the link here. Create a ticket and link it here if one does not exist.

Complete this checklist before you submit this PR

  • This PR contains no more than 200 lines of code, excluding test code.
  • This PR meets CasperLabs coding standards. (there are no Javascript standard in the linked doc)
  • If this PR adds a new feature, it includes tests related to this feature.
  • You assigned one person to review this PR. (it doesn't let me)
  • Your GitHub account is linked with our Drone CI system. This is necessary to run tests on this PR. (can't do that because I'm not the organization member)
  • Do not forget to run bors r+ if GitHub policy is not enforced, e.g. when merging into another feature branch. It may be omitted under some circumstances if this PR intentionally assumes that integration tests will fail but will be fixed with the future PRs.

Notes

I tried to preserve the original code structure. Not sure if it was needed.

Copy link
Contributor

@George-cl George-cl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants