Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][connector] Kinesis sink: fix NPE with KeyValue schema and no value #17959

Merged
merged 1 commit into from
Oct 7, 2022

Conversation

nicoloboschi
Copy link
Contributor

Motivation

In the Kinesis sink with FULL_MESSAGE_IN_JSON_EXPAND_VALUE, a message with KeyValue schema and with value=NULL you get this NPE

2022-10-07T07:27:33,698+0000 [public/default/kinesis-sink-0] INFO  org.apache.pulsar.functions.instance.JavaInstanceRunnable - Encountered exception in sink write: 
java.lang.NullPointerException: null
	at org.apache.pulsar.io.kinesis.Utils.toJsonSerializable(Utils.java:250) ~[HsqgvoaXuefCgCSXYBQtdA/:?]
	at org.apache.pulsar.io.kinesis.Utils.serializeRecordToJsonExpandingValue(Utils.java:222) ~[HsqgvoaXuefCgCSXYBQtdA/:?]
	at org.apache.pulsar.io.kinesis.KinesisSink.createKinesisMessage(KinesisSink.java:308) ~[HsqgvoaXuefCgCSXYBQtdA/:?]
	at org.apache.pulsar.io.kinesis.KinesisSink.write(KinesisSink.java:135) ~[HsqgvoaXuefCgCSXYBQtdA/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:424) ~[?:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.handleResult(JavaInstanceRunnable.java:394) ~[?:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:335) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
2022-10-07T07:27:33,712+0000 [public/default/kinesis-sink-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [public/default/kinesis-sink:0] Uncaught exception in Java Instance
java.lang.RuntimeException: Failed to process message: 31:4:-1:0
	at org.apache.pulsar.functions.source.PulsarSource.lambda$buildRecord$1(PulsarSource.java:152) ~[com.datastax.oss-pulsar-functions-instance-2.10.2.1.jar:2.10.2.1]
	at org.apache.pulsar.functions.source.PulsarRecord.fail(PulsarRecord.java:117) ~[com.datastax.oss-pulsar-functions-instance-2.10.2.1.jar:2.10.2.1]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:429) ~[com.datastax.oss-pulsar-functions-instance-2.10.2.1.jar:2.10.2.1]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.handleResult(JavaInstanceRunnable.java:394) ~[com.datastax.oss-pulsar-functions-instance-2.10.2.1.jar:2.10.2.1]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:335) ~[com.datastax.oss-pulsar-functions-instance-2.10.2.1.jar:2.10.2.1]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]

Modifications

  • Added NPE check for the KeyValue schema when serializing to JSON. If the key/value is null, the "key"/"value" entry is omitted from the final payload

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Oct 7, 2022
Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tisonkun
Copy link
Member

tisonkun commented Oct 7, 2022

/pulsarbot run-failure-checks

@nicoloboschi nicoloboschi merged commit 65fad77 into apache:master Oct 7, 2022
@nicoloboschi nicoloboschi added this to the 2.11.0 milestone Oct 7, 2022
nicoloboschi added a commit that referenced this pull request Oct 7, 2022
nicoloboschi added a commit to datastax/pulsar that referenced this pull request Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs ready-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants