-
Notifications
You must be signed in to change notification settings - Fork 530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[INLONG-11494][Sort] Add Kafka connector on Flink 1.18 #11501
Conversation
Sink function of this connector not tested yet. |
please add license if there are some copied codes from other project. |
Thanks for the reminder! |
Please ref to inlong/licenses/inlong-sort-connectors/LICENSE, you should add the copied files to this file. |
oh I see, sorry for the misunderstanding |
....18/sort-connectors/kafka/src/main/java/org/apache/inlong/sort/kafka/source/KafkaSource.java
Outdated
Show resolved
Hide resolved
....18/sort-connectors/kafka/src/main/java/org/apache/inlong/sort/kafka/source/KafkaSource.java
Outdated
Show resolved
Hide resolved
…mation in every copied class
Fixes #11494 [Feature][Sort] Add Kafka connector on Flink 1.18
Motivation
This PR adds a Kafka connector to
inlong-sort
for Flink 1.18 support.Modifications
pom.xml
to reflect the changes.Verifying this Change
This change is a trivial rework/code cleanup without any test coverage.
This change is already covered by existing tests, such as:
(please describe tests)
This change added tests and can be verified as follows:
Further formal testing will be introduced in #11495.
Currently, a
KafkaProducer
is used to send messages to Kafka, and these messages are consumed bysort-connectors-v1.18
, which is introduced in this PR.Since there is no verified functional sink yet, the built-in
file-system
sink is used in this test.In the example below, we can see that the messages in Kafka are successfully extracted and written to
/tmp/output_file
.Example Flink SQL:
Output:
The messages in Kafka have been successfully extracted and transported to
/tmp/output_file
. See the screenshot below for the result:Documentation