Skip to content

Commit

Permalink
Resolved #637.
Browse files Browse the repository at this point in the history
  • Loading branch information
j3-signalroom committed Dec 31, 2024
1 parent 1106b35 commit db4b3dd
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This project showcases the exceptional capabilities of Apache Flink, known for i
Flink App|Description
-|-
`Data Generator App`|This Flink App generates realistic flight data for the fictional airlines Sunset Air and Sky One Airlines, seamlessly integrating modern data streaming technologies. Flight events are published to dedicated Kafka topics (`airline.sunset` and `airline.skyone`), enabling real-time processing and analysis. Simultaneously, the synthetic data flows into the `apache_kickstarter.airlines.skyone_airline` and `apache_kickstarter.airlines.sunset_airline` Apache Iceberg Tables, respectively, stored in an AWS S3 bucket, providing a robust, scalable foundation for historical data analytics. The entire solution is implemented in Java, overcoming limitations in [**PyFlink**](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/python/overview/), which currently lacks a Python-based Apache Flink data generator source. This design ensures a powerful, flexible data pipeline, ideal for both real-time and batch use cases.
`Flight Consolidator App`|Imports flight data from `airline.sunset` and `airline.skyone` Kafka topics and standardizes them into a unified `airline.flight` Kafka topic, and , and the `apache_kickstarter.airlines.flight` Apache Iceberg Table. Implemented in both Java and Python.
`Flight Consolidator App`|Imports flight data from `airline.sunset` and `airline.skyone` Kafka topics and standardizes them into a unified `airline.flight` Kafka topic, and the `apache_kickstarter.airlines.flight` Apache Iceberg Table. Implemented in both Java and Python.
`Flyer Stats App`|Processes data from the `airline.flight` Kafka topic to aggregate flyer statistics into the `airline.flyer_stats` Kafka topic, and the `apache_kickstarter.airlines.flyer_stats` Apache Iceberg Table. Implemented in both Java and Python.

Created by [Wade Waldron](https://www.linkedin.com/in/wade-waldron/), Staff Software Practice Lead at [Confluent Inc.](https://www.confluent.io/), these applications are designed to handle enterprise-scale use cases to demonstrate Flink's potential for real-world applications. This will start with securely fetching Kafka Cluster and Schema Registry Cluster API keys via _**AWS Secrets Manager**_, and retrieving essential Kafka client configuration properties through the _**AWS Systems Manager Parameter**_.
Expand Down

0 comments on commit db4b3dd

Please sign in to comment.