forked from argoproj/argo-events
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: added kafka trigger (argoproj#567)
* fix: tests
- Loading branch information
1 parent
06be703
commit 532045c
Showing
3 changed files
with
73 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# Kafka Trigger | ||
|
||
Kafka trigger allows sensor to publish events on Kafka topic. This trigger helps source the events from outside world into your messaging queues. | ||
|
||
## Specification | ||
The Kafka trigger specification is available [here](https://github.com/argoproj/argo-events/blob/master/api/sensor.md#kafkatrigger). | ||
|
||
## Walkthrough | ||
|
||
1. Consider a scenario where you are expecting a file drop onto a Minio bucket and want to place that event | ||
on a Kafka topic. | ||
|
||
1. Set up the Minio Event Source and Gateway [here](https://argoproj.github.io/argo-events/setup/minio/). | ||
Do not create the Minio sensor, we are going to create it in next step. | ||
|
||
1. Lets create the sensor, | ||
|
||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Sensor | ||
metadata: | ||
name: minio-sensor | ||
labels: | ||
sensors.argoproj.io/sensor-controller-instanceid: argo-events | ||
spec: | ||
template: | ||
spec: | ||
containers: | ||
- name: sensor | ||
image: argoproj/sensor:v0.13.0 | ||
imagePullPolicy: Always | ||
serviceAccountName: argo-events-sa | ||
dependencies: | ||
- name: test-dep | ||
gatewayName: minio-gateway | ||
eventName: example | ||
subscription: | ||
http: | ||
port: 9300 | ||
triggers: | ||
- template: | ||
name: kafka-trigger | ||
kafka: | ||
# Kafka URL | ||
url: kafka.argo-events.svc:9092 | ||
# Name of the topic | ||
topic: minio-events | ||
# partition id | ||
partition: 0 | ||
payload: | ||
- src: | ||
dependencyName: test-dep | ||
dataKey: notification.0.s3.object.key | ||
dest: fileName | ||
- src: | ||
dependencyName: test-dep | ||
dataKey: notification.0.s3.bucket.name | ||
dest: bucket | ||
|
||
1. The Kafka message needs a body. In order to construct message based on the event data, sensor offers | ||
`payload` field as a part of the Kafka trigger. | ||
|
||
The `payload` contains the list of `src` which refers to the source event and `dest` which refers to destination key within result request payload. | ||
|
||
The `payload` declared above will generate a message body like below, | ||
|
||
{ | ||
"fileName": "hello.txt" // name/key of the object | ||
"bucket": "input" // name of the bucket | ||
} | ||
|
||
1. Drop a file called `hello.txt` onto the bucket `input` and you will receive the message on Kafka topic |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters