Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set the PartitionKey when publishing records #34

Open
tomharvey opened this issue Sep 14, 2021 · 3 comments
Open

Set the PartitionKey when publishing records #34

tomharvey opened this issue Sep 14, 2021 · 3 comments

Comments

@tomharvey
Copy link

tomharvey commented Sep 14, 2021

I'm guessing that this TODO means this isn't yet implemented?

# todo: custom partition key

Any thoughts or blockers on how this might be implemented or is that open to the community?

@hampsterx
Copy link
Owner

hi sorry for late reply.

OutputItem = namedtuple("OutputItem", ["size", "n", "data"])

those are the items being fed to the producer, I guess would need to add partitionKey and then add a second arg to put() of the producer.

I can have a crack later in the week~

@hampsterx
Copy link
Owner

hampsterx commented Sep 20, 2021

just realised one issue with implementing this is most of the processors are aggregating the records into batches (eg KPLAggregator, NewLineAggregator, etc).

awslabs/amazon-kinesis-producer#130

Seems like other approach is a batch method. eg put_records(batch, key) I guess that makes more sense.

@tomharvey
Copy link
Author

It's not super important right now. Hopefully I can help out when it becomes so for me :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants