Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Avoid writing (IPC) record batch into memory #928

Open
jorgecarleitao opened this issue Mar 27, 2022 · 1 comment
Open

Avoid writing (IPC) record batch into memory #928

jorgecarleitao opened this issue Mar 27, 2022 · 1 comment
Labels
investigation Issues or PRs that are investigations. Prs may or may not be merged.

Comments

@jorgecarleitao
Copy link
Owner

Currently the IPC writer prepares a record batch in memory before writing it into the writer.

The goal of this issue is to make so that the record batch is written directly to the writer, thereby avoiding an intermediary buffer.
@jorgecarleitao jorgecarleitao added the enhancement An improvement to an existing feature label Mar 27, 2022
@jorgecarleitao jorgecarleitao mentioned this issue Mar 27, 2022
Open
@jorgecarleitao jorgecarleitao added investigation Issues or PRs that are investigations. Prs may or may not be merged. and removed enhancement An improvement to an existing feature labels Apr 22, 2022
@ghuls ghuls mentioned this issue Jun 1, 2022
Closed
@ghuls
Copy link
Contributor

ghuls commented Oct 21, 2022

Would the memory mapped IPC reader code help to avoid this intermediary buffer?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
investigation Issues or PRs that are investigations. Prs may or may not be merged.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ghuls @jorgecarleitao