RingBuffer growing beyond expected limits

[danikov]

Driver version

2.1.0.9

Problem description

1. Expected behaviour: RingBuffer should adhere to set limits; fetchRingBufferSize is unset and should default to 1G which https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-configuration-options.html#jdbc20-enablefetchringbuffer-option implies is a 1GB limit
2. Actual behaviour: RingBuffer consumes entire heap and OOMs the JVM due to lack of bounds on large result sets, despite heap sizes well in excess of 1Gb.
3. Error message/stack trace: N/A due to OOM, but heap dump indicates a single RedshiftRowsBlockingQueue instance consuming far in excess of 1Gb of memory
4. Any other details that can be helpful: A dive into the code suggests that RedshiftRowsBlockingQueue's capacity parameter is based on the number of entries, e.g. rows, not a memory size. This suggests the documentation for this configuration is wrong and either should be changed to indicate number of rows, and/or needs additional functionality to detect and respond to overall memory usage.

[bhvkshah]

Thanks for submitting this issue, @danikov . I will take a look and get back to you when I have an update.

[bhvkshah]

@danikov could you please provide driver logs for this issue?

[danikov]

@bhvkshah sorry for not updating sooner, I understand the driver logs and other support has been provided by other channels?

[bhvkshah]

@danikov yes, we have a fix we are testing currently. I'll keep you posted on when it will be available publicly.

[bhvkshah]

Fix has been released in driver version 2.1.0.18. Thank you for your contribution to the Redshift JDBC driver!