get_value resource exhausted

[sverhoeven]

For lisflood we have a global model with

model.get_grid_shape(0)
(1500, 3600)

To get the discharge I use

model.get_value('Discharge')

which results in an error

_Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.RESOURCE_EXHAUSTED
	details = "Received message larger than max (43200010 vs. 4194304)"
	debug_error_string = "{"created":"@1596101195.183981143","description":"Received message larger than max (43200010 vs. 4194304)","file":"src/core/ext/filters/message_size/message_size_filter.cc","file_line":174,"grpc_status":8}"
>

Using get_value_at_indices I am able to fetch 500k blocks anything more gives same error.

model.get_value_at_indices('Discharge', np.array(range(500000,1000000)))

It would be nice if the get_value() and get_value_at_indices() had streaming implementation now I have to iterate (11x) myself.

[sverhoeven]

Or the library should handle the error more elagantly by for example catching the RESOURCE_EXHAUSTED error and falling back to reconstructing the value with get_value_at_indices calls by itself.

[goord]

Yes I think we should fall back to a streaming method if the message size is exceeded.

[sverhoeven]

The list flood model has a lot of nans adding compression support can give a intermediate fix. See https://grpc.github.io/grpc/python/grpc.html?highlight=compression#grpc.insecure_channel for Python client support.

[nielsdrost]

The list flood model has a lot of nans adding compression support can give a intermediate fix. See https://grpc.github.io/grpc/python/grpc.html?highlight=compression#grpc.insecure_channel for Python client support.

I think this is probably the best option for now. Can you pass this option to grpc4bmi already in some way? Or do we need an update to grpc4bmi?

[sverhoeven]

Nevermind turning on compression results in same error:

import grpc
from grpc4bmi.bmi_grpc_client import BmiClient
model = BmiClient(grpc.insecure_channel("localhost:55555", compression=grpc.Compression.Deflate))
from grpc4bmi.reserve import reserve_values
varname = model.get_output_var_names()[0]
client = model
client.get_value(varname, reserve_values(client, varname))

[sverhoeven]

To not have to rebuild the Docker images with grpc4bmi I will implement it client side only using indices to chunk the value.

[sverhoeven]

v0.2.8 and v0.3.1 have get_value which does chunking when needed.