Allow partial updates in the API

[hariso]

Feature description

Currently, our APIs mostly allow full updates only in the sense that, if certain properties are mutable, then the update request updates all of them. One example is the UpdateProcessorRequest:

message Config {
  map<string, string> settings = 1;
  int32 workers = 2;
}

message UpdateProcessorRequest {
  string id = 1;
  Processor.Config config = 2;
}

In the processor service, the whole config is then replaced:

instance.Config = cfg

Link

This implies that, if a client wants to update just the workers (for example), it first needs to first fetch the existing settings, set the new workers count, and then send the update request. We have similar examples in other APIs (pipelines, connectors).

To make the API more convenient to use, it would be good to allow partial updates that allow only certain parts of a configuration to be updated.

[lovromazgon]

I'll add some more info for when we decide to tackle this.

Currently the fields in our proto API definition are not marked with the optional keyword, meaning that field presence is not tracked (relevant proto docs). This means that we don't know if a field was explicitly set to the zero value or if it just wasn't supplied, so we can't know if we should save it unless we change the proto definition. If we ignore zero values, it would mean that the user can't delete an optional value (e.g. delete the description of a pipeline).

We might be able to do it for connector and processor configs without changing the proto definition though, since the config is supplied as a map and we can simply merge the map into the existing config map. If a key in the input map is set, whether to an actual value or an empty string, it is updated, otherwise it does not change.