Add support for Logs config API

[tt810]

Support for Logs public config API pipeline and pipelinelist. Pipelinelist has read and update functions. When a 'pipeline' is created, it's automatically included in the pipelinelist, thus there is no creation or deletion for pipelinelist.

[ajacquemot]

This method is semi generic, semi specific, e.g. we have generic types with specific argument names, can we make it fully generic or fully specific please ? Also why do we need to marshal unmarshal and not just cast ?

[gaetan-deputier]

I tend to agree that the 3 steps (json marshal/unmarshal and all marshal) seems heavy. Isn't there another way to manage it?

[tt810]

1. I will change the method name to be more specific.
2. Also why do we need to marshal unmarshal and not just cast? -> A simple cast will create an extra layer for Definition which does not exist in JSON format from config API. There are a few ways to do this step, this is the most concise one. As in terms of efficiency, the extra step of conversion is negligible compare to the API calls. After discussing with @bkabrda, I decide to go with the way "marshal/unmarshal and marshal again".

[tt810]

hmm. I retried the cast in an explicit method:

func buildProcessor(definition interface{}) map[string]interface{} {
	return definition.(map[string]interface{})
}

MarshalJSON() {
...
case ArithmeticProcessorType:
		mapProcessor = buildProcessor(processor.Definition.(ArithmeticProcessor))
}

It doesn't complain. But I was doing the same thing directly in the MarshalJSON method:

I can't assert a specific type to a general type.

[tt810]

I updated the comment of this method to explain why we need to do this extra converting. General reference: https://stackoverflow.com/questions/23589564/function-for-converting-a-struct-to-map-in-golang/42849112#42849112

[tt810]

Step back to see other approaches for the struct of Pipeline and Processor. We can:
define

type LogsPipeline struct {
	Id         *string              `json:"id,omitempty"`
	...
	Processors []map[string]interface{}      `json:"processors,omitempty"`
}

Or

type LogsPipeline struct {
	Id         *string              `json:"id,omitempty"`
	...
	Processors []interface{}      `json:"processors,omitempty"`
}

Both will lose the types/structures for processors. It omits the custom serialization/deserialization part (also means I might need to take care of special cases (e.g. nil/null conversion). But it adds weaker conversion on provider side(resource <-> struct). I don't see it better than the current version besides no need to do the marshal/unmarshal/marshal for Definition.
Another thing is current supported resources have their own types (widget in dashboard has similar definition). The current approach is compatible with the rest of resources.

[gaetan-deputier]

I tend to agree that the 3 steps (json marshal/unmarshal and all marshal) seems heavy. Isn't there another way to manage it?

[bkabrda]

This PR looks awesome, thanks for submitting it! I posted some questions/comments inline and I did thumbsup on some already existing comments so that some minor fixes are done. But overall this code looks really nice, so we should be able to merge it with only couple minor adjustments that were requested.

[bkabrda]

LGTM, merging. Thanks for the awesome PR!