feat(agent): configure retry strategy

[schoren]

This PR configures the gRPC client retry strategy to allow better handling of connection issues. In the case of connection interruption between the server and the agent for whatever reason (server restarted, networking issues, etc), the client will retry the operation using an exponential backoff strategy. With the currently configured values, this is the retry timeline:

Attempt 1 at 0.1 seconds, Backoff Time: 0.1 seconds
Attempt 2 at 0.3 seconds, Backoff Time: 0.2 seconds
Attempt 3 at 0.7 seconds, Backoff Time: 0.4 seconds
Attempt 4 at 1.5 seconds, Backoff Time: 0.8 seconds
Attempt 5 at 3.1 seconds, Backoff Time: 1.6 seconds
Attempt 6 at 6.3 seconds, Backoff Time: 3.2 seconds
Attempt 7 at 11.3 seconds, Backoff Time: 5.0 seconds
Attempt 8 at 16.3 seconds, Backoff Time: 5.0 seconds
Attempt 9 at 21.3 seconds, Backoff Time: 5.0 seconds
Attempt 10 at 26.3 seconds, Backoff Time: 5.0 seconds
Attempt 11 at 31.3 seconds, Backoff Time: 5.0 seconds
Attempt 12 at 36.3 seconds, Backoff Time: 5.0 seconds
Attempt 13 at 41.3 seconds, Backoff Time: 5.0 seconds
Attempt 14 at 46.3 seconds, Backoff Time: 5.0 seconds
Attempt 15 at 51.3 seconds, Backoff Time: 5.0 seconds
Attempt 16 at 56.3 seconds, Backoff Time: 5.0 seconds
Attempt 17 at 61.3 seconds, Backoff Time: 5.0 seconds
Attempt 18 at 66.3 seconds, Backoff Time: 5.0 seconds
Attempt 19 at 71.3 seconds, Backoff Time: 5.0 seconds
Attempt 20 at 76.3 seconds, Backoff Time: 5.0 seconds
Attempt 21 at 81.3 seconds, Backoff Time: 5.0 seconds
Attempt 22 at 86.3 seconds, Backoff Time: 5.0 seconds
Attempt 23 at 91.3 seconds, Backoff Time: 5.0 seconds
Attempt 24 at 96.3 seconds, Backoff Time: 5.0 seconds
Attempt 25 at 101.3 seconds, Backoff Time: 5.0 seconds
Attempt 26 at 106.3 seconds, Backoff Time: 5.0 seconds
Attempt 27 at 111.3 seconds, Backoff Time: 5.0 seconds
Attempt 28 at 116.3 seconds, Backoff Time: 5.0 seconds
Attempt 29 at 121.3 seconds, Backoff Time: 5.0 seconds

If  the operation was not succesful after the last attempt, it will fail, causing the agent to exit with an erro message.

This behaviour is specially useful on docker/k8s environments, since it kills the container and makes the orchestrator restart it, allowing the process to be restarted until a succesful connection is established

[mathnogueira]

why are you renaming it to c? why not leave it as client?

[schoren]

this was leftover from a previous attempt, but using client as the name overrides the client package import, and it's not a great practice

[mathnogueira]

I would be ok with c if it was an anchor variable. But as we use it for some config, I would like a more meaningful name like grpcClient

[danielbdias]

What if we use a verbose name that specifies what this client is? Something like tracetestClient or controlPlaneClient?

[schoren]

updated to controlPlaneClient, feels like the most meaningful to me