Health update from Nodes on Startup/Shutdown (CI/CD Deployment, Zero downtime)

[MichaelPeter]

Hello, I am developing a greenfield project and I am new to reverse proxies so excuse me if I didn't see a feature or if there are better ways to solve this with Reverse Proxies.

We have an On-Premise application and we'd like to keep it Zero Downtime especially when updating the nodes.

So when our TFS 2018 runs its CI deployment it would work like this:
Shutdown Node 1
Update Node 1 files
Startup Node 1
Health check Node 1
Shutdown Node 2
Update Node 2 files
Startup Node 2

Same with Nodes 3-N... Maybe even parallelized update.
So there is always at least one active node.

Now when shutting down we could let the Reverse Proxy run into a timeout for Node 1, but I think it would be preferable when the Node 1 service is shut down it tells the reverse proxy it is not available anymore. Same when Node 1 starts up it tells the reverse proxy it is available again.
At the same time when the reverse proxy would need restart it checks all nodes for their health.

In this scenario it would be required if a new node is added/removed they inform the reverse proxy. In a configuration the reverse proxy would need an restart when a node is added or removed.

I did not see any option to configure this yet, is there a solution for that? Or a buzzword?

[Tratcher]

The scenario makes sense, though direct communication between the nodes and the proxy requires fairly tight integration. Many apps don't have a direct line of communication to their proxy.

An alternative would be for this procedure to be managed by a central orchestrator. The orchestrator in this case is the one doing the deployments and telling the nodes to shut down. It could remove those nodes from the proxy config prior to shutting them down. When the deployment is complete it could re-add the nodes to the config.

It may also be wise to use two pools of nodes in this scenario to separate the versions of the software running.

1. Remove some nodes from pool1
2. Shut down those nodes
3. Upgrade and restart them
4. Add those nodes to pool2
5. Gradually transition a percentage of traffic from pool1 to pool2 and check for errors.
   Repeat until all nodes have been upgraded and moved.

How you add and remove nodes would depend on how you manage your configuration. The other mechanism we'd need to work on to enable this is the percentage based routing. #126 would cover that.

[samsp-msft]

We think the extensibility for being able to modify the config on the fly should cover this scenario, or feeding into the health state for the backend. This needs a write up for the different mechanisms that could be used in this case.

[AlwaysHC]

Please add the possibility to enable or disable backends by code. Then it will be easy to write code to integrate YARP with CI/CD systems

[MichaelPeter]

Yes beeing able to change the routes during runtime by code would solve my problem :)

[Tratcher]

Reloadable code-based config providers are covered here: https://microsoft.github.io/reverse-proxy/articles/configproviders.html.

[karelz]

Triage: This is more part of Orchestration. We do not think the code or docs belong to YARP itself. We can help guide of course -- for example, writing advanced ActiveHealthChecks.