Arc: configuration settings for GA

[gracewehner]

PR Description

• Remove Arc proxy cluster deployment for now until it's added back so our pipelines succeed
• Add in configuration-settings for Arc:
  • Use the Arc cluster cloud setting unless the customer has overridden it. Out-of-the-box, mooncake and fairfax should just work with this. But we still want an override parameter because there used to be an issue with Azure HCI clusters where this wasn't set correctly for public clouds. This was fixed, but we still want it to be backwards compatible and be able to address any similar issues in the future where the value from the Arc cluster itself is wrong.
  • Do the same as above for the Arc cluster distribution setting. We use this to evaluate if the cluster is AKS EE and forbids mounting the ubuntu CA cert directory.
  • Add general parameters to mount or not mount the Ubuntu/Mariner CA cert directories. This is for cases like GKE where the customer may have nodes that prevent mounting the ubuntu CA cert directory. We don't want to add this for all GKE distros though because some may have Ubuntu nodes
  • Customer can use the helm value/configuration setting "DeploymentReplicas" (which is set to default to 2) to increase the number of shards if needed, similarly to how we toggled AKS clusters
• Use a helper template for Arc settings so that the boolean conditions are only evaluated once and only follow one path in the code to prevent issues from typos or future changes

New Feature Checklist

• List telemetry added about the feature:
  • Added to telegraf:
    • isarcextension = "$IS_ARC_EXTENSION"
    • arcdistribution = "$ARC_DISTRIBUTION"
    • mountmarinercerts = "$MOUNT_MARINER_CERTS"
    • mountubuntucerts = "$MOUNT_UBUNTU_CERTS"

[gracewehner]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[vishiy]

grace, isn't the only way we are making the customer control replica count is thru deploymentReplicas param for chart ? So this merge will not evaluate to true for arc ?

[gracewehner]

Right, exactly. So the logic for this in the helper template is

hpaEnabled: true
{{- if or $isArcExtension (ne .Values.AzureMonitorMetrics.CollectorHPAEnabled true) }}
    hpaEnabled: false
{{- end }}

[vishiy]

ok thanks. all these new params will be accessible thru arc cli, i am assuming (2 mount settings to enable/disable mounts and also deployment replicas). Also , the deployment replicas isn't synced to goal state by arc extension? is it also not validated? can they have it as 1 or say 21 ?