feat: dynamic rebalancing of clusters across shards (#15036)

* Migrate Application Controller from Statefulset to Deployment

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add sharding deployment logic

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Update sharding logic and add comments

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add heartbeat as an environment variable

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add retry logic, heartbeat timeout environment variable

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* use the logic of pre-specified shard number on application controller pod

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* fix manifests

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* fix lint and e2e tests

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* comment out failing e2e test

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* increase readiness probe interval period

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* "comment out readiness probe to see if e2e tests succeed"

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* revert commented readiness probe

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* revert commented test case

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* read environment variable for application controller deployment name

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add nil check on replica count for deployment of application controller

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Address comments

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add Informer, Update documentation, add unit tests

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* update godoc

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* remove unwanted code and logs

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add more documentation

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* revert ApplicationController manifest to StatefulSet

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* reverting updated docs

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add documentation for the new dynamic distribution feature

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* update documentation

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add an overlay for application controller deployment and update documentation

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* fix nit

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Marking the feature as alpha

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* Add feature status link

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* revert go,mod changes

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

* update docs to avoid focusing on StatefulSet/Deployment (#26)

* update docs to avoid focusing on StatefulSet/Deployment

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* minor update to the doc

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>

---------

Signed-off-by: ishitasequeira <ishiseq29@gmail.com>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

common/common.go

@@ -12,6 +12,11 @@ import (
 	"google.golang.org/grpc/status"
 )
 
+// Component names
+const (
+	ApplicationController = "argocd-application-controller"
+)
+
 // Default service addresses and URLS of Argo CD internal services
 const (
 	// DefaultRepoServerAddr is the gRPC address of the Argo CD repo server
@@ -34,6 +39,8 @@ const (
 	// ArgoCDTLSCertsConfigMapName contains TLS certificate data for connecting repositories. Will get mounted as volume to pods
 	ArgoCDTLSCertsConfigMapName = "argocd-tls-certs-cm"
 	ArgoCDGPGKeysConfigMapName  = "argocd-gpg-keys-cm"
+	// ArgoCDAppControllerShardConfigMapName contains the application controller to shard mapping
+	ArgoCDAppControllerShardConfigMapName = "argocd-app-controller-shard-cm"
 )
 
 // Some default configurables
@@ -109,6 +116,8 @@ const (
 	// RoundRobinShardingAlgorithm is a flag value that can be opted for Sharding Algorithm it uses an equal distribution accross all shards
 	RoundRobinShardingAlgorithm = "round-robin"
 	DefaultShardingAlgorithm    = LegacyShardingAlgorithm
+	// AppControllerHeartbeatUpdateRetryCount is the retry count for updating the Shard Mapping to the Shard Mapping ConfigMap used by Application Controller
+	AppControllerHeartbeatUpdateRetryCount = 3
 )
 
 // Dex related constants
@@ -209,6 +218,8 @@ const (
 	EnvPauseGenerationRequests = "ARGOCD_PAUSE_GEN_REQUESTS"
 	// EnvControllerReplicas is the number of controller replicas
 	EnvControllerReplicas = "ARGOCD_CONTROLLER_REPLICAS"
+	// EnvControllerHeartbeatTime will update the heartbeat for application controller to claim shard
+	EnvControllerHeartbeatTime = "ARGOCD_CONTROLLER_HEARTBEAT_TIME"
 	// EnvControllerShard is the shard number that should be handled by controller
 	EnvControllerShard = "ARGOCD_CONTROLLER_SHARD"
 	// EnvControllerShardingAlgorithm is the distribution sharding algorithm to be used: legacy or round-robin