feat: source hydrator (#20345)

* feat(hydrator): add sourceHydrator types

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix(codegen): use kube_codegen.sh deepcopy and client gen correctly

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

deepcopy gen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): add commit-server component

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

go mod tidy

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

one test file for both implementations

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix test for linux

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix git client mock

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix git client mock

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

address comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

unit tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

lint

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix image, fix health checks, fix merge issue

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix lint issues

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

remove code that doesn't work for GHE

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

changes from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

handle project-scoped repo creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codegen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

improve docs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fixes from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* set hydrator enabled key when using hydrator manifests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix manifests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

improve docs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

handle project-scoped repo creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codegen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

improve docs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fixes from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): add sourceHydrator types

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix(codegen): use kube_codegen.sh deepcopy and client gen correctly

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

deepcopy gen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

handle project-scoped repo creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codegen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): write credentials handling + UI

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): write credentials handling + UI

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

WIP: add new APIs for write creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

write api and template api

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix time function

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix lint issues

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't enrich with read creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert tls change

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't disable buttons in UI

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

ask repo server for specific revision

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fixes

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

lint ui

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

remove unnecessary change

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix test and lint

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

lint

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

enable hydrator for e2e tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* free disk space for e2e tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't free disk space

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* free disk space for e2e tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* remove comment that breaks auth

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* try removing extra function

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* cleanup from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* fix test

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
This commit is contained in:
Michael Crenshaw
2024-12-16 16:59:09 -05:00
committed by GitHub
parent dc3f40c31e
commit 433b317c35
152 changed files with 103036 additions and 1587 deletions

View File

@@ -26,6 +26,8 @@ const (
const (
// DefaultRepoServerAddr is the gRPC address of the Argo CD repo server
DefaultRepoServerAddr = "argocd-repo-server:8081"
// DefaultCommitServerAddr is the gRPC address of the Argo CD commit server
DefaultCommitServerAddr = "argocd-commit-server:8086"
// DefaultDexServerAddr is the HTTP address of the Dex OIDC server, which we run a reverse proxy against
DefaultDexServerAddr = "argocd-dex-server:5556"
// DefaultRedisAddr is the default redis address
@@ -62,15 +64,19 @@ const (
DefaultPortArgoCDMetrics = 8082
DefaultPortArgoCDAPIServerMetrics = 8083
DefaultPortRepoServerMetrics = 8084
DefaultPortCommitServer = 8086
DefaultPortCommitServerMetrics = 8087
)
// DefaultAddressAPIServer for ArgoCD components
const (
DefaultAddressAdminDashboard = "localhost"
DefaultAddressAPIServer = "0.0.0.0"
DefaultAddressAPIServerMetrics = "0.0.0.0"
DefaultAddressRepoServer = "0.0.0.0"
DefaultAddressRepoServerMetrics = "0.0.0.0"
DefaultAddressAdminDashboard = "localhost"
DefaultAddressAPIServer = "0.0.0.0"
DefaultAddressAPIServerMetrics = "0.0.0.0"
DefaultAddressRepoServer = "0.0.0.0"
DefaultAddressRepoServerMetrics = "0.0.0.0"
DefaultAddressCommitServer = "0.0.0.0"
DefaultAddressCommitServerMetrics = "0.0.0.0"
)
// Default paths on the pod's file system
@@ -175,6 +181,8 @@ const (
LabelValueSecretTypeRepository = "repository"
// LabelValueSecretTypeRepoCreds indicates a secret type of repository credentials
LabelValueSecretTypeRepoCreds = "repo-creds"
// LabelValueSecretTypeRepositoryWrite indicates a secret type of repository credentials for writing
LabelValueSecretTypeRepositoryWrite = "repository-write"
// LabelValueSecretTypeSCMCreds indicates a secret type of SCM credentials
LabelValueSecretTypeSCMCreds = "scm-creds"