Compare commits

...

77 Commits

Author SHA1 Message Date
github-actions[bot]
d107d4e41a Bump version to 2.14.7 on release-2.14 branch (#22412)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-03-19 13:04:41 -04:00
Michael Crenshaw
39407827d3 chore(deps): bump gitops engine (#22405)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-03-19 10:51:18 -04:00
github-actions[bot]
fe2a6e91b6 Bump version to 2.14.6 on release-2.14 branch (#22393)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: rumstead <37445536+rumstead@users.noreply.github.com>
2025-03-18 06:23:50 -07:00
rumstead
38c03769af feat(server): make deep copies of objects returned by informers (#22173) (#22179) (#22340)
Signed-off-by: rumstead <37445536+rumstead@users.noreply.github.com>
2025-03-13 11:56:50 -07:00
Anand Francis Joseph
defd4be943 chore(deps): Update go-git from 5.12.0 to 5.13.2 to include several CVE fixes (#22313)
Signed-off-by: anandf <anjoseph@redhat.com>
2025-03-13 09:49:44 -04:00
github-actions[bot]
f463a945d5 Bump version to 2.14.5 on release-2.14 branch (#22292)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: ishitasequeira <46771830+ishitasequeira@users.noreply.github.com>
2025-03-10 23:11:20 -04:00
Anand Francis Joseph
ed242b9eee chore(deps): bump github.com/redis/go-redis/v9 from 9.7.0 to 9.7.1 (#21957) (#22255)
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: anandf <anjoseph@redhat.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-09 19:39:46 +02:00
github-actions[bot]
3d901f2037 Bump version to 2.14.4 on release-2.14 branch (#22176)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-03-04 15:31:29 -05:00
Michael Crenshaw
2b1e829986 chore(deps): switch gitops-engine back to release-2.14 branch (#22163)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-03-04 15:23:48 -05:00
gcp-cherry-pick-bot[bot]
52231dbc09 fix(actions): don't run empty Lua scripts (#22084) (cherry-pick #22161) (#22172)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-03-04 15:23:32 -05:00
Michael Crenshaw
2eab10a3cb chore(deps): revert accidental upgrade of go.mod packages (#22162)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-03-04 14:08:08 -05:00
gcp-cherry-pick-bot[bot]
962d7a9ad9 fix(ci): use pinned Helm version for init-release (#22164) (cherry-pick #22165) (#22171)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-03-04 13:51:52 -05:00
gcp-cherry-pick-bot[bot]
54170a4fd8 fix: make codegen permissions (cherry-pick #21667) (#22145)
Signed-off-by: Brett C. Dudo <brett@dudo.io>
Signed-off-by: Brett Dudo <brett@dudo.io>
Co-authored-by: Brett Dudo <brett@dudo.io>
Co-authored-by: Ishita Sequeira <46771830+ishitasequeira@users.noreply.github.com>
2025-03-04 09:57:17 +05:30
gcp-cherry-pick-bot[bot]
71fd4e501d fix: Check placement exists before length check (#22060) (cherry-pick #22057) (#22089)
Signed-off-by: Dale Haiducek <19750917+dhaiducek@users.noreply.github.com>
Co-authored-by: Dale Haiducek <19750917+dhaiducek@users.noreply.github.com>
2025-02-28 11:41:20 -05:00
Nitish Kumar
cb1df5d35f fix: correct lookup for the kustomization file when applying patches (cherry-pick #22024) (#22086)
Signed-off-by: nitishfy <justnitish06@gmail.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-02-28 21:36:55 +05:30
github-actions[bot]
66db4b6876 Bump version to 2.14.3 on release-2.14 branch (#22070)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-02-27 15:53:50 -05:00
Michael Crenshaw
3adb83c1df fix(hydrator): refresh by annotation instead of work queue (#22016) (#22067)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-02-27 15:30:14 -05:00
Michael Crenshaw
63edc3eb9c fix: accidental v3 imports (#22068)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-02-27 15:29:47 -05:00
gcp-cherry-pick-bot[bot]
2dd70dede8 fix(hydrator): don't use manifest-generate-paths (#22039) (cherry-pick #22015) (#22061)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-02-27 13:01:13 -05:00
Michael Crenshaw
d79185a4fe fix(hydrator): don't get cluster or API versions for hydrator (#21985) (#22038)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-02-26 15:33:09 -05:00
gcp-cherry-pick-bot[bot]
8f925c6754 fix: fetch syncedRevision in UpdateRevisionForPaths (#21014) (cherry-pick #21015) (#22011)
Signed-off-by: toyamagu2021 <toyamagu2021@gmail.com>
Signed-off-by: toyamagu-2021 <toyamagu2021@gmail.com>
Co-authored-by: gussan <83329336+toyamagu-2021@users.noreply.github.com>
2025-02-26 20:12:02 +05:30
gcp-cherry-pick-bot[bot]
b5be1df890 docs: document source hydrator maturity (cherry-pick #21969) (#21970)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-02-24 10:41:39 -05:00
gcp-cherry-pick-bot[bot]
92a3c3d727 fix: correctly set compareWith when requesting app refresh with delay (fixes #18998) (cherry-pick #21298) (#21952)
Signed-off-by: Xiaonan Shen <s@sxn.dev>
Co-authored-by: Xiaonan Shen <s@sxn.dev>
Co-authored-by: 沈啸楠 <sxn@shenxiaonandeMacBook-Pro.local>
2025-02-23 22:51:30 +05:30
gcp-cherry-pick-bot[bot]
aaed35c6d4 fix(applicationset): ApplicationSets with rolling sync stuck in Pending (cherry-pick #20230) (#21948)
Signed-off-by: Fabián Sellés <fabian.selles@adevinta.com>
Co-authored-by: Fabián Sellés Rosa <1088313+Fsero@users.noreply.github.com>
Co-authored-by: Thibault Jamet <tjamet@users.noreply.github.com>
Co-authored-by: Carlos Rejano <carlosrejanoromeu@gmail.com>
2025-02-22 15:07:42 -05:00
Nitish Kumar
2b422d2c70 chore: add cherry pick for v2.14 (#21901)
Signed-off-by: nitishfy <justnitish06@gmail.com>
2025-02-18 11:34:01 -08:00
Andrii Korotkov
896a461ae6 fix: New kube applier for server side diff dry run with refactoring (#21488) (#21819)
Signed-off-by: Andrii Korotkov <andrii.korotkov@verkada.com>
2025-02-07 17:57:27 -05:00
github-actions[bot]
ad2724661b Bump version to 2.14.2 on release-2.14 branch (#21797)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: leoluz <44989+leoluz@users.noreply.github.com>
2025-02-05 18:39:56 -05:00
gcp-cherry-pick-bot[bot]
efd9c32e25 fix: Add proxy registry key by dest server + name (cherry-pick #21791) (#21794)
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Co-authored-by: Leonardo Luz Almeida <leoluz@users.noreply.github.com>
2025-02-05 16:13:27 -05:00
github-actions[bot]
3345d05a43 Bump version to 2.14.1 on release-2.14 branch (#21758)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-02-03 16:21:20 -05:00
rumstead
4745e08d4f docs(2.14): adding basic upgrading docs for 2.14 (#21744) (#21752) 2025-02-03 21:22:47 +02:00
gcp-cherry-pick-bot[bot]
46f494592c fix(ui): Solve issue with navigating with dropdown from an application's page (cherry-pick #21737) (#21746)
Signed-off-by: Amit Oren <amit@coralogix.com>
Co-authored-by: Amit Oren <amit@coralogix.com>
2025-02-03 11:13:52 -05:00
github-actions[bot]
5964abd6af Bump version to 2.14.0-rc7 on release-2.14 branch (#21712)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: ishitasequeira <46771830+ishitasequeira@users.noreply.github.com>
2025-01-29 15:21:23 -05:00
Siddhesh Ghadi
d59c85c5eb Merge commit from fork
Signed-off-by: Siddhesh Ghadi <sghadi1203@gmail.com>
2025-01-29 13:41:18 -05:00
Alexandre Gaudreault
e4599e1a90 feat(rbac): add disable fine-grained inheritance flag (#20600) (#21553)
Signed-off-by: Matt Finkel <finkel.matt@gmail.com>
Signed-off-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Matt Finkel <finkel.matt@gmail.com>
2025-01-24 17:04:41 -05:00
Ishita Sequeira
67b2336cac chore(deps): fix bump golang.org/x/net from 0.32.0 to 0.34.0 - CVE-2024-45338 (#21628)
Signed-off-by: Ishita Sequeira <ishiseq29@gmail.com>
2025-01-22 11:08:15 -05:00
gcp-cherry-pick-bot[bot]
8a8fc37f3c fix: Policy/policy.open-cluster-management.io stuck in progressing status when no clusters match the policy (#21296) (cherry-pick #21297) (#21614)
Signed-off-by: Michele Baldessari <michele@acksyn.org>
Co-authored-by: Michele Baldessari <michele@acksyn.org>
2025-01-21 19:36:28 -05:00
github-actions[bot]
2ef67d3e5c Bump version to 2.14.0-rc6 on release-2.14 branch (#21611)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-01-21 13:38:57 -05:00
gcp-cherry-pick-bot[bot]
479b182552 fix(appset): reverted Gitlab SCM HasPath search and consider 404 errors as file not found (#16253) (cherry-pick #21597) (#21602)
Signed-off-by: Prune <prune@lecentre.net>
Co-authored-by: Prune Sebastien THOMAS <prune@lecentre.net>
2025-01-21 13:14:35 -05:00
gcp-cherry-pick-bot[bot]
bb8185e2ec docs: add mkdocs configuration stanza to .readthedocs.yaml (cherry-pick #21475) (#21608)
Signed-off-by: reggie-k <regina.voloshin@codefresh.io>
Co-authored-by: Regina Voloshin <regina.voloshin@codefresh.io>
2025-01-21 13:13:55 -05:00
gcp-cherry-pick-bot[bot]
70ea86523e fix: resolve the failing e2e appset tests for ksonnet applications (cherry-pick #21580) (#21604)
Signed-off-by: reggie-k <regina.voloshin@codefresh.io>
Co-authored-by: Regina Voloshin <regina.voloshin@codefresh.io>
2025-01-21 13:13:04 -05:00
gcp-cherry-pick-bot[bot]
35174dc196 fix(hydrator): UI nil checks (cherry-pick #21598) (#21601)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-01-21 11:56:46 -05:00
gcp-cherry-pick-bot[bot]
bab2c41e10 docs(hydrator): document signature verification limitation (cherry-pick #21504) (#21585)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-01-20 23:44:01 -05:00
Eadred
bd755104ed fix(appset): events not honouring configured namespaces (#21219) (#21241) (#21519)
* fix: 21219 Honour ARGOCD_APPLICATIONSET_CONTROLLER_NAMESPACES for all ApplicationSet events

Namespace filtering is applied to Update, Delete and Generic events.

Fixes https://github.com/argoproj/argo-cd/issues/21219



* fix: 21219 Add tests for ignoreNotAllowedNamespaces



* fix: 21219 Remove redundant package import



---------

Signed-off-by: eadred <eadred77@googlemail.com>
2025-01-17 10:59:37 -05:00
gcp-cherry-pick-bot[bot]
2bf5dc6ed1 Fix application url for custom base href (#21377) (#21516)
Signed-off-by: Amit Oren <amit@coralogix.com>
Co-authored-by: Amit Oren <amit@coralogix.com>
2025-01-16 00:46:38 -05:00
gcp-cherry-pick-bot[bot]
ebf754e3ab fix(appset): update gitlab SCM provider to search on parent folder (#16253) (#21491) (#21503)
* (fix:appset) update gitlab SCM provider to search on parent folder

fix https://github.com/argoproj/argo-cd/issues/16253



* adding test-case that replicated the new Gitlab API behaviour



* add comments to the case



---------

Signed-off-by: Prune <prune@lecentre.net>
Co-authored-by: Prune Sebastien THOMAS <prune@lecentre.net>
2025-01-15 14:59:39 -05:00
github-actions[bot]
97704acded Bump version to 2.14.0-rc5 (#21424)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-01-08 14:35:06 -05:00
gcp-cherry-pick-bot[bot]
51471b3b8b fix(controller): rename cluster batch param and add to argocd-cmd-params-cm (#21402) (#21419)
* fix(controller): rename cluster batch param and add to argocd-cmd-params-cm



* parameterize deployment too



* consistency



---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-01-08 10:27:58 -05:00
gcp-cherry-pick-bot[bot]
c13c9c1be3 fix(ci): updating action-gh-release after upstream fix (#21407) (#21408)
Signed-off-by: rumstead <37445536+rumstead@users.noreply.github.com>
Co-authored-by: rumstead <37445536+rumstead@users.noreply.github.com>
2025-01-07 16:40:42 -05:00
github-actions[bot]
a4c1bffbea Bump version to 2.14.0-rc4 (#21349)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2025-01-03 10:05:22 -05:00
gcp-cherry-pick-bot[bot]
e2eb655e41 chore: Fix data race detection failures in application tests (#21271) (#21302)
* chore: Fix race detection failures in application tests



* Fix failing TestGetCachedAppState tests



---------

Signed-off-by: eadred <eadred77@googlemail.com>
Co-authored-by: Eadred <eadred77@googlemail.com>
2024-12-23 13:05:32 -05:00
gcp-cherry-pick-bot[bot]
0a26e0f465 fix: Change applicationset generate HTTP method to avoid route conflicts (#20758) (#21299)
* Change applicationset generate HTTP method to avoid route conflicts



* Update server/applicationset/applicationset.proto




* Codegen



---------

Signed-off-by: Amit Oren <amit@coralogix.com>
Signed-off-by: Amit Oren <github@amitoren.dev>
Co-authored-by: Amit Oren <amit@coralogix.com>
Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
2024-12-23 13:04:41 -05:00
github-actions[bot]
90146498fe Bump version to 2.14.0-rc3 (#21246)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: crenshaw-dev <350466+crenshaw-dev@users.noreply.github.com>
2024-12-18 13:33:48 -05:00
gcp-cherry-pick-bot[bot]
018014c4b0 chore: Graceful shutdown for API Server (#18642) (#21224) (#21229)
* fix: Graceful shutdown for the API server (#18642) (#20981)

* fix: Graceful shutdown for the API server (#18642)

Closes #18642

Implements a graceful shutdown the the API server. Without this, ArgoCD API server will eventually return 502 during rolling update. However, healthcheck would return 503 if the server is terminating.





* Init server only once, but keep re-initializing listeners



* Check error for SetParamInSettingConfigMap as needed after fresh master



* Prevent a data race



* Remove unused variable, don't pass lock when not necessary



* Try overriding URL instead of additional URLs



* Use a more specific url



---------





* Use a custom signal for graceful restart



* Re-run tests



---------

Signed-off-by: Andrii Korotkov <andrii.korotkov@verkada.com>
Co-authored-by: Andrii Korotkov <137232734+andrii-korotkov-verkada@users.noreply.github.com>
Co-authored-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-18 15:11:03 +02:00
github-actions[bot]
a89d01266b Bump version to 2.14.0-rc2 (#21223)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: pasha-codefresh <39732895+pasha-codefresh@users.noreply.github.com>
2024-12-17 19:49:32 +02:00
gcp-cherry-pick-bot[bot]
684ee0bceb Revert "fix: Graceful shutdown for the API server (#18642) (#20981)" (#21221) (#21222) 2024-12-17 18:57:11 +02:00
github-actions[bot]
2ac03b5152 Bump version to 2.14.0-rc1 (#21218)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: pasha-codefresh <39732895+pasha-codefresh@users.noreply.github.com>
2024-12-17 17:08:19 +02:00
Michael Crenshaw
9203dd16af chore(server): simplify project validation logic (#21191)
* chore(server): simplify project validation logic

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* improve tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-17 13:01:06 +05:30
Michael Crenshaw
0de5f60cdc chore(appset): reduce dupe code w/ DB (#21192)
* chore(appset): reduce dupe code w/ DB

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* fix imports

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-17 12:57:37 +05:30
AvivGuiser
1a69663a70 docs: add link to sprig website in the template docs site (#21184)
* add link to sprig website in the template docs site

Signed-off-by: AvivGuiser <avivguiser@gmail.com>

* Update docs/operator-manual/notifications/templates.md

Co-authored-by: Nitish Kumar <justnitish06@gmail.com>
Signed-off-by: AvivGuiser <avivguiser@gmail.com>

---------

Signed-off-by: AvivGuiser <avivguiser@gmail.com>
Co-authored-by: Nitish Kumar <justnitish06@gmail.com>
2024-12-16 16:34:09 -07:00
Michael Crenshaw
433b317c35 feat: source hydrator (#20345)
* feat(hydrator): add sourceHydrator types

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix(codegen): use kube_codegen.sh deepcopy and client gen correctly

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

deepcopy gen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): add commit-server component

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

go mod tidy

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

one test file for both implementations

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix test for linux

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix git client mock

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix git client mock

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

address comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

unit tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

lint

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix image, fix health checks, fix merge issue

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix lint issues

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

remove code that doesn't work for GHE

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

changes from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

handle project-scoped repo creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codegen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

improve docs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fixes from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* set hydrator enabled key when using hydrator manifests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix manifests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

improve docs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

handle project-scoped repo creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codegen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

improve docs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fixes from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): add sourceHydrator types

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix(codegen): use kube_codegen.sh deepcopy and client gen correctly

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

deepcopy gen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

handle project-scoped repo creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codegen

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* feat(hydrator): write credentials handling + UI

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): enable controller

Co-authored-by: Alexandre Gaudreault <alexandre_gaudreault@intuit.com>
Co-authored-by: Omer Azmon <omer_azmon@intuit.com>
Co-authored-by: daengdaengLee <gunho1020@gmail.com>
Co-authored-by: Juwon Hwang (Kevin) <juwon8891@gmail.com>
Co-authored-by: thisishwan2 <feel000617@gmail.com>
Co-authored-by: mirageoasis <kimhw0820@naver.com>
Co-authored-by: Robin Lieb <robin.j.lieb@gmail.com>
Co-authored-by: miiiinju1 <gms07073@ynu.ac.kr>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

allow opt-in

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

separation between app controller and hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify diff

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

todos

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

simplify

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add dry sha to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

add app name to logs

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

more logging, no caching

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix cluster install

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't interrupt an ongoing hydrate operation

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert hydrate loop fix

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

feat(hydrator): write credentials handling + UI

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

WIP: add new APIs for write creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

write api and template api

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix time function

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix lint issues

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't enrich with read creds

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

revert tls change

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't disable buttons in UI

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

ask repo server for specific revision

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fixes

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

lint ui

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

remove unnecessary change

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

fix test and lint

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

lint

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

enable hydrator for e2e tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* free disk space for e2e tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

don't free disk space

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* free disk space for e2e tests

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* remove comment that breaks auth

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* try removing extra function

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* cleanup from comments

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* fix test

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-16 16:59:09 -05:00
Mykola Pelekh
dc3f40c31e fix: avoid resources lock contention (#8172) (#20329)
* fix: avoid resources lock contention

Signed-off-by: Mykola Pelekh <mpelekh@demonware.net>

* feat: allow enabling batch events processing

Signed-off-by: Mykola Pelekh <mpelekh@demonware.net>

* fix: update ParseDurationFromEnv to handle duration in ms

Signed-off-by: Mykola Pelekh <mpelekh@demonware.net>

* feat: make eventProcessingInterval option configurable (default is 0.1s)

Signed-off-by: Mykola Pelekh <mpelekh@demonware.net>

* use upstream gitops-engine

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Mykola Pelekh <mpelekh@demonware.net>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-16 12:18:11 -05:00
Suraj yadav
c090f849b0 pruned-icon-changed-to-trash (#21088)
Signed-off-by: Surajyadav <harrypotter1108@gmail.com>
2024-12-16 16:59:31 +05:30
Suraj yadav
a94a07ecd6 feat(ui): Added title label for filters (#21149)
* added-filter-title

Signed-off-by: Surajyadav <harrypotter1108@gmail.com>

* text-color

Signed-off-by: Surajyadav <harrypotter1108@gmail.com>

---------

Signed-off-by: Surajyadav <harrypotter1108@gmail.com>
2024-12-16 16:55:43 +05:30
dependabot[bot]
065700c5e1 chore(deps): bump nanoid from 3.3.7 to 3.3.8 in /ui (#21131)
Bumps [nanoid](https://github.com/ai/nanoid) from 3.3.7 to 3.3.8.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.3.7...3.3.8)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 15:44:49 +05:30
dependabot[bot]
8d4ae26686 chore(deps): bump library/busybox in /test/e2e/multiarch-container (#21145)
Bumps library/busybox from `768e5c6` to `2919d01`.

---
updated-dependencies:
- dependency-name: library/busybox
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 15:44:15 +05:30
GuySaar8
8a9de6a8d3 fix(ui): ArgoCD history tab shows latest values in all recent releases (#13006) (#21161)
* fix(ISSUE-13006): ArgoCD history tab shows latest values in all recent releases

Signed-off-by: Guy Saar <guysaar8@gmail.com>

* chore: added org to USER.md

Signed-off-by: Guy Saar <guysaar8@gmail.com>

chore: added org to USER.md

Signed-off-by: Guy Saar <guysaar8@gmail.com>

* chore: update USER.md based on PR review

Signed-off-by: Guy Saar <guysaar8@gmail.com>

chore: added newline to USER.md

Signed-off-by: Guy Saar <guysaar8@gmail.com>

---------

Signed-off-by: Guy Saar <guysaar8@gmail.com>
2024-12-16 15:43:34 +05:30
Yusuke Abe
4d17bf3d8b docs: update sync-wave documentation (#21155)
Signed-off-by: chansuke <moonset20@gmail.com>
2024-12-16 15:41:38 +05:30
dependabot[bot]
75b0b3c8ee chore(deps): bump go.opentelemetry.io/otel/sdk from 1.32.0 to 1.33.0 (#21165)
Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) from 1.32.0 to 1.33.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.32.0...v1.33.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 15:37:55 +05:30
OpenGuidou
bce16e9daf fix(appset): Fix appset generate in --core mode for cluster gen (#21170)
Signed-off-by: OpenGuidou <guillaume.doussin@gmail.com>
2024-12-16 14:52:44 +05:30
Michael Crenshaw
e878ad5f31 chore: remove unused defaults from image workflow (#21183)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-16 14:51:00 +05:30
dependabot[bot]
19eaeb9aca chore(deps): bump github.com/Azure/kubelogin from 0.1.5 to 0.1.6 (#21193)
Bumps [github.com/Azure/kubelogin](https://github.com/Azure/kubelogin) from 0.1.5 to 0.1.6.
- [Release notes](https://github.com/Azure/kubelogin/releases)
- [Changelog](https://github.com/Azure/kubelogin/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/kubelogin/compare/v0.1.5...v0.1.6)

---
updated-dependencies:
- dependency-name: github.com/Azure/kubelogin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 14:50:09 +05:30
Michael Crenshaw
5cdb1a0a15 chore: use new fake k8s client constructor (#21186)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-16 14:49:30 +05:30
Michael Crenshaw
4471603de2 fix(api): send to closed channel in mergeLogStreams (#7006) (#21178)
* fix(api): send to closed channel in mergeLogStreams (#7006)

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* more intense test

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* even more intense

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* remove unnecessary comment

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* fix the race condition

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2024-12-15 19:44:34 -05:00
Clint Chester
99efafb55a feat: Confluent Connector Resource Health Checker - #17695 (#17697)
* Adding Synergy as a ArgoCD user

Signed-off-by: GitHub <noreply@github.com>

* Health checking Kafka Connector resources

Signed-off-by: Clint Chester <clint.chester@synergy.net.au>

* Includes Kafka Connect Task Failures

Signed-off-by: Clint Chester <clint.chester@synergy.net.au>

---------

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Clint Chester <clint.chester@synergy.net.au>
2024-12-15 15:58:30 -05:00
1102
fdf539dc6a feat: add health check for ClusterResourceSet (#20746)
Signed-off-by: nueavv <nuguni@kakao.com>
2024-12-15 15:56:34 -05:00
github-actions[bot]
22fe65b4eb [Bot] docs: Update Snyk reports (#21180)
Signed-off-by: CI <ci@argoproj.com>
Co-authored-by: CI <ci@argoproj.com>
2024-12-15 20:49:57 +00:00
Michael Crenshaw
b60d28c71a docs(proposal): manifest hydrator (#17755)
* docs(proposal): manifest hydrator

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* whitespace

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* whitespace

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* remove old references to drySources as an array

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* rename fields

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* opinions

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* document limitations

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* updates

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* updates

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* multi-source is nondeterministic

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

* Update docs/proposals/manifest-hydrator/commit-server/README.md

Co-authored-by: joe miller <joeym@joeym.net>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

---------

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
Co-authored-by: joe miller <joeym@joeym.net>
2024-12-15 15:45:10 -05:00
457 changed files with 108527 additions and 3092 deletions

View File

@@ -370,11 +370,11 @@ jobs:
path: test-results
- name: combine-go-coverage
# We generate coverage reports for all Argo CD components, but only the applicationset-controller,
# app-controller, and repo-server report contain coverage data. The other components currently don't shut down
# gracefully, so no coverage data is produced. Once those components are fixed, we can add references to their
# coverage output directories.
# app-controller, repo-server, and commit-server report contain coverage data. The other components currently
# don't shut down gracefully, so no coverage data is produced. Once those components are fixed, we can add
# references to their coverage output directories.
run: |
go tool covdata percent -i=test-results,e2e-code-coverage/applicationset-controller,e2e-code-coverage/repo-server,e2e-code-coverage/app-controller -o test-results/full-coverage.out
go tool covdata percent -i=test-results,e2e-code-coverage/applicationset-controller,e2e-code-coverage/repo-server,e2e-code-coverage/app-controller,e2e-code-coverage/commit-server -o test-results/full-coverage.out
- name: Upload code coverage information to codecov.io
uses: codecov/codecov-action@b9fd7d16f6d7d1b5d2bec1a2887e65ceed900238 # v4.6.0
with:
@@ -393,7 +393,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
uses: SonarSource/sonarqube-scan-action@1b442ee39ac3fa7c2acdd410208dcb2bcfaae6c4 # v4.1.0
uses: SonarSource/sonarqube-scan-action@bfd4e558cda28cda6b5defafb9232d191be8c203 # v4.2.1
if: env.sonar_secret != ''
test-e2e:
name: Run end-to-end tests
@@ -429,6 +429,13 @@ jobs:
GITHUB_TOKEN: ${{ secrets.E2E_TEST_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
GITLAB_TOKEN: ${{ secrets.E2E_TEST_GITLAB_TOKEN }}
steps:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be
with:
large-packages: false
docker-images: false
swap-storage: false
tool-cache: false
- name: Checkout code
uses: actions/checkout@8410ad0602e1e429cee44a835ae9f77f654a6694 # v4.0.0
- name: Setup Golang
@@ -542,4 +549,4 @@ jobs:
exit 0
else
exit 1
fi
fi

View File

@@ -17,11 +17,9 @@ on:
platforms:
required: true
type: string
default: linux/amd64
push:
required: true
type: boolean
default: false
target:
required: false
type: string

View File

@@ -195,7 +195,7 @@ jobs:
echo "hashes=$(sha256sum /tmp/sbom.tar.gz | base64 -w0)" >> "$GITHUB_OUTPUT"
- name: Upload SBOM
uses: softprops/action-gh-release@7b4da11513bf3f43f9999e90eabced41ab8bb048 # v2.2.0
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # v2.2.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:

View File

@@ -26,6 +26,13 @@ packages:
github.com/argoproj/argo-cd/v2/applicationset/utils:
interfaces:
Renderer:
github.com/argoproj/argo-cd/v2/commitserver/commit:
interfaces:
RepoClientFactory:
github.com/argoproj/argo-cd/v2/commitserver/apiclient:
interfaces:
CommitServiceClient:
Clientset:
github.com/argoproj/argo-cd/v2/controller/cache:
interfaces:
LiveStateCache:
@@ -67,3 +74,6 @@ packages:
github.com/argoproj/argo-cd/v2/pkg/apiclient/cluster:
interfaces:
ClusterServiceServer:
github.com/argoproj/argo-cd/v2/pkg/client/clientset/versioned/typed/application/v1alpha1:
interfaces:
AppProjectInterface:

View File

@@ -2,6 +2,7 @@ version: 2
formats: all
mkdocs:
fail_on_warning: false
configuration: mkdocs.yml
python:
install:
- requirements: docs/requirements.txt

View File

@@ -140,7 +140,8 @@ RUN ln -s /usr/local/bin/argocd /usr/local/bin/argocd-server && \
ln -s /usr/local/bin/argocd /usr/local/bin/argocd-dex && \
ln -s /usr/local/bin/argocd /usr/local/bin/argocd-notifications && \
ln -s /usr/local/bin/argocd /usr/local/bin/argocd-applicationset-controller && \
ln -s /usr/local/bin/argocd /usr/local/bin/argocd-k8s-auth
ln -s /usr/local/bin/argocd /usr/local/bin/argocd-k8s-auth && \
ln -s /usr/local/bin/argocd /usr/local/bin/argocd-commit-server
USER $ARGOCD_USER_ID
ENTRYPOINT ["/usr/bin/tini", "--"]

View File

@@ -472,6 +472,7 @@ start-e2e-local: mod-vendor-local dep-ui-local cli-local
mkdir -p /tmp/coverage/repo-server
mkdir -p /tmp/coverage/applicationset-controller
mkdir -p /tmp/coverage/notification
mkdir -p /tmp/coverage/commit-server
# set paths for locally managed ssh known hosts and tls certs data
ARGOCD_SSH_DATA_PATH=/tmp/argo-e2e/app/config/ssh \
ARGOCD_TLS_DATA_PATH=/tmp/argo-e2e/app/config/tls \
@@ -489,6 +490,7 @@ start-e2e-local: mod-vendor-local dep-ui-local cli-local
ARGOCD_APPLICATIONSET_CONTROLLER_TOKENREF_STRICT_MODE=true \
ARGOCD_APPLICATIONSET_CONTROLLER_ALLOWED_SCM_PROVIDERS=http://127.0.0.1:8341,http://127.0.0.1:8342,http://127.0.0.1:8343,http://127.0.0.1:8344 \
ARGOCD_E2E_TEST=true \
ARGOCD_HYDRATOR_ENABLED=true \
goreman -f $(ARGOCD_PROCFILE) start ${ARGOCD_START}
ls -lrt /tmp/coverage

View File

@@ -1,9 +1,10 @@
controller: [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "GOCOVERDIR=${ARGOCD_COVERAGE_DIR:-/tmp/coverage/app-controller} HOSTNAME=testappcontroller-1 FORCE_LOG_COLORS=1 ARGOCD_FAKE_IN_CLUSTER=true ARGOCD_TLS_DATA_PATH=${ARGOCD_TLS_DATA_PATH:-/tmp/argocd-local/tls} ARGOCD_SSH_DATA_PATH=${ARGOCD_SSH_DATA_PATH:-/tmp/argocd-local/ssh} ARGOCD_BINARY_NAME=argocd-application-controller $COMMAND --loglevel debug --redis localhost:${ARGOCD_E2E_REDIS_PORT:-6379} --repo-server localhost:${ARGOCD_E2E_REPOSERVER_PORT:-8081} --otlp-address=${ARGOCD_OTLP_ADDRESS} --application-namespaces=${ARGOCD_APPLICATION_NAMESPACES:-''} --server-side-diff-enabled=${ARGOCD_APPLICATION_CONTROLLER_SERVER_SIDE_DIFF:-'false'}"
api-server: [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "GOCOVERDIR=${ARGOCD_COVERAGE_DIR:-/tmp/coverage/api-server} FORCE_LOG_COLORS=1 ARGOCD_FAKE_IN_CLUSTER=true ARGOCD_TLS_DATA_PATH=${ARGOCD_TLS_DATA_PATH:-/tmp/argocd-local/tls} ARGOCD_SSH_DATA_PATH=${ARGOCD_SSH_DATA_PATH:-/tmp/argocd-local/ssh} ARGOCD_BINARY_NAME=argocd-server $COMMAND --loglevel debug --redis localhost:${ARGOCD_E2E_REDIS_PORT:-6379} --disable-auth=${ARGOCD_E2E_DISABLE_AUTH:-'true'} --insecure --dex-server http://localhost:${ARGOCD_E2E_DEX_PORT:-5556} --repo-server localhost:${ARGOCD_E2E_REPOSERVER_PORT:-8081} --port ${ARGOCD_E2E_APISERVER_PORT:-8080} --otlp-address=${ARGOCD_OTLP_ADDRESS} --application-namespaces=${ARGOCD_APPLICATION_NAMESPACES:-''}"
controller: [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "GOCOVERDIR=${ARGOCD_COVERAGE_DIR:-/tmp/coverage/app-controller} HOSTNAME=testappcontroller-1 FORCE_LOG_COLORS=1 ARGOCD_FAKE_IN_CLUSTER=true ARGOCD_TLS_DATA_PATH=${ARGOCD_TLS_DATA_PATH:-/tmp/argocd-local/tls} ARGOCD_SSH_DATA_PATH=${ARGOCD_SSH_DATA_PATH:-/tmp/argocd-local/ssh} ARGOCD_BINARY_NAME=argocd-application-controller $COMMAND --loglevel debug --redis localhost:${ARGOCD_E2E_REDIS_PORT:-6379} --repo-server localhost:${ARGOCD_E2E_REPOSERVER_PORT:-8081} --commit-server localhost:${ARGOCD_E2E_COMMITSERVER_PORT:-8086} --otlp-address=${ARGOCD_OTLP_ADDRESS} --application-namespaces=${ARGOCD_APPLICATION_NAMESPACES:-''} --server-side-diff-enabled=${ARGOCD_APPLICATION_CONTROLLER_SERVER_SIDE_DIFF:-'false'} --hydrator-enabled=${ARGOCD_HYDRATOR_ENABLED:='false'}"
api-server: [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "GOCOVERDIR=${ARGOCD_COVERAGE_DIR:-/tmp/coverage/api-server} FORCE_LOG_COLORS=1 ARGOCD_FAKE_IN_CLUSTER=true ARGOCD_TLS_DATA_PATH=${ARGOCD_TLS_DATA_PATH:-/tmp/argocd-local/tls} ARGOCD_SSH_DATA_PATH=${ARGOCD_SSH_DATA_PATH:-/tmp/argocd-local/ssh} ARGOCD_BINARY_NAME=argocd-server $COMMAND --loglevel debug --redis localhost:${ARGOCD_E2E_REDIS_PORT:-6379} --disable-auth=${ARGOCD_E2E_DISABLE_AUTH:-'true'} --insecure --dex-server http://localhost:${ARGOCD_E2E_DEX_PORT:-5556} --repo-server localhost:${ARGOCD_E2E_REPOSERVER_PORT:-8081} --port ${ARGOCD_E2E_APISERVER_PORT:-8080} --otlp-address=${ARGOCD_OTLP_ADDRESS} --application-namespaces=${ARGOCD_APPLICATION_NAMESPACES:-''} --hydrator-enabled=${ARGOCD_HYDRATOR_ENABLED:='false'}"
dex: sh -c "ARGOCD_BINARY_NAME=argocd-dex go run github.com/argoproj/argo-cd/v2/cmd gendexcfg -o `pwd`/dist/dex.yaml && (test -f dist/dex.yaml || { echo 'Failed to generate dex configuration'; exit 1; }) && docker run --rm -p ${ARGOCD_E2E_DEX_PORT:-5556}:${ARGOCD_E2E_DEX_PORT:-5556} -v `pwd`/dist/dex.yaml:/dex.yaml ghcr.io/dexidp/dex:$(grep "image: ghcr.io/dexidp/dex" manifests/base/dex/argocd-dex-server-deployment.yaml | cut -d':' -f3) dex serve /dex.yaml"
redis: hack/start-redis-with-password.sh
repo-server: [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "GOCOVERDIR=${ARGOCD_COVERAGE_DIR:-/tmp/coverage/repo-server} FORCE_LOG_COLORS=1 ARGOCD_FAKE_IN_CLUSTER=true ARGOCD_GNUPGHOME=${ARGOCD_GNUPGHOME:-/tmp/argocd-local/gpg/keys} ARGOCD_PLUGINSOCKFILEPATH=${ARGOCD_PLUGINSOCKFILEPATH:-./test/cmp} ARGOCD_GPG_DATA_PATH=${ARGOCD_GPG_DATA_PATH:-/tmp/argocd-local/gpg/source} ARGOCD_TLS_DATA_PATH=${ARGOCD_TLS_DATA_PATH:-/tmp/argocd-local/tls} ARGOCD_SSH_DATA_PATH=${ARGOCD_SSH_DATA_PATH:-/tmp/argocd-local/ssh} ARGOCD_BINARY_NAME=argocd-repo-server ARGOCD_GPG_ENABLED=${ARGOCD_GPG_ENABLED:-false} $COMMAND --loglevel debug --port ${ARGOCD_E2E_REPOSERVER_PORT:-8081} --redis localhost:${ARGOCD_E2E_REDIS_PORT:-6379} --otlp-address=${ARGOCD_OTLP_ADDRESS}"
cmp-server: [ "$ARGOCD_E2E_TEST" = 'true' ] && exit 0 || [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "FORCE_LOG_COLORS=1 ARGOCD_FAKE_IN_CLUSTER=true ARGOCD_BINARY_NAME=argocd-cmp-server ARGOCD_PLUGINSOCKFILEPATH=${ARGOCD_PLUGINSOCKFILEPATH:-./test/cmp} $COMMAND --config-dir-path ./test/cmp --loglevel debug --otlp-address=${ARGOCD_OTLP_ADDRESS}"
commit-server: [ "$BIN_MODE" = 'true' ] && COMMAND=./dist/argocd || COMMAND='go run ./cmd/main.go' && sh -c "GOCOVERDIR=${ARGOCD_COVERAGE_DIR:-/tmp/coverage/commit-server} FORCE_LOG_COLORS=1 ARGOCD_BINARY_NAME=argocd-commit-server $COMMAND --loglevel debug --port ${ARGOCD_E2E_COMMITSERVER_PORT:-8086}"
ui: sh -c 'cd ui && ${ARGOCD_E2E_YARN_CMD:-yarn} start'
git-server: test/fixture/testrepos/start-git.sh
helm-registry: test/fixture/testrepos/start-helm-registry.sh

View File

@@ -335,6 +335,7 @@ Currently, the following organizations are **officially** using Argo CD:
1. [Swisscom](https://www.swisscom.ch)
1. [Swissquote](https://github.com/swissquote)
1. [Syncier](https://syncier.com/)
1. [Synergy](https://synergy.net.au)
1. [Syself](https://syself.com)
1. [TableCheck](https://tablecheck.com/)
1. [Tailor Brands](https://www.tailorbrands.com)

View File

@@ -1 +1 @@
2.14.0
2.14.7

View File

@@ -525,11 +525,9 @@ func (r *ApplicationSetReconciler) getMinRequeueAfter(applicationSetInfo *argov1
}
func ignoreNotAllowedNamespaces(namespaces []string) predicate.Predicate {
return predicate.Funcs{
CreateFunc: func(e event.CreateEvent) bool {
return utils.IsNamespaceAllowed(namespaces, e.Object.GetNamespace())
},
}
return predicate.NewPredicateFuncs(func(object client.Object) bool {
return utils.IsNamespaceAllowed(namespaces, object.GetNamespace())
})
}
func appControllerIndexer(rawObj client.Object) []string {
@@ -1062,19 +1060,20 @@ func (r *ApplicationSetReconciler) updateApplicationSetApplicationStatus(ctx con
Message: "No Application status found, defaulting status to Waiting.",
Status: "Waiting",
Step: strconv.Itoa(getAppStep(app.Name, appStepMap)),
TargetRevisions: app.Status.GetRevisions(),
}
} else {
// we have an existing AppStatus
currentAppStatus = applicationSet.Status.ApplicationStatus[idx]
// upgrade any existing AppStatus that might have been set by an older argo-cd version
// note: currentAppStatus.TargetRevisions may be set to empty list earlier during migrations,
// to prevent other usage of r.Client.Status().Update to fail before reaching here.
if len(currentAppStatus.TargetRevisions) == 0 {
currentAppStatus.TargetRevisions = app.Status.GetRevisions()
if !reflect.DeepEqual(currentAppStatus.TargetRevisions, app.Status.GetRevisions()) {
currentAppStatus.Message = "Application has pending changes, setting status to Waiting."
}
}
if !reflect.DeepEqual(currentAppStatus.TargetRevisions, app.Status.GetRevisions()) {
currentAppStatus.TargetRevisions = app.Status.GetRevisions()
currentAppStatus.Status = "Waiting"
currentAppStatus.LastTransitionTime = &now
currentAppStatus.Step = strconv.Itoa(getAppStep(currentAppStatus.Application, appStepMap))
}
appOutdated := false
if progressiveSyncsRollingSyncStrategyEnabled(applicationSet) {
@@ -1087,25 +1086,15 @@ func (r *ApplicationSetReconciler) updateApplicationSetApplicationStatus(ctx con
currentAppStatus.Status = "Waiting"
currentAppStatus.Message = "Application has pending changes, setting status to Waiting."
currentAppStatus.Step = strconv.Itoa(getAppStep(currentAppStatus.Application, appStepMap))
currentAppStatus.TargetRevisions = app.Status.GetRevisions()
}
if currentAppStatus.Status == "Pending" {
if operationPhaseString == "Succeeded" {
revisions := []string{}
if len(app.Status.OperationState.SyncResult.Revisions) > 0 {
revisions = app.Status.OperationState.SyncResult.Revisions
} else if app.Status.OperationState.SyncResult.Revision != "" {
revisions = append(revisions, app.Status.OperationState.SyncResult.Revision)
}
if reflect.DeepEqual(currentAppStatus.TargetRevisions, revisions) {
logCtx.Infof("Application %v has completed a sync successfully, updating its ApplicationSet status to Progressing", app.Name)
currentAppStatus.LastTransitionTime = &now
currentAppStatus.Status = "Progressing"
currentAppStatus.Message = "Application resource completed a sync successfully, updating status from Pending to Progressing."
currentAppStatus.Step = strconv.Itoa(getAppStep(currentAppStatus.Application, appStepMap))
}
if !appOutdated && operationPhaseString == "Succeeded" {
logCtx.Infof("Application %v has completed a sync successfully, updating its ApplicationSet status to Progressing", app.Name)
currentAppStatus.LastTransitionTime = &now
currentAppStatus.Status = "Progressing"
currentAppStatus.Message = "Application resource completed a sync successfully, updating status from Pending to Progressing."
currentAppStatus.Step = strconv.Itoa(getAppStep(currentAppStatus.Application, appStepMap))
} else if operationPhaseString == "Running" || healthStatusString == "Progressing" {
logCtx.Infof("Application %v has entered Progressing status, updating its ApplicationSet status to Progressing", app.Name)
currentAppStatus.LastTransitionTime = &now

View File

@@ -4733,6 +4733,9 @@ func TestUpdateApplicationSetApplicationStatus(t *testing.T) {
Health: v1alpha1.HealthStatus{
Status: health.HealthStatusProgressing,
},
Sync: v1alpha1.SyncStatus{
Revision: "Next",
},
},
},
},
@@ -4796,7 +4799,8 @@ func TestUpdateApplicationSetApplicationStatus(t *testing.T) {
Phase: common.OperationRunning,
},
Sync: v1alpha1.SyncStatus{
Status: v1alpha1.SyncStatusCodeSynced,
Status: v1alpha1.SyncStatusCodeSynced,
Revision: "Current",
},
},
},
@@ -4861,7 +4865,8 @@ func TestUpdateApplicationSetApplicationStatus(t *testing.T) {
Phase: common.OperationSucceeded,
},
Sync: v1alpha1.SyncStatus{
Status: v1alpha1.SyncStatusCodeSynced,
Status: v1alpha1.SyncStatusCodeSynced,
Revision: "Next",
},
},
},
@@ -4926,7 +4931,8 @@ func TestUpdateApplicationSetApplicationStatus(t *testing.T) {
Phase: common.OperationSucceeded,
},
Sync: v1alpha1.SyncStatus{
Status: v1alpha1.SyncStatusCodeSynced,
Revision: "Current",
Status: v1alpha1.SyncStatusCodeSynced,
},
},
},
@@ -5165,86 +5171,6 @@ func TestUpdateApplicationSetApplicationStatus(t *testing.T) {
},
},
},
{
name: "does not progresses a pending application with a successful sync triggered by controller with invalid revision to progressing",
appSet: v1alpha1.ApplicationSet{
ObjectMeta: metav1.ObjectMeta{
Name: "name",
Namespace: "argocd",
},
Spec: v1alpha1.ApplicationSetSpec{
Strategy: &v1alpha1.ApplicationSetStrategy{
Type: "RollingSync",
RollingSync: &v1alpha1.ApplicationSetRolloutStrategy{
Steps: []v1alpha1.ApplicationSetRolloutStep{
{
MatchExpressions: []v1alpha1.ApplicationMatchExpression{},
},
{
MatchExpressions: []v1alpha1.ApplicationMatchExpression{},
},
},
},
},
},
Status: v1alpha1.ApplicationSetStatus{
ApplicationStatus: []v1alpha1.ApplicationSetApplicationStatus{
{
Application: "app1",
LastTransitionTime: &metav1.Time{
Time: time.Now().Add(time.Duration(-1) * time.Minute),
},
Message: "",
Status: "Pending",
Step: "1",
TargetRevisions: []string{"Next"},
},
},
},
},
apps: []v1alpha1.Application{
{
ObjectMeta: metav1.ObjectMeta{
Name: "app1",
},
Status: v1alpha1.ApplicationStatus{
Health: v1alpha1.HealthStatus{
Status: health.HealthStatusDegraded,
},
OperationState: &v1alpha1.OperationState{
Phase: common.OperationSucceeded,
StartedAt: metav1.Time{
Time: time.Now(),
},
Operation: v1alpha1.Operation{
InitiatedBy: v1alpha1.OperationInitiator{
Username: "applicationset-controller",
Automated: true,
},
},
SyncResult: &v1alpha1.SyncOperationResult{
Revision: "Previous",
},
},
Sync: v1alpha1.SyncStatus{
Status: v1alpha1.SyncStatusCodeSynced,
},
},
},
},
appStepMap: map[string]int{
"app1": 0,
},
expectedAppStatus: []v1alpha1.ApplicationSetApplicationStatus{
{
Application: "app1",
Message: "",
Status: "Pending",
Step: "1",
TargetRevisions: []string{"Next"},
},
},
},
{
name: "removes the appStatus for applications that no longer exist",
appSet: v1alpha1.ApplicationSet{
@@ -5299,7 +5225,77 @@ func TestUpdateApplicationSetApplicationStatus(t *testing.T) {
Phase: common.OperationSucceeded,
},
Sync: v1alpha1.SyncStatus{
Status: v1alpha1.SyncStatusCodeSynced,
Status: v1alpha1.SyncStatusCodeSynced,
Revision: "Current",
},
},
},
},
appStepMap: map[string]int{
"app1": 0,
},
expectedAppStatus: []v1alpha1.ApplicationSetApplicationStatus{
{
Application: "app1",
Message: "Application resource is already Healthy, updating status from Waiting to Healthy.",
Status: "Healthy",
Step: "1",
TargetRevisions: []string{"Current"},
},
},
},
{
name: "progresses a pending synced application with an old revision to progressing with the Current one",
appSet: v1alpha1.ApplicationSet{
ObjectMeta: metav1.ObjectMeta{
Name: "name",
Namespace: "argocd",
},
Spec: v1alpha1.ApplicationSetSpec{
Strategy: &v1alpha1.ApplicationSetStrategy{
Type: "RollingSync",
RollingSync: &v1alpha1.ApplicationSetRolloutStrategy{
Steps: []v1alpha1.ApplicationSetRolloutStep{
{
MatchExpressions: []v1alpha1.ApplicationMatchExpression{},
},
{
MatchExpressions: []v1alpha1.ApplicationMatchExpression{},
},
},
},
},
},
Status: v1alpha1.ApplicationSetStatus{
ApplicationStatus: []v1alpha1.ApplicationSetApplicationStatus{
{
Application: "app1",
Message: "",
Status: "Pending",
Step: "1",
TargetRevisions: []string{"Old"},
},
},
},
},
apps: []v1alpha1.Application{
{
ObjectMeta: metav1.ObjectMeta{
Name: "app1",
},
Status: v1alpha1.ApplicationStatus{
Health: v1alpha1.HealthStatus{
Status: health.HealthStatusHealthy,
},
OperationState: &v1alpha1.OperationState{
Phase: common.OperationSucceeded,
SyncResult: &v1alpha1.SyncOperationResult{
Revision: "Current",
},
},
Sync: v1alpha1.SyncStatus{
Status: v1alpha1.SyncStatusCodeSynced,
Revisions: []string{"Current"},
},
},
},
@@ -6657,3 +6653,86 @@ func TestMigrateStatus(t *testing.T) {
})
}
}
func TestIgnoreNotAllowedNamespaces(t *testing.T) {
tests := []struct {
name string
namespaces []string
objectNS string
expected bool
}{
{
name: "Namespace allowed",
namespaces: []string{"allowed-namespace"},
objectNS: "allowed-namespace",
expected: true,
},
{
name: "Namespace not allowed",
namespaces: []string{"allowed-namespace"},
objectNS: "not-allowed-namespace",
expected: false,
},
{
name: "Empty allowed namespaces",
namespaces: []string{},
objectNS: "any-namespace",
expected: false,
},
{
name: "Multiple allowed namespaces",
namespaces: []string{"allowed-namespace-1", "allowed-namespace-2"},
objectNS: "allowed-namespace-2",
expected: true,
},
{
name: "Namespace not in multiple allowed namespaces",
namespaces: []string{"allowed-namespace-1", "allowed-namespace-2"},
objectNS: "not-allowed-namespace",
expected: false,
},
{
name: "Namespace matched by glob pattern",
namespaces: []string{"allowed-namespace-*"},
objectNS: "allowed-namespace-1",
expected: true,
},
{
name: "Namespace matched by regex pattern",
namespaces: []string{"/^allowed-namespace-[^-]+$/"},
objectNS: "allowed-namespace-1",
expected: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
predicate := ignoreNotAllowedNamespaces(tt.namespaces)
object := &v1alpha1.ApplicationSet{
ObjectMeta: metav1.ObjectMeta{
Namespace: tt.objectNS,
},
}
t.Run(tt.name+":Create", func(t *testing.T) {
result := predicate.Create(event.CreateEvent{Object: object})
assert.Equal(t, tt.expected, result)
})
t.Run(tt.name+":Update", func(t *testing.T) {
result := predicate.Update(event.UpdateEvent{ObjectNew: object})
assert.Equal(t, tt.expected, result)
})
t.Run(tt.name+":Delete", func(t *testing.T) {
result := predicate.Delete(event.DeleteEvent{Object: object})
assert.Equal(t, tt.expected, result)
})
t.Run(tt.name+":Generic", func(t *testing.T) {
result := predicate.Generic(event.GenericEvent{Object: object})
assert.Equal(t, tt.expected, result)
})
})
}
}

View File

@@ -5,7 +5,7 @@ import (
"net/http"
"github.com/bradleyfalzon/ghinstallation/v2"
"github.com/google/go-github/v63/github"
"github.com/google/go-github/v66/github"
"github.com/argoproj/argo-cd/v2/applicationset/services/github_app_auth"
)

View File

@@ -5,7 +5,7 @@ import (
"fmt"
"os"
"github.com/google/go-github/v63/github"
"github.com/google/go-github/v66/github"
"golang.org/x/oauth2"
)

View File

@@ -3,7 +3,7 @@ package pull_request
import (
"testing"
"github.com/google/go-github/v63/github"
"github.com/google/go-github/v66/github"
"github.com/stretchr/testify/require"
)

View File

@@ -6,7 +6,7 @@ import (
"net/http"
"os"
"github.com/google/go-github/v63/github"
"github.com/google/go-github/v66/github"
"golang.org/x/oauth2"
)

View File

@@ -2,10 +2,10 @@ package scm_provider
import (
"context"
"errors"
"fmt"
"net/http"
"os"
pathpkg "path"
"github.com/hashicorp/go-retryablehttp"
"github.com/xanzy/go-gitlab"
@@ -129,40 +129,31 @@ func (g *GitlabProvider) ListRepos(ctx context.Context, cloneProtocol string) ([
func (g *GitlabProvider) RepoHasPath(_ context.Context, repo *Repository, path string) (bool, error) {
p, _, err := g.client.Projects.GetProject(repo.Organization+"/"+repo.Repository, nil)
if err != nil {
return false, err
return false, fmt.Errorf("error getting Project Info: %w", err)
}
directories := []string{
path,
pathpkg.Dir(path),
}
for _, directory := range directories {
options := gitlab.ListTreeOptions{
Path: &directory,
Ref: &repo.Branch,
}
for {
treeNode, resp, err := g.client.Repositories.ListTree(p.ID, &options)
// search if the path is a file and exists in the repo
fileOptions := gitlab.GetFileOptions{Ref: &repo.Branch}
_, _, err = g.client.RepositoryFiles.GetFile(p.ID, path, &fileOptions)
if err != nil {
if errors.Is(err, gitlab.ErrNotFound) {
// no file found, check for a directory
options := gitlab.ListTreeOptions{
Path: &path,
Ref: &repo.Branch,
}
_, _, err := g.client.Repositories.ListTree(p.ID, &options)
if err != nil {
if errors.Is(err, gitlab.ErrNotFound) {
return false, nil // no file or directory found
}
return false, err
}
if path == directory {
if resp.TotalItems > 0 {
return true, nil
}
}
for i := range treeNode {
if treeNode[i].Path == path {
return true, nil
}
}
if resp.NextPage == 0 {
// no future pages
break
}
options.Page = resp.NextPage
return true, nil // directory found
}
return false, err
}
return false, nil
return true, nil // file found
}
func (g *GitlabProvider) listBranches(_ context.Context, repo *Repository) ([]gitlab.Branch, error) {

View File

@@ -20,6 +20,7 @@ func gitlabMockHandler(t *testing.T) func(http.ResponseWriter, *http.Request) {
t.Helper()
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Println(r.RequestURI)
switch r.RequestURI {
case "/api/v4":
fmt.Println("here1")
@@ -1040,6 +1041,32 @@ func gitlabMockHandler(t *testing.T) func(http.ResponseWriter, *http.Request) {
if err != nil {
t.Fail()
}
// Recent versions of the Gitlab API (v17.7+) listTree return 404 not only when a file doesn't exist, but also
// when a path is to a file instead of a directory. Code was refactored to explicitly search for file then
// search for directory, catching 404 errors as "file not found".
case "/api/v4/projects/27084533/repository/files/argocd?ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/files/argocd%2Finstall%2Eyaml?ref=master":
_, err := io.WriteString(w, `{"file_name":"install.yaml","file_path":"argocd/install.yaml","size":0,"encoding":"base64","content_sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","ref":"main","blob_id":"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391","commit_id":"6d4c0f9d34534ccc73aa3f3180b25e2aebe630eb","last_commit_id":"b50eb63f9c0e09bfdb070db26fd32c7210291f52","execute_filemode":false,"content":""}`)
if err != nil {
t.Fail()
}
case "/api/v4/projects/27084533/repository/files/notathing?ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/tree?path=notathing&ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/files/argocd%2Fnotathing%2Eyaml?ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/tree?path=argocd%2Fnotathing.yaml&ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/files/notathing%2Fnotathing%2Eyaml?ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/tree?path=notathing%2Fnotathing.yaml&ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/files/notathing%2Fnotathing%2Fnotathing%2Eyaml?ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/tree?path=notathing%2Fnotathing%2Fnotathing.yaml&ref=master":
w.WriteHeader(http.StatusNotFound)
case "/api/v4/projects/27084533/repository/branches/foo":
w.WriteHeader(http.StatusNotFound)
default:
@@ -1194,6 +1221,16 @@ func TestGitlabHasPath(t *testing.T) {
path: "argocd/notathing.yaml",
exists: false,
},
{
name: "noexistent file in noexistent directory",
path: "notathing/notathing.yaml",
exists: false,
},
{
name: "noexistent file in nested noexistent directory",
path: "notathing/notathing/notathing.yaml",
exists: false,
},
}
for _, c := range cases {

View File

@@ -2,22 +2,15 @@ package utils
import (
"context"
"encoding/json"
"fmt"
"strconv"
"strings"
"sync"
"time"
log "github.com/sirupsen/logrus"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/argoproj/argo-cd/v2/common"
appv1 "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/util/db"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/utils/ptr"
)
// The contents of this file are from
@@ -126,11 +119,15 @@ func ListClusters(ctx context.Context, clientset kubernetes.Interface, namespace
hasInClusterCredentials := false
for i, clusterSecret := range clusterSecrets {
// This line has changed from the original Argo CD code: now receives an error, and handles it
cluster, err := secretToCluster(&clusterSecret)
cluster, err := db.SecretToCluster(&clusterSecret)
if err != nil || cluster == nil {
return nil, fmt.Errorf("unable to convert cluster secret to cluster object '%s': %w", clusterSecret.Name, err)
}
// db.SecretToCluster populates these, but they're not meant to be available to the caller.
cluster.Labels = nil
cluster.Annotations = nil
clusterList.Items[i] = *cluster
if cluster.Server == appv1.KubernetesInternalAPIServerAddr {
hasInClusterCredentials = true
@@ -167,48 +164,3 @@ func getLocalCluster(clientset kubernetes.Interface) *appv1.Cluster {
cluster.ConnectionState.ModifiedAt = &now
return cluster
}
// secretToCluster converts a secret into a Cluster object
func secretToCluster(s *corev1.Secret) (*appv1.Cluster, error) {
var config appv1.ClusterConfig
if len(s.Data["config"]) > 0 {
if err := json.Unmarshal(s.Data["config"], &config); err != nil {
// This line has changed from the original Argo CD: now returns an error rather than panicing.
return nil, err
}
}
var namespaces []string
for _, ns := range strings.Split(string(s.Data["namespaces"]), ",") {
if ns = strings.TrimSpace(ns); ns != "" {
namespaces = append(namespaces, ns)
}
}
var refreshRequestedAt *metav1.Time
if v, found := s.Annotations[appv1.AnnotationKeyRefresh]; found {
requestedAt, err := time.Parse(time.RFC3339, v)
if err != nil {
log.Warnf("Error while parsing date in cluster secret '%s': %v", s.Name, err)
} else {
refreshRequestedAt = &metav1.Time{Time: requestedAt}
}
}
var shard *int64
if shardStr := s.Data["shard"]; shardStr != nil {
if val, err := strconv.Atoi(string(shardStr)); err != nil {
log.Warnf("Error while parsing shard in cluster secret '%s': %v", s.Name, err)
} else {
shard = ptr.To(int64(val))
}
}
cluster := appv1.Cluster{
ID: string(s.UID),
Server: strings.TrimRight(string(s.Data["server"]), "/"),
Name: string(s.Data["name"]),
Namespaces: namespaces,
Config: config,
RefreshRequestedAt: refreshRequestedAt,
Shard: shard,
}
return &cluster, nil
}

View File

@@ -20,51 +20,6 @@ const (
fakeNamespace = "fake-ns"
)
// From Argo CD util/db/cluster_test.go
func Test_secretToCluster(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "mycluster",
Namespace: fakeNamespace,
},
Data: map[string][]byte{
"name": []byte("test"),
"server": []byte("http://mycluster"),
"config": []byte("{\"username\":\"foo\", \"disableCompression\":true}"),
},
}
cluster, err := secretToCluster(secret)
require.NoError(t, err)
assert.Equal(t, argoappv1.Cluster{
Name: "test",
Server: "http://mycluster",
Config: argoappv1.ClusterConfig{
Username: "foo",
DisableCompression: true,
},
}, *cluster)
}
// From Argo CD util/db/cluster_test.go
func Test_secretToCluster_NoConfig(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "mycluster",
Namespace: fakeNamespace,
},
Data: map[string][]byte{
"name": []byte("test"),
"server": []byte("http://mycluster"),
},
}
cluster, err := secretToCluster(secret)
require.NoError(t, err)
assert.Equal(t, argoappv1.Cluster{
Name: "test",
Server: "http://mycluster",
}, *cluster)
}
func createClusterSecret(secretName string, clusterName string, clusterServer string) *corev1.Secret {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{

View File

@@ -10,6 +10,7 @@ p, role:readonly, applications, get, */*, allow
p, role:readonly, certificates, get, *, allow
p, role:readonly, clusters, get, *, allow
p, role:readonly, repositories, get, *, allow
p, role:readonly, write-repositories, get, *, allow
p, role:readonly, projects, get, *, allow
p, role:readonly, accounts, get, *, allow
p, role:readonly, gpgkeys, get, *, allow
@@ -17,7 +18,9 @@ p, role:readonly, logs, get, */*, allow
p, role:admin, applications, create, */*, allow
p, role:admin, applications, update, */*, allow
p, role:admin, applications, update/*, */*, allow
p, role:admin, applications, delete, */*, allow
p, role:admin, applications, delete/*, */*, allow
p, role:admin, applications, sync, */*, allow
p, role:admin, applications, override, */*, allow
p, role:admin, applications, action/*, */*, allow
@@ -34,6 +37,9 @@ p, role:admin, clusters, delete, *, allow
p, role:admin, repositories, create, *, allow
p, role:admin, repositories, update, *, allow
p, role:admin, repositories, delete, *, allow
p, role:admin, write-repositories, create, *, allow
p, role:admin, write-repositories, update, *, allow
p, role:admin, write-repositories, delete, *, allow
p, role:admin, projects, create, *, allow
p, role:admin, projects, update, *, allow
p, role:admin, projects, delete, *, allow
@@ -43,4 +49,4 @@ p, role:admin, gpgkeys, delete, *, allow
p, role:admin, exec, create, */*, allow
g, role:admin, role:readonly
g, admin, role:admin
g, admin, role:admin
1 # Built-in policy which defines two roles: role:readonly and role:admin,
10 p, role:readonly, clusters, get, *, allow
11 p, role:readonly, repositories, get, *, allow
12 p, role:readonly, projects, get, *, allow p, role:readonly, write-repositories, get, *, allow
13 p, role:readonly, projects, get, *, allow
14 p, role:readonly, accounts, get, *, allow
15 p, role:readonly, gpgkeys, get, *, allow
16 p, role:readonly, logs, get, */*, allow
18 p, role:admin, applications, update, */*, allow
19 p, role:admin, applications, delete, */*, allow p, role:admin, applications, update/*, */*, allow
20 p, role:admin, applications, sync, */*, allow p, role:admin, applications, delete, */*, allow
21 p, role:admin, applications, delete/*, */*, allow
22 p, role:admin, applications, override, */*, allow p, role:admin, applications, sync, */*, allow
23 p, role:admin, applications, override, */*, allow
24 p, role:admin, applications, action/*, */*, allow
25 p, role:admin, applicationsets, get, */*, allow
26 p, role:admin, applicationsets, create, */*, allow
37 p, role:admin, repositories, delete, *, allow
38 p, role:admin, projects, create, *, allow p, role:admin, write-repositories, create, *, allow
39 p, role:admin, projects, update, *, allow p, role:admin, write-repositories, update, *, allow
40 p, role:admin, write-repositories, delete, *, allow
41 p, role:admin, projects, create, *, allow
42 p, role:admin, projects, update, *, allow
43 p, role:admin, projects, delete, *, allow
44 p, role:admin, accounts, update, *, allow
45 p, role:admin, gpgkeys, create, *, allow
49 g, admin, role:admin
50
51
52

657
assets/swagger.json generated
View File

@@ -1990,6 +1990,39 @@
}
}
},
"/api/v1/applicationsets/generate": {
"post": {
"tags": [
"ApplicationSetService"
],
"summary": "Generate generates",
"operationId": "ApplicationSetService_Generate",
"parameters": [
{
"name": "body",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/applicationsetApplicationSetGenerateRequest"
}
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/applicationsetApplicationSetGenerateResponse"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/applicationsets/{name}": {
"get": {
"tags": [
@@ -4084,6 +4117,504 @@
}
}
},
"/api/v1/write-repocreds": {
"get": {
"tags": [
"RepoCredsService"
],
"summary": "ListWriteRepositoryCredentials gets a list of all configured repository credential sets that have write access",
"operationId": "RepoCredsService_ListWriteRepositoryCredentials",
"parameters": [
{
"type": "string",
"description": "Repo URL for query.",
"name": "url",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1RepoCredsList"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
},
"post": {
"tags": [
"RepoCredsService"
],
"summary": "CreateWriteRepositoryCredentials creates a new repository credential set with write access",
"operationId": "RepoCredsService_CreateWriteRepositoryCredentials",
"parameters": [
{
"description": "Repository definition",
"name": "body",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/v1alpha1RepoCreds"
}
},
{
"type": "boolean",
"description": "Whether to create in upsert mode.",
"name": "upsert",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1RepoCreds"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/write-repocreds/{creds.url}": {
"put": {
"tags": [
"RepoCredsService"
],
"summary": "UpdateWriteRepositoryCredentials updates a repository credential set with write access",
"operationId": "RepoCredsService_UpdateWriteRepositoryCredentials",
"parameters": [
{
"type": "string",
"description": "URL is the URL to which these credentials match",
"name": "creds.url",
"in": "path",
"required": true
},
{
"name": "body",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/v1alpha1RepoCreds"
}
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1RepoCreds"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/write-repocreds/{url}": {
"delete": {
"tags": [
"RepoCredsService"
],
"summary": "DeleteWriteRepositoryCredentials deletes a repository credential set with write access from the configuration",
"operationId": "RepoCredsService_DeleteWriteRepositoryCredentials",
"parameters": [
{
"type": "string",
"name": "url",
"in": "path",
"required": true
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/repocredsRepoCredsResponse"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/write-repositories": {
"get": {
"tags": [
"RepositoryService"
],
"summary": "ListWriteRepositories gets a list of all configured write repositories",
"operationId": "RepositoryService_ListWriteRepositories",
"parameters": [
{
"type": "string",
"description": "Repo URL for query.",
"name": "repo",
"in": "query"
},
{
"type": "boolean",
"description": "Whether to force a cache refresh on repo's connection state.",
"name": "forceRefresh",
"in": "query"
},
{
"type": "string",
"description": "App project for query.",
"name": "appProject",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1RepositoryList"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
},
"post": {
"tags": [
"RepositoryService"
],
"summary": "CreateWriteRepository creates a new write repository configuration",
"operationId": "RepositoryService_CreateWriteRepository",
"parameters": [
{
"description": "Repository definition",
"name": "body",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/v1alpha1Repository"
}
},
{
"type": "boolean",
"description": "Whether to create in upsert mode.",
"name": "upsert",
"in": "query"
},
{
"type": "boolean",
"description": "Whether to operate on credential set instead of repository.",
"name": "credsOnly",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1Repository"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/write-repositories/{repo.repo}": {
"put": {
"tags": [
"RepositoryService"
],
"summary": "UpdateWriteRepository updates a write repository configuration",
"operationId": "RepositoryService_UpdateWriteRepository",
"parameters": [
{
"type": "string",
"description": "Repo contains the URL to the remote repository",
"name": "repo.repo",
"in": "path",
"required": true
},
{
"name": "body",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/v1alpha1Repository"
}
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1Repository"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/write-repositories/{repo}": {
"get": {
"tags": [
"RepositoryService"
],
"summary": "GetWrite returns a repository or its write credentials",
"operationId": "RepositoryService_GetWrite",
"parameters": [
{
"type": "string",
"description": "Repo URL for query",
"name": "repo",
"in": "path",
"required": true
},
{
"type": "boolean",
"description": "Whether to force a cache refresh on repo's connection state.",
"name": "forceRefresh",
"in": "query"
},
{
"type": "string",
"description": "App project for query.",
"name": "appProject",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v1alpha1Repository"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
},
"delete": {
"tags": [
"RepositoryService"
],
"summary": "DeleteWriteRepository deletes a write repository from the configuration",
"operationId": "RepositoryService_DeleteWriteRepository",
"parameters": [
{
"type": "string",
"description": "Repo URL for query",
"name": "repo",
"in": "path",
"required": true
},
{
"type": "boolean",
"description": "Whether to force a cache refresh on repo's connection state.",
"name": "forceRefresh",
"in": "query"
},
{
"type": "string",
"description": "App project for query.",
"name": "appProject",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/repositoryRepoResponse"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/v1/write-repositories/{repo}/validate": {
"post": {
"tags": [
"RepositoryService"
],
"summary": "ValidateWriteAccess validates write access to a repository with given parameters",
"operationId": "RepositoryService_ValidateWriteAccess",
"parameters": [
{
"type": "string",
"description": "The URL to the repo",
"name": "repo",
"in": "path",
"required": true
},
{
"description": "The URL to the repo",
"name": "body",
"in": "body",
"required": true,
"schema": {
"type": "string"
}
},
{
"type": "string",
"description": "Username for accessing repo.",
"name": "username",
"in": "query"
},
{
"type": "string",
"description": "Password for accessing repo.",
"name": "password",
"in": "query"
},
{
"type": "string",
"description": "Private key data for accessing SSH repository.",
"name": "sshPrivateKey",
"in": "query"
},
{
"type": "boolean",
"description": "Whether to skip certificate or host key validation.",
"name": "insecure",
"in": "query"
},
{
"type": "string",
"description": "TLS client cert data for accessing HTTPS repository.",
"name": "tlsClientCertData",
"in": "query"
},
{
"type": "string",
"description": "TLS client cert key for accessing HTTPS repository.",
"name": "tlsClientCertKey",
"in": "query"
},
{
"type": "string",
"description": "The type of the repo.",
"name": "type",
"in": "query"
},
{
"type": "string",
"description": "The name of the repo.",
"name": "name",
"in": "query"
},
{
"type": "boolean",
"description": "Whether helm-oci support should be enabled for this repo.",
"name": "enableOci",
"in": "query"
},
{
"type": "string",
"description": "Github App Private Key PEM data.",
"name": "githubAppPrivateKey",
"in": "query"
},
{
"type": "string",
"format": "int64",
"description": "Github App ID of the app used to access the repo.",
"name": "githubAppID",
"in": "query"
},
{
"type": "string",
"format": "int64",
"description": "Github App Installation ID of the installed GitHub App.",
"name": "githubAppInstallationID",
"in": "query"
},
{
"type": "string",
"description": "Github App Enterprise base url if empty will default to https://api.github.com.",
"name": "githubAppEnterpriseBaseUrl",
"in": "query"
},
{
"type": "string",
"description": "HTTP/HTTPS proxy to access the repository.",
"name": "proxy",
"in": "query"
},
{
"type": "string",
"description": "Reference between project and repository that allow you automatically to be added as item inside SourceRepos project entity.",
"name": "project",
"in": "query"
},
{
"type": "string",
"description": "Google Cloud Platform service account key.",
"name": "gcpServiceAccountKey",
"in": "query"
},
{
"type": "boolean",
"description": "Whether to force HTTP basic auth.",
"name": "forceHttpBasicAuth",
"in": "query"
}
],
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/repositoryRepoResponse"
}
},
"default": {
"description": "An unexpected error response.",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
}
}
},
"/api/version": {
"get": {
"tags": [
@@ -4725,6 +5256,9 @@
"help": {
"$ref": "#/definitions/clusterHelp"
},
"hydratorEnabled": {
"type": "boolean"
},
"impersonationEnabled": {
"type": "boolean"
},
@@ -6857,6 +7391,9 @@
"source": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
},
"sourceHydrator": {
"$ref": "#/definitions/v1alpha1SourceHydrator"
},
"sources": {
"type": "array",
"title": "Sources is a reference to the location of the application's manifests or chart",
@@ -6914,6 +7451,9 @@
"$ref": "#/definitions/applicationv1alpha1ResourceStatus"
}
},
"sourceHydrator": {
"$ref": "#/definitions/v1alpha1SourceHydratorStatus"
},
"sourceType": {
"type": "string",
"title": "SourceType specifies the type of this application"
@@ -7341,6 +7881,24 @@
}
}
},
"v1alpha1DrySource": {
"description": "DrySource specifies a location for dry \"don't repeat yourself\" manifest source information.",
"type": "object",
"properties": {
"path": {
"type": "string",
"title": "Path is a directory path within the Git repository where the manifests are located"
},
"repoURL": {
"type": "string",
"title": "RepoURL is the URL to the git repository that contains the application manifests"
},
"targetRevision": {
"type": "string",
"title": "TargetRevision defines the revision of the source to hydrate"
}
}
},
"v1alpha1DuckTypeGenerator": {
"description": "DuckType defines a generator to match against clusters registered with ArgoCD.",
"type": "object",
@@ -7595,6 +8153,47 @@
}
}
},
"v1alpha1HydrateOperation": {
"type": "object",
"title": "HydrateOperation contains information about the most recent hydrate operation",
"properties": {
"drySHA": {
"type": "string",
"title": "DrySHA holds the resolved revision (sha) of the dry source as of the most recent reconciliation"
},
"finishedAt": {
"$ref": "#/definitions/v1Time"
},
"hydratedSHA": {
"type": "string",
"title": "HydratedSHA holds the resolved revision (sha) of the hydrated source as of the most recent reconciliation"
},
"message": {
"type": "string",
"title": "Message contains a message describing the current status of the hydrate operation"
},
"phase": {
"type": "string",
"title": "Phase indicates the status of the hydrate operation"
},
"sourceHydrator": {
"$ref": "#/definitions/v1alpha1SourceHydrator"
},
"startedAt": {
"$ref": "#/definitions/v1Time"
}
}
},
"v1alpha1HydrateTo": {
"description": "HydrateTo specifies a location to which hydrated manifests should be pushed as a \"staging area\" before being moved to\nthe SyncSource. The RepoURL and Path are assumed based on the associated SyncSource config in the SourceHydrator.",
"type": "object",
"properties": {
"targetBranch": {
"type": "string",
"title": "TargetBranch is the branch to which hydrated manifests should be committed"
}
}
},
"v1alpha1Info": {
"type": "object",
"properties": {
@@ -9202,6 +9801,50 @@
}
}
},
"v1alpha1SourceHydrator": {
"description": "SourceHydrator specifies a dry \"don't repeat yourself\" source for manifests, a sync source from which to sync\nhydrated manifests, and an optional hydrateTo location to act as a \"staging\" aread for hydrated manifests.",
"type": "object",
"properties": {
"drySource": {
"$ref": "#/definitions/v1alpha1DrySource"
},
"hydrateTo": {
"$ref": "#/definitions/v1alpha1HydrateTo"
},
"syncSource": {
"$ref": "#/definitions/v1alpha1SyncSource"
}
}
},
"v1alpha1SourceHydratorStatus": {
"type": "object",
"title": "SourceHydratorStatus contains information about the current state of source hydration",
"properties": {
"currentOperation": {
"$ref": "#/definitions/v1alpha1HydrateOperation"
},
"lastSuccessfulOperation": {
"$ref": "#/definitions/v1alpha1SuccessfulHydrateOperation"
}
}
},
"v1alpha1SuccessfulHydrateOperation": {
"type": "object",
"title": "SuccessfulHydrateOperation contains information about the most recent successful hydrate operation",
"properties": {
"drySHA": {
"type": "string",
"title": "DrySHA holds the resolved revision (sha) of the dry source as of the most recent reconciliation"
},
"hydratedSHA": {
"type": "string",
"title": "HydratedSHA holds the resolved revision (sha) of the hydrated source as of the most recent reconciliation"
},
"sourceHydrator": {
"$ref": "#/definitions/v1alpha1SourceHydrator"
}
}
},
"v1alpha1SyncOperation": {
"description": "SyncOperation contains details about a sync operation.",
"type": "object",
@@ -9361,6 +10004,20 @@
}
}
},
"v1alpha1SyncSource": {
"description": "SyncSource specifies a location from which hydrated manifests may be synced. RepoURL is assumed based on the\nassociated DrySource config in the SourceHydrator.",
"type": "object",
"properties": {
"path": {
"description": "Path is a directory path within the git repository where hydrated manifests should be committed to and synced\nfrom. If hydrateTo is set, this is just the path from which hydrated manifests will be synced.",
"type": "string"
},
"targetBranch": {
"type": "string",
"title": "TargetBranch is the branch to which hydrated manifests should be committed"
}
}
},
"v1alpha1SyncStatus": {
"type": "object",
"title": "SyncStatus contains information about the currently observed live and desired states of an application",

View File

@@ -19,6 +19,7 @@ import (
"k8s.io/client-go/tools/clientcmd"
cmdutil "github.com/argoproj/argo-cd/v2/cmd/util"
commitclient "github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/common"
"github.com/argoproj/argo-cd/v2/controller"
"github.com/argoproj/argo-cd/v2/controller/sharding"
@@ -58,6 +59,7 @@ func NewCommand() *cobra.Command {
repoErrorGracePeriod int64
repoServerAddress string
repoServerTimeoutSeconds int
commitServerAddress string
selfHealTimeoutSeconds int
selfHealBackoffTimeoutSeconds int
selfHealBackoffFactor int
@@ -87,7 +89,8 @@ func NewCommand() *cobra.Command {
ignoreNormalizerOpts normalizers.IgnoreNormalizerOpts
// argocd k8s event logging flag
enableK8sEvent []string
enableK8sEvent []string
hydratorEnabled bool
)
command := cobra.Command{
Use: cliName,
@@ -157,6 +160,8 @@ func NewCommand() *cobra.Command {
repoClientset := apiclient.NewRepoServerClientset(repoServerAddress, repoServerTimeoutSeconds, tlsConfig)
commitClientset := commitclient.NewCommitServerClientset(commitServerAddress)
cache, err := cacheSource()
errors.CheckError(err)
cache.Cache.SetClient(cacheutil.NewTwoLevelClient(cache.Cache.GetClient(), 10*time.Minute))
@@ -183,6 +188,7 @@ func NewCommand() *cobra.Command {
kubeClient,
appClient,
repoClientset,
commitClientset,
cache,
kubectl,
resyncDuration,
@@ -205,6 +211,7 @@ func NewCommand() *cobra.Command {
enableDynamicClusterDistribution,
ignoreNormalizerOpts,
enableK8sEvent,
hydratorEnabled,
)
errors.CheckError(err)
cacheutil.CollectMetrics(redisClient, appController.GetMetricsServer(), nil)
@@ -247,6 +254,7 @@ func NewCommand() *cobra.Command {
command.Flags().Int64Var(&repoErrorGracePeriod, "repo-error-grace-period-seconds", int64(env.ParseDurationFromEnv("ARGOCD_REPO_ERROR_GRACE_PERIOD_SECONDS", defaultAppResyncPeriod*time.Second, 0, math.MaxInt64).Seconds()), "Grace period in seconds for ignoring consecutive errors while communicating with repo server.")
command.Flags().StringVar(&repoServerAddress, "repo-server", env.StringFromEnv("ARGOCD_APPLICATION_CONTROLLER_REPO_SERVER", common.DefaultRepoServerAddr), "Repo server address.")
command.Flags().IntVar(&repoServerTimeoutSeconds, "repo-server-timeout-seconds", env.ParseNumFromEnv("ARGOCD_APPLICATION_CONTROLLER_REPO_SERVER_TIMEOUT_SECONDS", 60, 0, math.MaxInt64), "Repo server RPC call timeout seconds.")
command.Flags().StringVar(&commitServerAddress, "commit-server", env.StringFromEnv("ARGOCD_APPLICATION_CONTROLLER_COMMIT_SERVER", common.DefaultCommitServerAddr), "Commit server address.")
command.Flags().IntVar(&statusProcessors, "status-processors", env.ParseNumFromEnv("ARGOCD_APPLICATION_CONTROLLER_STATUS_PROCESSORS", 20, 0, math.MaxInt32), "Number of application status processors")
command.Flags().IntVar(&operationProcessors, "operation-processors", env.ParseNumFromEnv("ARGOCD_APPLICATION_CONTROLLER_OPERATION_PROCESSORS", 10, 0, math.MaxInt32), "Number of application operation processors")
command.Flags().StringVar(&cmdutil.LogFormat, "logformat", env.StringFromEnv("ARGOCD_APPLICATION_CONTROLLER_LOGFORMAT", "text"), "Set the logging format. One of: text|json")
@@ -285,7 +293,7 @@ func NewCommand() *cobra.Command {
command.Flags().DurationVar(&ignoreNormalizerOpts.JQExecutionTimeout, "ignore-normalizer-jq-execution-timeout-seconds", env.ParseDurationFromEnv("ARGOCD_IGNORE_NORMALIZER_JQ_TIMEOUT", 0*time.Second, 0, math.MaxInt64), "Set ignore normalizer JQ execution timeout")
// argocd k8s event logging flag
command.Flags().StringSliceVar(&enableK8sEvent, "enable-k8s-event", env.StringsFromEnv("ARGOCD_ENABLE_K8S_EVENT", argo.DefaultEnableEventList(), ","), "Enable ArgoCD to use k8s event. For disabling all events, set the value as `none`. (e.g --enable-k8s-event=none), For enabling specific events, set the value as `event reason`. (e.g --enable-k8s-event=StatusRefreshed,ResourceCreated)")
command.Flags().BoolVar(&hydratorEnabled, "hydrator-enabled", env.ParseBoolFromEnv("ARGOCD_HYDRATOR_ENABLED", false), "Feature flag to enable Hydrator. Default (\"false\")")
cacheSource = appstatecache.AddCacheFlagsToCmd(&command, cacheutil.Options{
OnClientCreated: func(client *redis.Client) {
redisClient = client

View File

@@ -0,0 +1,117 @@
package commands
import (
"fmt"
"net"
"net/http"
"os"
"os/signal"
"sync"
"syscall"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"google.golang.org/grpc/health/grpc_health_v1"
cmdutil "github.com/argoproj/argo-cd/v2/cmd/util"
"github.com/argoproj/argo-cd/v2/commitserver"
"github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/commitserver/metrics"
"github.com/argoproj/argo-cd/v2/common"
"github.com/argoproj/argo-cd/v2/util/askpass"
"github.com/argoproj/argo-cd/v2/util/cli"
"github.com/argoproj/argo-cd/v2/util/env"
"github.com/argoproj/argo-cd/v2/util/errors"
"github.com/argoproj/argo-cd/v2/util/healthz"
ioutil "github.com/argoproj/argo-cd/v2/util/io"
)
// NewCommand returns a new instance of an argocd-commit-server command
func NewCommand() *cobra.Command {
var (
listenHost string
listenPort int
metricsPort int
metricsHost string
)
command := &cobra.Command{
Use: "argocd-commit-server",
Short: "Run Argo CD Commit Server",
Long: "Argo CD Commit Server is an internal service which commits and pushes hydrated manifests to git. This command runs Commit Server in the foreground.",
RunE: func(cmd *cobra.Command, args []string) error {
vers := common.GetVersion()
vers.LogStartupInfo(
"Argo CD Commit Server",
map[string]any{
"port": listenPort,
},
)
cli.SetLogFormat(cmdutil.LogFormat)
cli.SetLogLevel(cmdutil.LogLevel)
metricsServer := metrics.NewMetricsServer()
http.Handle("/metrics", metricsServer.GetHandler())
go func() { errors.CheckError(http.ListenAndServe(fmt.Sprintf("%s:%d", metricsHost, metricsPort), nil)) }()
askPassServer := askpass.NewServer(askpass.CommitServerSocketPath)
go func() { errors.CheckError(askPassServer.Run()) }()
server := commitserver.NewServer(askPassServer, metricsServer)
grpc := server.CreateGRPC()
listener, err := net.Listen("tcp", fmt.Sprintf("%s:%d", listenHost, listenPort))
errors.CheckError(err)
healthz.ServeHealthCheck(http.DefaultServeMux, func(r *http.Request) error {
if val, ok := r.URL.Query()["full"]; ok && len(val) > 0 && val[0] == "true" {
// connect to itself to make sure commit server is able to serve connection
// used by liveness probe to auto restart commit server
conn, err := apiclient.NewConnection(fmt.Sprintf("localhost:%d", listenPort))
if err != nil {
return err
}
defer ioutil.Close(conn)
client := grpc_health_v1.NewHealthClient(conn)
res, err := client.Check(r.Context(), &grpc_health_v1.HealthCheckRequest{})
if err != nil {
return err
}
if res.Status != grpc_health_v1.HealthCheckResponse_SERVING {
return fmt.Errorf("grpc health check status is '%v'", res.Status)
}
return nil
}
return nil
})
// Graceful shutdown code adapted from here: https://gist.github.com/embano1/e0bf49d24f1cdd07cffad93097c04f0a
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
s := <-sigCh
log.Printf("got signal %v, attempting graceful shutdown", s)
grpc.GracefulStop()
wg.Done()
}()
log.Println("starting grpc server")
err = grpc.Serve(listener)
errors.CheckError(err)
wg.Wait()
log.Println("clean shutdown")
return nil
},
}
command.Flags().StringVar(&cmdutil.LogFormat, "logformat", env.StringFromEnv("ARGOCD_COMMIT_SERVER_LOGFORMAT", "text"), "Set the logging format. One of: text|json")
command.Flags().StringVar(&cmdutil.LogLevel, "loglevel", env.StringFromEnv("ARGOCD_COMMIT_SERVER_LOGLEVEL", "info"), "Set the logging level. One of: debug|info|warn|error")
command.Flags().StringVar(&listenHost, "address", env.StringFromEnv("ARGOCD_COMMIT_SERVER_LISTEN_ADDRESS", common.DefaultAddressCommitServer), "Listen on given address for incoming connections")
command.Flags().IntVar(&listenPort, "port", common.DefaultPortCommitServer, "Listen on given port for incoming connections")
command.Flags().StringVar(&metricsHost, "metrics-address", env.StringFromEnv("ARGOCD_COMMIT_SERVER_METRICS_LISTEN_ADDRESS", common.DefaultAddressCommitServerMetrics), "Listen on given address for metrics")
command.Flags().IntVar(&metricsPort, "metrics-port", common.DefaultPortCommitServerMetrics, "Start metrics server on given port")
return command
}

View File

@@ -9,7 +9,7 @@ import (
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"github.com/argoproj/argo-cd/v2/reposerver/askpass"
"github.com/argoproj/argo-cd/v2/util/askpass"
"github.com/argoproj/argo-cd/v2/util/errors"
grpc_util "github.com/argoproj/argo-cd/v2/util/grpc"
"github.com/argoproj/argo-cd/v2/util/io"

View File

@@ -23,10 +23,10 @@ import (
"github.com/argoproj/argo-cd/v2/common"
"github.com/argoproj/argo-cd/v2/reposerver"
"github.com/argoproj/argo-cd/v2/reposerver/apiclient"
"github.com/argoproj/argo-cd/v2/reposerver/askpass"
reposervercache "github.com/argoproj/argo-cd/v2/reposerver/cache"
"github.com/argoproj/argo-cd/v2/reposerver/metrics"
"github.com/argoproj/argo-cd/v2/reposerver/repository"
"github.com/argoproj/argo-cd/v2/util/askpass"
cacheutil "github.com/argoproj/argo-cd/v2/util/cache"
"github.com/argoproj/argo-cd/v2/util/cli"
"github.com/argoproj/argo-cd/v2/util/env"

View File

@@ -87,6 +87,7 @@ func NewCommand() *cobra.Command {
applicationNamespaces []string
enableProxyExtension bool
webhookParallelism int
hydratorEnabled bool
// ApplicationSet
enableNewGitFileGlobbing bool
@@ -243,6 +244,7 @@ func NewCommand() *cobra.Command {
EnableProxyExtension: enableProxyExtension,
WebhookParallelism: webhookParallelism,
EnableK8sEvent: enableK8sEvent,
HydratorEnabled: hydratorEnabled,
}
appsetOpts := server.ApplicationSetOpts{
@@ -321,6 +323,7 @@ func NewCommand() *cobra.Command {
command.Flags().BoolVar(&enableProxyExtension, "enable-proxy-extension", env.ParseBoolFromEnv("ARGOCD_SERVER_ENABLE_PROXY_EXTENSION", false), "Enable Proxy Extension feature")
command.Flags().IntVar(&webhookParallelism, "webhook-parallelism-limit", env.ParseNumFromEnv("ARGOCD_SERVER_WEBHOOK_PARALLELISM_LIMIT", 50, 1, 1000), "Number of webhook requests processed concurrently")
command.Flags().StringSliceVar(&enableK8sEvent, "enable-k8s-event", env.StringsFromEnv("ARGOCD_ENABLE_K8S_EVENT", argo.DefaultEnableEventList(), ","), "Enable ArgoCD to use k8s event. For disabling all events, set the value as `none`. (e.g --enable-k8s-event=none), For enabling specific events, set the value as `event reason`. (e.g --enable-k8s-event=StatusRefreshed,ResourceCreated)")
command.Flags().BoolVar(&hydratorEnabled, "hydrator-enabled", env.ParseBoolFromEnv("ARGOCD_HYDRATOR_ENABLED", false), "Feature flag to enable Hydrator. Default (\"false\")")
// Flags related to the applicationSet component.
command.Flags().StringVar(&scmRootCAPath, "appset-scm-root-ca-path", env.StringFromEnv("ARGOCD_APPLICATIONSET_CONTROLLER_SCM_ROOT_CA_PATH", ""), "Provide Root CA Path for self-signed TLS Certificates")

View File

@@ -9,6 +9,7 @@ import (
"sort"
"time"
"github.com/argoproj/gitops-engine/pkg/utils/kube"
"github.com/spf13/cobra"
apiv1 "k8s.io/api/core/v1"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -402,7 +403,26 @@ func reconcileApplications(
)
appStateManager := controller.NewAppStateManager(
argoDB, appClientset, repoServerClient, namespace, kubeutil.NewKubectl(), settingsMgr, stateCache, projInformer, server, cache, time.Second, argo.NewResourceTracking(), false, 0, serverSideDiff, ignoreNormalizerOpts)
argoDB,
appClientset,
repoServerClient,
namespace,
kubeutil.NewKubectl(),
func(_ string) (kube.CleanupFunc, error) {
return func() {}, nil
},
settingsMgr,
stateCache,
projInformer,
server,
cache,
time.Second,
argo.NewResourceTracking(),
false,
0,
serverSideDiff,
ignoreNormalizerOpts,
)
appsList, err := appClientset.ArgoprojV1alpha1().Applications(namespace).List(ctx, v1.ListOptions{LabelSelector: selector})
if err != nil {

View File

@@ -91,7 +91,7 @@ func TestGetReconcileResults_Refresh(t *testing.T) {
appClientset := appfake.NewSimpleClientset(app, proj)
deployment := test.NewDeployment()
kubeClientset := kubefake.NewSimpleClientset(deployment, &cm)
kubeClientset := kubefake.NewClientset(deployment, &cm)
clusterCache := clustermocks.ClusterCache{}
clusterCache.On("IsNamespaced", mock.Anything).Return(true, nil)
clusterCache.On("GetGVKParser", mock.Anything).Return(nil)

View File

@@ -183,13 +183,12 @@ func getControllerReplicas(ctx context.Context, kubeClient *kubernetes.Clientset
func NewClusterShardsCommand(clientOpts *argocdclient.ClientOptions) *cobra.Command {
var (
shard int
replicas int
shardingAlgorithm string
clientConfig clientcmd.ClientConfig
cacheSrc func() (*appstatecache.Cache, error)
portForwardRedis bool
redisCompressionStr string
shard int
replicas int
shardingAlgorithm string
clientConfig clientcmd.ClientConfig
cacheSrc func() (*appstatecache.Cache, error)
portForwardRedis bool
)
command := cobra.Command{
Use: "shards",
@@ -213,7 +212,7 @@ func NewClusterShardsCommand(clientOpts *argocdclient.ClientOptions) *cobra.Comm
if replicas == 0 {
return
}
clusters, err := loadClusters(ctx, kubeClient, appClient, replicas, shardingAlgorithm, namespace, portForwardRedis, cacheSrc, shard, clientOpts.RedisName, clientOpts.RedisHaProxyName, redisCompressionStr)
clusters, err := loadClusters(ctx, kubeClient, appClient, replicas, shardingAlgorithm, namespace, portForwardRedis, cacheSrc, shard, clientOpts.RedisName, clientOpts.RedisHaProxyName, clientOpts.RedisCompression)
errors.CheckError(err)
if len(clusters) == 0 {
return
@@ -234,7 +233,6 @@ func NewClusterShardsCommand(clientOpts *argocdclient.ClientOptions) *cobra.Comm
// we can ignore unchecked error here as the command will be parsed again and checked when command.Execute() is run later
// nolint:errcheck
command.ParseFlags(os.Args[1:])
redisCompressionStr, _ = command.Flags().GetString(cacheutil.CLIFlagRedisCompress)
return &command
}
@@ -466,13 +464,12 @@ func NewClusterDisableNamespacedMode() *cobra.Command {
func NewClusterStatsCommand(clientOpts *argocdclient.ClientOptions) *cobra.Command {
var (
shard int
replicas int
shardingAlgorithm string
clientConfig clientcmd.ClientConfig
cacheSrc func() (*appstatecache.Cache, error)
portForwardRedis bool
redisCompressionStr string
shard int
replicas int
shardingAlgorithm string
clientConfig clientcmd.ClientConfig
cacheSrc func() (*appstatecache.Cache, error)
portForwardRedis bool
)
command := cobra.Command{
Use: "stats",
@@ -502,7 +499,7 @@ argocd admin cluster stats target-cluster`,
replicas, err = getControllerReplicas(ctx, kubeClient, namespace, clientOpts.AppControllerName)
errors.CheckError(err)
}
clusters, err := loadClusters(ctx, kubeClient, appClient, replicas, shardingAlgorithm, namespace, portForwardRedis, cacheSrc, shard, clientOpts.RedisName, clientOpts.RedisHaProxyName, redisCompressionStr)
clusters, err := loadClusters(ctx, kubeClient, appClient, replicas, shardingAlgorithm, namespace, portForwardRedis, cacheSrc, shard, clientOpts.RedisName, clientOpts.RedisHaProxyName, clientOpts.RedisCompression)
errors.CheckError(err)
w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0)
@@ -524,7 +521,6 @@ argocd admin cluster stats target-cluster`,
// we can ignore unchecked error here as the command will be parsed again and checked when command.Execute() is run later
// nolint:errcheck
command.ParseFlags(os.Args[1:])
redisCompressionStr, _ = command.Flags().GetString(cacheutil.CLIFlagRedisCompress)
return &command
}
@@ -617,7 +613,7 @@ func NewGenClusterConfigCommand(pathOpts *clientcmd.PathOptions) *cobra.Command
clientConfig := clientcmd.NewDefaultClientConfig(*cfgAccess, &overrides)
conf, err := clientConfig.ClientConfig()
errors.CheckError(err)
kubeClientset := fake.NewSimpleClientset()
kubeClientset := fake.NewClientset()
var awsAuthConf *v1alpha1.AWSAuthConfig
var execProviderConf *v1alpha1.ExecProviderConfig

View File

@@ -12,17 +12,14 @@ import (
"github.com/argoproj/argo-cd/v2/cmd/argocd/commands/initialize"
"github.com/argoproj/argo-cd/v2/common"
argocdclient "github.com/argoproj/argo-cd/v2/pkg/apiclient"
"github.com/argoproj/argo-cd/v2/util/cache"
"github.com/argoproj/argo-cd/v2/util/env"
"github.com/argoproj/argo-cd/v2/util/errors"
)
func NewDashboardCommand(clientOpts *argocdclient.ClientOptions) *cobra.Command {
var (
port int
address string
compressionStr string
clientConfig clientcmd.ClientConfig
port int
address string
clientConfig clientcmd.ClientConfig
)
cmd := &cobra.Command{
Use: "dashboard",
@@ -30,10 +27,8 @@ func NewDashboardCommand(clientOpts *argocdclient.ClientOptions) *cobra.Command
Run: func(cmd *cobra.Command, args []string) {
ctx := cmd.Context()
compression, err := cache.CompressionTypeFromString(compressionStr)
errors.CheckError(err)
clientOpts.Core = true
errors.CheckError(headless.MaybeStartLocalServer(ctx, clientOpts, initialize.RetrieveContextIfChanged(cmd.Flag("context")), &port, &address, compression, clientConfig))
errors.CheckError(headless.MaybeStartLocalServer(ctx, clientOpts, initialize.RetrieveContextIfChanged(cmd.Flag("context")), &port, &address, clientConfig))
println(fmt.Sprintf("Argo CD UI is available at http://%s:%d", address, port))
<-ctx.Done()
},
@@ -50,6 +45,5 @@ $ argocd admin dashboard --redis-compress gzip
clientConfig = cli.AddKubectlFlagsToSet(cmd.Flags())
cmd.Flags().IntVar(&port, "port", common.DefaultPortAPIServer, "Listen on given port")
cmd.Flags().StringVar(&address, "address", common.DefaultAddressAdminDashboard, "Listen on given address")
cmd.Flags().StringVar(&compressionStr, "redis-compress", env.StringFromEnv("REDIS_COMPRESSION", string(cache.RedisCompressionGZip)), "Enable this if the application controller is configured with redis compression enabled. (possible values: gzip, none)")
return cmd
}

View File

@@ -150,7 +150,7 @@ func NewGenRepoSpecCommand() *cobra.Command {
},
},
}
kubeClientset := fake.NewSimpleClientset(argoCDCM)
kubeClientset := fake.NewClientset(argoCDCM)
settingsMgr := settings.NewSettingsManager(ctx, kubeClientset, ArgoCDNamespace)
argoDB := db.NewDB(ArgoCDNamespace, settingsMgr, kubeClientset)

View File

@@ -119,7 +119,7 @@ func (opts *settingsOpts) createSettingsManager(ctx context.Context) (*settings.
}
}
setSettingsMeta(argocdSecret)
clientset := fake.NewSimpleClientset(argocdSecret, argocdCM)
clientset := fake.NewClientset(argocdSecret, argocdCM)
manager := settings.NewSettingsManager(ctx, clientset, "default")
errors.CheckError(manager.ResyncInformers())

View File

@@ -147,7 +147,7 @@ func Test_PolicyFromK8s(t *testing.T) {
ctx := context.Background()
require.NoError(t, err)
kubeclientset := fake.NewSimpleClientset(&v1.ConfigMap{
kubeclientset := fake.NewClientset(&v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "argocd-rbac-cm",
Namespace: "argocd",
@@ -280,7 +280,7 @@ p, role:user, logs, get, .*/.*, allow
p, role:user, exec, create, .*/.*, allow
`
kubeclientset := fake.NewSimpleClientset(&v1.ConfigMap{
kubeclientset := fake.NewClientset(&v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "argocd-rbac-cm",
Namespace: "argocd",

View File

@@ -45,7 +45,7 @@ func captureStdout(callback func()) (string, error) {
func newSettingsManager(data map[string]string) *settings.SettingsManager {
ctx := context.Background()
clientset := fake.NewSimpleClientset(&v1.ConfigMap{
clientset := fake.NewClientset(&v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Namespace: "default",
Name: common.ArgoCDConfigMapName,

View File

@@ -112,6 +112,7 @@ type watchOpts struct {
suspended bool
degraded bool
delete bool
hydrated bool
}
// NewApplicationCreateCommand returns a new instance of an `argocd app create` command
@@ -1883,6 +1884,7 @@ func NewApplicationWaitCommand(clientOpts *argocdclient.ClientOptions) *cobra.Co
command.Flags().BoolVar(&watch.suspended, "suspended", false, "Wait for suspended")
command.Flags().BoolVar(&watch.degraded, "degraded", false, "Wait for degraded")
command.Flags().BoolVar(&watch.delete, "delete", false, "Wait for delete")
command.Flags().BoolVar(&watch.hydrated, "hydrated", false, "Wait for hydration operations")
command.Flags().StringVarP(&selector, "selector", "l", "", "Wait for apps by label. Supports '=', '==', '!=', in, notin, exists & not exists. Matching apps must satisfy all of the specified label constraints.")
command.Flags().StringArrayVar(&resources, "resource", []string{}, fmt.Sprintf("Sync only specific resources as GROUP%[1]sKIND%[1]sNAME or %[2]sGROUP%[1]sKIND%[1]sNAME. Fields may be blank and '*' can be used. This option may be specified repeatedly", resourceFieldDelimiter, resourceExcludeIndicator))
command.Flags().BoolVar(&watch.operation, "operation", false, "Wait for pending operations")
@@ -2450,7 +2452,7 @@ func groupResourceStates(app *argoappv1.Application, selectedResources []*argoap
}
// check if resource health, sync and operation statuses matches watch options
func checkResourceStatus(watch watchOpts, healthStatus string, syncStatus string, operationStatus *argoappv1.Operation) bool {
func checkResourceStatus(watch watchOpts, healthStatus string, syncStatus string, operationStatus *argoappv1.Operation, hydrationFinished bool) bool {
if watch.delete {
return false
}
@@ -2480,7 +2482,8 @@ func checkResourceStatus(watch watchOpts, healthStatus string, syncStatus string
synced := !watch.sync || syncStatus == string(argoappv1.SyncStatusCodeSynced)
operational := !watch.operation || operationStatus == nil
return synced && healthCheckPassed && operational
hydrated := !watch.hydrated || hydrationFinished
return synced && healthCheckPassed && operational && hydrated
}
// resourceParentChild gets the latest state of the app and the latest state of the app's resource tree and then
@@ -2644,13 +2647,15 @@ func waitOnApplicationStatus(ctx context.Context, acdClient argocdclient.Client,
}
}
hydrationFinished := app.Status.SourceHydrator.CurrentOperation != nil && app.Status.SourceHydrator.CurrentOperation.Phase == argoappv1.HydrateOperationPhaseHydrated && app.Status.SourceHydrator.CurrentOperation.SourceHydrator.DeepEquals(app.Status.SourceHydrator.LastSuccessfulOperation.SourceHydrator) && app.Status.SourceHydrator.CurrentOperation.DrySHA == app.Status.SourceHydrator.LastSuccessfulOperation.DrySHA
var selectedResourcesAreReady bool
// If selected resources are included, wait only on those resources, otherwise wait on the application as a whole.
if len(selectedResources) > 0 {
selectedResourcesAreReady = true
for _, state := range getResourceStates(app, selectedResources) {
resourceIsReady := checkResourceStatus(watch, state.Health, state.Status, appEvent.Application.Operation)
resourceIsReady := checkResourceStatus(watch, state.Health, state.Status, appEvent.Application.Operation, hydrationFinished)
if !resourceIsReady {
selectedResourcesAreReady = false
break
@@ -2658,7 +2663,7 @@ func waitOnApplicationStatus(ctx context.Context, acdClient argocdclient.Client,
}
} else {
// Wait on the application as a whole
selectedResourcesAreReady = checkResourceStatus(watch, string(app.Status.Health.Status), string(app.Status.Sync.Status), appEvent.Application.Operation)
selectedResourcesAreReady = checkResourceStatus(watch, string(app.Status.Health.Status), string(app.Status.Sync.Status), appEvent.Application.Operation, hydrationFinished)
}
if selectedResourcesAreReady && (!operationInProgress || !watch.operation) {

View File

@@ -1705,7 +1705,7 @@ func TestCheckResourceStatus(t *testing.T) {
suspended: true,
health: true,
degraded: true,
}, string(health.HealthStatusHealthy), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusHealthy), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Degraded, Suspended and health status failed", func(t *testing.T) {
@@ -1713,57 +1713,57 @@ func TestCheckResourceStatus(t *testing.T) {
suspended: true,
health: true,
degraded: true,
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.False(t, res)
})
t.Run("Suspended and health status passed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{
suspended: true,
health: true,
}, string(health.HealthStatusHealthy), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusHealthy), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Suspended and health status failed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{
suspended: true,
health: true,
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.False(t, res)
})
t.Run("Suspended passed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{
suspended: true,
health: false,
}, string(health.HealthStatusSuspended), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusSuspended), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Suspended failed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{
suspended: true,
health: false,
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.False(t, res)
})
t.Run("Health passed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{
suspended: false,
health: true,
}, string(health.HealthStatusHealthy), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusHealthy), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Health failed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{
suspended: false,
health: true,
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.False(t, res)
})
t.Run("Synced passed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
res := checkResourceStatus(watchOpts{}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Synced failed", func(t *testing.T) {
res := checkResourceStatus(watchOpts{}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeOutOfSync), &v1alpha1.Operation{})
res := checkResourceStatus(watchOpts{}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeOutOfSync), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Degraded passed", func(t *testing.T) {
@@ -1771,7 +1771,7 @@ func TestCheckResourceStatus(t *testing.T) {
suspended: false,
health: false,
degraded: true,
}, string(health.HealthStatusDegraded), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusDegraded), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.True(t, res)
})
t.Run("Degraded failed", func(t *testing.T) {
@@ -1779,7 +1779,7 @@ func TestCheckResourceStatus(t *testing.T) {
suspended: false,
health: false,
degraded: true,
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{})
}, string(health.HealthStatusProgressing), string(v1alpha1.SyncStatusCodeSynced), &v1alpha1.Operation{}, true)
assert.False(t, res)
})
}

View File

@@ -14,7 +14,8 @@ import (
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
corev1 "k8s.io/api/core/v1"
metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
runtimeUtil "k8s.io/apimachinery/pkg/util/runtime"
"k8s.io/client-go/dynamic"
@@ -128,7 +129,7 @@ func (c *forwardRepoClientset) NewRepoServerClient() (io.Closer, repoapiclient.R
}
repoServerName := c.repoServerName
repoServererviceLabelSelector := common.LabelKeyComponentRepoServer + "=" + common.LabelValueComponentRepoServer
repoServerServices, err := c.kubeClientset.CoreV1().Services(c.namespace).List(context.Background(), v1.ListOptions{LabelSelector: repoServererviceLabelSelector})
repoServerServices, err := c.kubeClientset.CoreV1().Services(c.namespace).List(context.Background(), metaV1.ListOptions{LabelSelector: repoServererviceLabelSelector})
if err != nil {
c.err = err
return
@@ -176,7 +177,7 @@ func testAPI(ctx context.Context, clientOpts *apiclient.ClientOptions) error {
//
// If the clientOpts enables core mode, but the local config does not have core mode enabled, this function will
// not start the local server.
func MaybeStartLocalServer(ctx context.Context, clientOpts *apiclient.ClientOptions, ctxStr string, port *int, address *string, compression cache.RedisCompressionType, clientConfig clientcmd.ClientConfig) error {
func MaybeStartLocalServer(ctx context.Context, clientOpts *apiclient.ClientOptions, ctxStr string, port *int, address *string, clientConfig clientcmd.ClientConfig) error {
if clientConfig == nil {
flags := pflag.NewFlagSet("tmp", pflag.ContinueOnError)
clientConfig = cli.AddKubectlFlagsToSet(flags)
@@ -243,6 +244,10 @@ func MaybeStartLocalServer(ctx context.Context, clientOpts *apiclient.ClientOpti
if err != nil {
return fmt.Errorf("error adding argo resources to scheme: %w", err)
}
err = corev1.AddToScheme(scheme)
if err != nil {
return fmt.Errorf("error adding corev1 resources to scheme: %w", err)
}
controllerClientset, err := client.New(restConfig, client.Options{
Scheme: scheme,
})
@@ -265,7 +270,7 @@ func MaybeStartLocalServer(ctx context.Context, clientOpts *apiclient.ClientOpti
log.Warnf("Failed to fetch & set redis password for namespace %s: %v", namespace, err)
}
appstateCache := appstatecache.NewCache(cache.NewCache(&forwardCacheClient{namespace: namespace, context: ctxStr, compression: compression, redisHaProxyName: clientOpts.RedisHaProxyName, redisName: clientOpts.RedisName, redisPassword: redisOptions.Password}), time.Hour)
appstateCache := appstatecache.NewCache(cache.NewCache(&forwardCacheClient{namespace: namespace, context: ctxStr, compression: cache.RedisCompressionType(clientOpts.RedisCompression), redisHaProxyName: clientOpts.RedisHaProxyName, redisName: clientOpts.RedisName, redisPassword: redisOptions.Password}), time.Hour)
srv := server.NewServer(ctx, server.ArgoCDServerOpts{
EnableGZip: false,
Namespace: namespace,
@@ -316,7 +321,7 @@ func NewClientOrDie(opts *apiclient.ClientOptions, c *cobra.Command) apiclient.C
ctxStr := initialize.RetrieveContextIfChanged(c.Flag("context"))
// If we're in core mode, start the API server on the fly and configure the client `opts` to use it.
// If we're not in core mode, this function call will do nothing.
err := MaybeStartLocalServer(ctx, opts, ctxStr, nil, nil, cache.RedisCompressionNone, nil)
err := MaybeStartLocalServer(ctx, opts, ctxStr, nil, nil, nil)
if err != nil {
log.Fatal(err)
}

View File

@@ -11,6 +11,7 @@ import (
cmdutil "github.com/argoproj/argo-cd/v2/cmd/util"
"github.com/argoproj/argo-cd/v2/common"
argocdclient "github.com/argoproj/argo-cd/v2/pkg/apiclient"
"github.com/argoproj/argo-cd/v2/util/cache"
"github.com/argoproj/argo-cd/v2/util/cli"
"github.com/argoproj/argo-cd/v2/util/config"
"github.com/argoproj/argo-cd/v2/util/env"
@@ -87,6 +88,7 @@ func NewCommand() *cobra.Command {
command.PersistentFlags().StringVar(&clientOpts.RedisHaProxyName, "redis-haproxy-name", env.StringFromEnv(common.EnvRedisHaProxyName, common.DefaultRedisHaProxyName), fmt.Sprintf("Name of the Redis HA Proxy; set this or the %s environment variable when the HA Proxy's name label differs from the default, for example when installing via the Helm chart", common.EnvRedisHaProxyName))
command.PersistentFlags().StringVar(&clientOpts.RedisName, "redis-name", env.StringFromEnv(common.EnvRedisName, common.DefaultRedisName), fmt.Sprintf("Name of the Redis deployment; set this or the %s environment variable when the Redis's name label differs from the default, for example when installing via the Helm chart", common.EnvRedisName))
command.PersistentFlags().StringVar(&clientOpts.RepoServerName, "repo-server-name", env.StringFromEnv(common.EnvRepoServerName, common.DefaultRepoServerName), fmt.Sprintf("Name of the Argo CD Repo server; set this or the %s environment variable when the server's name label differs from the default, for example when installing via the Helm chart", common.EnvRepoServerName))
command.PersistentFlags().StringVar(&clientOpts.RedisCompression, "redis-compress", env.StringFromEnv("REDIS_COMPRESSION", string(cache.RedisCompressionGZip)), "Enable this if the application controller is configured with redis compression enabled. (possible values: gzip, none)")
command.PersistentFlags().BoolVar(&clientOpts.PromptsEnabled, "prompts-enabled", localconfig.GetPromptsEnabled(true), "Force optional interactive prompts to be enabled or disabled, overriding local configuration. If not specified, the local configuration value will be used, which is false by default.")
clientOpts.KubeOverrides = &clientcmd.ConfigOverrides{}

View File

@@ -11,6 +11,7 @@ import (
appcontroller "github.com/argoproj/argo-cd/v2/cmd/argocd-application-controller/commands"
applicationset "github.com/argoproj/argo-cd/v2/cmd/argocd-applicationset-controller/commands"
cmpserver "github.com/argoproj/argo-cd/v2/cmd/argocd-cmp-server/commands"
commitserver "github.com/argoproj/argo-cd/v2/cmd/argocd-commit-server/commands"
dex "github.com/argoproj/argo-cd/v2/cmd/argocd-dex/commands"
gitaskpass "github.com/argoproj/argo-cd/v2/cmd/argocd-git-ask-pass/commands"
k8sauth "github.com/argoproj/argo-cd/v2/cmd/argocd-k8s-auth/commands"
@@ -46,6 +47,8 @@ func main() {
case "argocd-cmp-server":
command = cmpserver.NewCommand()
isCLI = true
case "argocd-commit-server":
command = commitserver.NewCommand()
case "argocd-dex":
command = dex.NewCommand()
case "argocd-notifications":

View File

@@ -91,6 +91,12 @@ type AppOptions struct {
retryBackoffFactor int64
ref string
SourceName string
drySourceRepo string
drySourceRevision string
drySourcePath string
syncSourceBranch string
syncSourcePath string
hydrateToBranch string
}
// SetAutoMaxProcs sets the GOMAXPROCS value based on the binary name.
@@ -112,6 +118,12 @@ func AddAppFlags(command *cobra.Command, opts *AppOptions) {
command.Flags().StringVar(&opts.chart, "helm-chart", "", "Helm Chart name")
command.Flags().StringVar(&opts.env, "env", "", "Application environment to monitor")
command.Flags().StringVar(&opts.revision, "revision", "", "The tracking source branch, tag, commit or Helm chart version the application will sync to")
command.Flags().StringVar(&opts.drySourceRepo, "dry-source-repo", "", "Repository URL of the app dry source")
command.Flags().StringVar(&opts.drySourceRevision, "dry-source-revision", "", "Revision of the app dry source")
command.Flags().StringVar(&opts.drySourcePath, "dry-source-path", "", "Path in repository to the app directory for the dry source")
command.Flags().StringVar(&opts.syncSourceBranch, "sync-source-branch", "", "The branch from which the app will sync")
command.Flags().StringVar(&opts.syncSourcePath, "sync-source-path", "", "The path in the repository from which the app will sync")
command.Flags().StringVar(&opts.hydrateToBranch, "hydrate-to-branch", "", "The branch to hydrate the app to")
command.Flags().IntVar(&opts.revisionHistoryLimit, "revision-history-limit", argoappv1.RevisionHistoryLimit, "How many items to keep in revision history")
command.Flags().StringVar(&opts.destServer, "dest-server", "", "K8s cluster URL (e.g. https://kubernetes.default.svc)")
command.Flags().StringVar(&opts.destName, "dest-name", "", "K8s cluster Name (e.g. minikube)")
@@ -175,21 +187,27 @@ func SetAppSpecOptions(flags *pflag.FlagSet, spec *argoappv1.ApplicationSpec, ap
if flags == nil {
return visited
}
source := spec.GetSourcePtrByPosition(sourcePosition)
if source == nil {
source = &argoappv1.ApplicationSource{}
}
source, visited = ConstructSource(source, *appOpts, flags)
if spec.HasMultipleSources() {
if sourcePosition == 0 {
spec.Sources[sourcePosition] = *source
} else if sourcePosition > 0 {
spec.Sources[sourcePosition-1] = *source
} else {
spec.Sources = append(spec.Sources, *source)
}
var h *argoappv1.SourceHydrator
h, hasHydratorFlag := constructSourceHydrator(spec.SourceHydrator, *appOpts, flags)
if hasHydratorFlag {
spec.SourceHydrator = h
} else {
spec.Source = source
source := spec.GetSourcePtrByPosition(sourcePosition)
if source == nil {
source = &argoappv1.ApplicationSource{}
}
source, visited = ConstructSource(source, *appOpts, flags)
if spec.HasMultipleSources() {
if sourcePosition == 0 {
spec.Sources[sourcePosition] = *source
} else if sourcePosition > 0 {
spec.Sources[sourcePosition-1] = *source
} else {
spec.Sources = append(spec.Sources, *source)
}
} else {
spec.Source = source
}
}
flags.Visit(func(f *pflag.Flag) {
visited++
@@ -592,9 +610,7 @@ func constructAppsBaseOnName(appName string, labels, annotations, args []string,
Name: appName,
Namespace: appNs,
},
Spec: argoappv1.ApplicationSpec{
Source: &argoappv1.ApplicationSource{},
},
Spec: argoappv1.ApplicationSpec{},
}
SetAppSpecOptions(flags, &app.Spec, &appOpts, 0)
SetParameterOverrides(app, appOpts.Parameters, 0)
@@ -768,6 +784,47 @@ func ConstructSource(source *argoappv1.ApplicationSource, appOpts AppOptions, fl
return source, visited
}
// constructSourceHydrator constructs a source hydrator from the command line flags. It returns the modified source
// hydrator and a boolean indicating if any hydrator flags were set. We return instead of just modifying the source
// hydrator in place because the given hydrator `h` might be nil. In that case, we need to create a new source hydrator
// and return it.
func constructSourceHydrator(h *argoappv1.SourceHydrator, appOpts AppOptions, flags *pflag.FlagSet) (*argoappv1.SourceHydrator, bool) {
hasHydratorFlag := false
ensureNotNil := func(notEmpty bool) {
hasHydratorFlag = true
if notEmpty && h == nil {
h = &argoappv1.SourceHydrator{}
}
}
flags.Visit(func(f *pflag.Flag) {
switch f.Name {
case "dry-source-repo":
ensureNotNil(appOpts.drySourceRepo != "")
h.DrySource.RepoURL = appOpts.drySourceRepo
case "dry-source-path":
ensureNotNil(appOpts.drySourcePath != "")
h.DrySource.Path = appOpts.drySourcePath
case "dry-source-revision":
ensureNotNil(appOpts.drySourceRevision != "")
h.DrySource.TargetRevision = appOpts.drySourceRevision
case "sync-source-branch":
ensureNotNil(appOpts.syncSourceBranch != "")
h.SyncSource.TargetBranch = appOpts.syncSourceBranch
case "sync-source-path":
ensureNotNil(appOpts.syncSourcePath != "")
h.SyncSource.Path = appOpts.syncSourcePath
case "hydrate-to-branch":
ensureNotNil(appOpts.hydrateToBranch != "")
if appOpts.hydrateToBranch == "" {
h.HydrateTo = nil
} else {
h.HydrateTo = &argoappv1.HydrateTo{TargetBranch: appOpts.hydrateToBranch}
}
}
})
return h, hasHydratorFlag
}
func mergeLabels(app *argoappv1.Application, labels []string) {
mapLabels, err := label.Parse(labels)
errors.CheckError(err)

View File

@@ -295,6 +295,28 @@ func Test_setAppSpecOptions(t *testing.T) {
require.NoError(t, f.SetFlag("helm-api-versions", "v2"))
assert.Equal(t, []string{"v1", "v2"}, f.spec.Source.Helm.APIVersions)
})
t.Run("source hydrator", func(t *testing.T) {
require.NoError(t, f.SetFlag("dry-source-repo", "https://github.com/argoproj/argocd-example-apps"))
assert.Equal(t, "https://github.com/argoproj/argocd-example-apps", f.spec.SourceHydrator.DrySource.RepoURL)
require.NoError(t, f.SetFlag("dry-source-path", "apps"))
assert.Equal(t, "apps", f.spec.SourceHydrator.DrySource.Path)
require.NoError(t, f.SetFlag("dry-source-revision", "HEAD"))
assert.Equal(t, "HEAD", f.spec.SourceHydrator.DrySource.TargetRevision)
require.NoError(t, f.SetFlag("sync-source-branch", "env/test"))
assert.Equal(t, "env/test", f.spec.SourceHydrator.SyncSource.TargetBranch)
require.NoError(t, f.SetFlag("sync-source-path", "apps"))
assert.Equal(t, "apps", f.spec.SourceHydrator.SyncSource.Path)
require.NoError(t, f.SetFlag("hydrate-to-branch", "env/test-next"))
assert.Equal(t, "env/test-next", f.spec.SourceHydrator.HydrateTo.TargetBranch)
require.NoError(t, f.SetFlag("hydrate-to-branch", ""))
assert.Nil(t, f.spec.SourceHydrator.HydrateTo)
})
}
func newMultiSourceAppOptionsFixture() *appOptionsFixture {

View File

@@ -162,7 +162,7 @@ func TestGetKubePublicEndpoint(t *testing.T) {
if tc.clusterInfo != nil {
objects = append(objects, tc.clusterInfo)
}
clientset := fake.NewSimpleClientset(objects...)
clientset := fake.NewClientset(objects...)
endpoint, err := GetKubePublicEndpoint(clientset)
if tc.expectError {
require.Error(t, err)

View File

@@ -0,0 +1,49 @@
package apiclient
import (
"fmt"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"github.com/argoproj/argo-cd/v2/util/io"
)
// Clientset represents commit server api clients
type Clientset interface {
NewCommitServerClient() (io.Closer, CommitServiceClient, error)
}
type clientSet struct {
address string
}
// NewCommitServerClient creates new instance of commit server client
func (c *clientSet) NewCommitServerClient() (io.Closer, CommitServiceClient, error) {
conn, err := NewConnection(c.address)
if err != nil {
return nil, nil, fmt.Errorf("failed to open a new connection to commit server: %w", err)
}
return conn, NewCommitServiceClient(conn), nil
}
// NewConnection creates new connection to commit server
func NewConnection(address string) (*grpc.ClientConn, error) {
var opts []grpc.DialOption
opts = append(opts, grpc.WithTransportCredentials(insecure.NewCredentials()))
// TODO: switch to grpc.NewClient.
// nolint:staticcheck
conn, err := grpc.Dial(address, opts...)
if err != nil {
log.Errorf("Unable to connect to commit service with address %s", address)
return nil, err
}
return conn, nil
}
// NewCommitServerClientset creates new instance of commit server Clientset
func NewCommitServerClientset(address string) Clientset {
return &clientSet{address: address}
}

1382
commitserver/apiclient/commit.pb.go generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,68 @@
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
apiclient "github.com/argoproj/argo-cd/v2/commitserver/apiclient"
io "github.com/argoproj/argo-cd/v2/util/io"
mock "github.com/stretchr/testify/mock"
)
// Clientset is an autogenerated mock type for the Clientset type
type Clientset struct {
mock.Mock
}
// NewCommitServerClient provides a mock function with given fields:
func (_m *Clientset) NewCommitServerClient() (io.Closer, apiclient.CommitServiceClient, error) {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for NewCommitServerClient")
}
var r0 io.Closer
var r1 apiclient.CommitServiceClient
var r2 error
if rf, ok := ret.Get(0).(func() (io.Closer, apiclient.CommitServiceClient, error)); ok {
return rf()
}
if rf, ok := ret.Get(0).(func() io.Closer); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(io.Closer)
}
}
if rf, ok := ret.Get(1).(func() apiclient.CommitServiceClient); ok {
r1 = rf()
} else {
if ret.Get(1) != nil {
r1 = ret.Get(1).(apiclient.CommitServiceClient)
}
}
if rf, ok := ret.Get(2).(func() error); ok {
r2 = rf()
} else {
r2 = ret.Error(2)
}
return r0, r1, r2
}
// NewClientset creates a new instance of Clientset. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewClientset(t interface {
mock.TestingT
Cleanup(func())
}) *Clientset {
mock := &Clientset{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}

View File

@@ -0,0 +1,69 @@
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
context "context"
apiclient "github.com/argoproj/argo-cd/v2/commitserver/apiclient"
grpc "google.golang.org/grpc"
mock "github.com/stretchr/testify/mock"
)
// CommitServiceClient is an autogenerated mock type for the CommitServiceClient type
type CommitServiceClient struct {
mock.Mock
}
// CommitHydratedManifests provides a mock function with given fields: ctx, in, opts
func (_m *CommitServiceClient) CommitHydratedManifests(ctx context.Context, in *apiclient.CommitHydratedManifestsRequest, opts ...grpc.CallOption) (*apiclient.CommitHydratedManifestsResponse, error) {
_va := make([]interface{}, len(opts))
for _i := range opts {
_va[_i] = opts[_i]
}
var _ca []interface{}
_ca = append(_ca, ctx, in)
_ca = append(_ca, _va...)
ret := _m.Called(_ca...)
if len(ret) == 0 {
panic("no return value specified for CommitHydratedManifests")
}
var r0 *apiclient.CommitHydratedManifestsResponse
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, *apiclient.CommitHydratedManifestsRequest, ...grpc.CallOption) (*apiclient.CommitHydratedManifestsResponse, error)); ok {
return rf(ctx, in, opts...)
}
if rf, ok := ret.Get(0).(func(context.Context, *apiclient.CommitHydratedManifestsRequest, ...grpc.CallOption) *apiclient.CommitHydratedManifestsResponse); ok {
r0 = rf(ctx, in, opts...)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(*apiclient.CommitHydratedManifestsResponse)
}
}
if rf, ok := ret.Get(1).(func(context.Context, *apiclient.CommitHydratedManifestsRequest, ...grpc.CallOption) error); ok {
r1 = rf(ctx, in, opts...)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// NewCommitServiceClient creates a new instance of CommitServiceClient. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewCommitServiceClient(t interface {
mock.TestingT
Cleanup(func())
}) *CommitServiceClient {
mock := &CommitServiceClient{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}

View File

@@ -0,0 +1,225 @@
package commit
import (
"context"
"fmt"
"os"
"time"
log "github.com/sirupsen/logrus"
"github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/commitserver/metrics"
"github.com/argoproj/argo-cd/v2/util/git"
"github.com/argoproj/argo-cd/v2/util/io/files"
)
// Service is the service that handles commit requests.
type Service struct {
gitCredsStore git.CredsStore
metricsServer *metrics.Server
repoClientFactory RepoClientFactory
}
// NewService returns a new instance of the commit service.
func NewService(gitCredsStore git.CredsStore, metricsServer *metrics.Server) *Service {
return &Service{
gitCredsStore: gitCredsStore,
metricsServer: metricsServer,
repoClientFactory: NewRepoClientFactory(gitCredsStore, metricsServer),
}
}
// CommitHydratedManifests handles a commit request. It clones the repository, checks out the sync branch, checks out
// the target branch, clears the repository contents, writes the manifests to the repository, commits the changes, and
// pushes the changes. It returns the hydrated revision SHA and an error if one occurred.
func (s *Service) CommitHydratedManifests(ctx context.Context, r *apiclient.CommitHydratedManifestsRequest) (*apiclient.CommitHydratedManifestsResponse, error) {
// This method is intentionally short. It's a wrapper around handleCommitRequest that adds metrics and logging.
// Keep logic here minimal and put most of the logic in handleCommitRequest.
startTime := time.Now()
// We validate for a nil repo in handleCommitRequest, but we need to check for a nil repo here to get the repo URL
// for metrics.
var repoURL string
if r.Repo != nil {
repoURL = r.Repo.Repo
}
var err error
s.metricsServer.IncPendingCommitRequest(repoURL)
defer func() {
s.metricsServer.DecPendingCommitRequest(repoURL)
commitResponseType := metrics.CommitResponseTypeSuccess
if err != nil {
commitResponseType = metrics.CommitResponseTypeFailure
}
s.metricsServer.IncCommitRequest(repoURL, commitResponseType)
s.metricsServer.ObserveCommitRequestDuration(repoURL, commitResponseType, time.Since(startTime))
}()
logCtx := log.WithFields(log.Fields{"branch": r.TargetBranch, "drySHA": r.DrySha})
out, sha, err := s.handleCommitRequest(logCtx, r)
if err != nil {
logCtx.WithError(err).WithField("output", out).Error("failed to handle commit request")
// No need to wrap this error, sufficient context is build in handleCommitRequest.
return &apiclient.CommitHydratedManifestsResponse{}, err
}
logCtx.Info("Successfully handled commit request")
return &apiclient.CommitHydratedManifestsResponse{
HydratedSha: sha,
}, nil
}
// handleCommitRequest handles the commit request. It clones the repository, checks out the sync branch, checks out the
// target branch, clears the repository contents, writes the manifests to the repository, commits the changes, and pushes
// the changes. It returns the output of the git commands and an error if one occurred.
func (s *Service) handleCommitRequest(logCtx *log.Entry, r *apiclient.CommitHydratedManifestsRequest) (string, string, error) {
if r.Repo == nil {
return "", "", fmt.Errorf("repo is required")
}
if r.Repo.Repo == "" {
return "", "", fmt.Errorf("repo URL is required")
}
if r.TargetBranch == "" {
return "", "", fmt.Errorf("target branch is required")
}
if r.SyncBranch == "" {
return "", "", fmt.Errorf("sync branch is required")
}
logCtx = logCtx.WithField("repo", r.Repo.Repo)
logCtx.Debug("Initiating git client")
gitClient, dirPath, cleanup, err := s.initGitClient(logCtx, r)
if err != nil {
return "", "", fmt.Errorf("failed to init git client: %w", err)
}
defer cleanup()
logCtx.Debugf("Checking out sync branch %s", r.SyncBranch)
var out string
out, err = gitClient.CheckoutOrOrphan(r.SyncBranch, false)
if err != nil {
return out, "", fmt.Errorf("failed to checkout sync branch: %w", err)
}
logCtx.Debugf("Checking out target branch %s", r.TargetBranch)
out, err = gitClient.CheckoutOrNew(r.TargetBranch, r.SyncBranch, false)
if err != nil {
return out, "", fmt.Errorf("failed to checkout target branch: %w", err)
}
logCtx.Debug("Clearing repo contents")
out, err = gitClient.RemoveContents()
if err != nil {
return out, "", fmt.Errorf("failed to clear repo: %w", err)
}
logCtx.Debug("Writing manifests")
err = WriteForPaths(dirPath, r.Repo.Repo, r.DrySha, r.Paths)
if err != nil {
return "", "", fmt.Errorf("failed to write manifests: %w", err)
}
logCtx.Debug("Committing and pushing changes")
out, err = gitClient.CommitAndPush(r.TargetBranch, r.CommitMessage)
if err != nil {
return out, "", fmt.Errorf("failed to commit and push: %w", err)
}
logCtx.Debug("Getting commit SHA")
sha, err := gitClient.CommitSHA()
if err != nil {
return "", "", fmt.Errorf("failed to get commit SHA: %w", err)
}
return "", sha, nil
}
// initGitClient initializes a git client for the given repository and returns the client, the path to the directory where
// the repository is cloned, a cleanup function that should be called when the directory is no longer needed, and an error
// if one occurred.
func (s *Service) initGitClient(logCtx *log.Entry, r *apiclient.CommitHydratedManifestsRequest) (git.Client, string, func(), error) {
dirPath, err := files.CreateTempDir("/tmp/_commit-service")
if err != nil {
return nil, "", nil, fmt.Errorf("failed to create temp dir: %w", err)
}
// Call cleanupOrLog in this function if an error occurs to ensure the temp dir is cleaned up.
cleanupOrLog := func() {
err := os.RemoveAll(dirPath)
if err != nil {
logCtx.WithError(err).Error("failed to cleanup temp dir")
}
}
gitClient, err := s.repoClientFactory.NewClient(r.Repo, dirPath)
if err != nil {
cleanupOrLog()
return nil, "", nil, fmt.Errorf("failed to create git client: %w", err)
}
logCtx.Debugf("Initializing repo %s", r.Repo.Repo)
err = gitClient.Init()
if err != nil {
cleanupOrLog()
return nil, "", nil, fmt.Errorf("failed to init git client: %w", err)
}
logCtx.Debugf("Fetching repo %s", r.Repo.Repo)
err = gitClient.Fetch("")
if err != nil {
cleanupOrLog()
return nil, "", nil, fmt.Errorf("failed to clone repo: %w", err)
}
// FIXME: make it work for GHE
//logCtx.Debugf("Getting user info for repo credentials")
//gitCreds := r.Repo.GetGitCreds(s.gitCredsStore)
//startTime := time.Now()
//authorName, authorEmail, err := gitCreds.GetUserInfo(ctx)
//s.metricsServer.ObserveUserInfoRequestDuration(r.Repo.Repo, getCredentialType(r.Repo), time.Since(startTime))
//if err != nil {
// cleanupOrLog()
// return nil, "", nil, fmt.Errorf("failed to get github app info: %w", err)
//}
var authorName, authorEmail string
if authorName == "" {
authorName = "Argo CD"
}
if authorEmail == "" {
logCtx.Warnf("Author email not available, using 'argo-cd@example.com'.")
authorEmail = "argo-cd@example.com"
}
logCtx.Debugf("Setting author %s <%s>", authorName, authorEmail)
_, err = gitClient.SetAuthor(authorName, authorEmail)
if err != nil {
cleanupOrLog()
return nil, "", nil, fmt.Errorf("failed to set author: %w", err)
}
return gitClient, dirPath, cleanupOrLog, nil
}
type hydratorMetadataFile struct {
RepoURL string `json:"repoURL"`
DrySHA string `json:"drySha"`
Commands []string `json:"commands"`
}
// TODO: make this configurable via ConfigMap.
var manifestHydrationReadmeTemplate = `
# Manifest Hydration
To hydrate the manifests in this repository, run the following commands:
` + "```shell\n" + `
git clone {{ .RepoURL }}
# cd into the cloned directory
git checkout {{ .DrySHA }}
{{ range $command := .Commands -}}
{{ $command }}
{{ end -}}` + "```"

View File

@@ -0,0 +1,50 @@
syntax = "proto3";
option go_package = "github.com/argoproj/argo-cd/v2/commitserver/apiclient";
import "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1/generated.proto";
// CommitHydratedManifestsRequest is the request to commit hydrated manifests to a repository.
message CommitHydratedManifestsRequest {
// Repo contains repository information including, at minimum, the URL of the repository. Generally it will contain
// repo credentials.
github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.Repository repo = 1;
// SyncBranch is the branch Argo CD syncs from, i.e. the hydrated branch.
string syncBranch = 2;
// TargetBranch is the branch Argo CD is committing to, i.e. the branch that will be updated.
string targetBranch = 3;
// DrySha is the commit SHA from the dry branch, i.e. pre-rendered manifest branch.
string drySha = 4;
// CommitMessage is the commit message to use when committing changes.
string commitMessage = 5;
// Paths contains the paths to write hydrated manifests to, along with the manifests and commands to execute.
repeated PathDetails paths = 6;
}
// PathDetails holds information about hydrated manifests to be written to a particular path in the hydrated manifests
// commit.
message PathDetails {
// Path is the path to write the hydrated manifests to.
string path = 1;
// Manifests contains the manifests to write to the path.
repeated HydratedManifestDetails manifests = 2;
// Commands contains the commands executed when hydrating the manifests.
repeated string commands = 3;
}
// ManifestDetails contains the hydrated manifests.
message HydratedManifestDetails {
// ManifestJSON is the hydrated manifest as JSON.
string manifestJSON = 1;
}
// ManifestsResponse is the response to the ManifestsRequest.
message CommitHydratedManifestsResponse {
// HydratedSha is the commit SHA of the hydrated manifests commit.
string hydratedSha = 1;
}
// CommitService is the service for committing hydrated manifests to a repository.
service CommitService {
// Commit commits hydrated manifests to a repository.
rpc CommitHydratedManifests (CommitHydratedManifestsRequest) returns (CommitHydratedManifestsResponse);
}

View File

@@ -0,0 +1,125 @@
package commit
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/commitserver/commit/mocks"
"github.com/argoproj/argo-cd/v2/commitserver/metrics"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/util/git"
gitmocks "github.com/argoproj/argo-cd/v2/util/git/mocks"
)
func Test_CommitHydratedManifests(t *testing.T) {
t.Parallel()
validRequest := &apiclient.CommitHydratedManifestsRequest{
Repo: &v1alpha1.Repository{
Repo: "https://github.com/argoproj/argocd-example-apps.git",
},
TargetBranch: "main",
SyncBranch: "env/test",
CommitMessage: "test commit message",
}
t.Run("missing repo", func(t *testing.T) {
t.Parallel()
service, _ := newServiceWithMocks(t)
request := &apiclient.CommitHydratedManifestsRequest{}
_, err := service.CommitHydratedManifests(context.Background(), request)
require.Error(t, err)
assert.ErrorContains(t, err, "repo is required")
})
t.Run("missing repo URL", func(t *testing.T) {
t.Parallel()
service, _ := newServiceWithMocks(t)
request := &apiclient.CommitHydratedManifestsRequest{
Repo: &v1alpha1.Repository{},
}
_, err := service.CommitHydratedManifests(context.Background(), request)
require.Error(t, err)
assert.ErrorContains(t, err, "repo URL is required")
})
t.Run("missing target branch", func(t *testing.T) {
t.Parallel()
service, _ := newServiceWithMocks(t)
request := &apiclient.CommitHydratedManifestsRequest{
Repo: &v1alpha1.Repository{
Repo: "https://github.com/argoproj/argocd-example-apps.git",
},
}
_, err := service.CommitHydratedManifests(context.Background(), request)
require.Error(t, err)
assert.ErrorContains(t, err, "target branch is required")
})
t.Run("missing sync branch", func(t *testing.T) {
t.Parallel()
service, _ := newServiceWithMocks(t)
request := &apiclient.CommitHydratedManifestsRequest{
Repo: &v1alpha1.Repository{
Repo: "https://github.com/argoproj/argocd-example-apps.git",
},
TargetBranch: "main",
}
_, err := service.CommitHydratedManifests(context.Background(), request)
require.Error(t, err)
assert.ErrorContains(t, err, "sync branch is required")
})
t.Run("failed to create git client", func(t *testing.T) {
t.Parallel()
service, mockRepoClientFactory := newServiceWithMocks(t)
mockRepoClientFactory.On("NewClient", mock.Anything, mock.Anything).Return(nil, assert.AnError).Once()
_, err := service.CommitHydratedManifests(context.Background(), validRequest)
require.Error(t, err)
assert.ErrorIs(t, err, assert.AnError)
})
t.Run("happy path", func(t *testing.T) {
t.Parallel()
service, mockRepoClientFactory := newServiceWithMocks(t)
mockGitClient := gitmocks.NewClient(t)
mockGitClient.On("Init").Return(nil).Once()
mockGitClient.On("Fetch", mock.Anything).Return(nil).Once()
mockGitClient.On("SetAuthor", "Argo CD", "argo-cd@example.com").Return("", nil).Once()
mockGitClient.On("CheckoutOrOrphan", "env/test", false).Return("", nil).Once()
mockGitClient.On("CheckoutOrNew", "main", "env/test", false).Return("", nil).Once()
mockGitClient.On("RemoveContents").Return("", nil).Once()
mockGitClient.On("CommitAndPush", "main", "test commit message").Return("", nil).Once()
mockGitClient.On("CommitSHA").Return("it-worked!", nil).Once()
mockRepoClientFactory.On("NewClient", mock.Anything, mock.Anything).Return(mockGitClient, nil).Once()
resp, err := service.CommitHydratedManifests(context.Background(), validRequest)
require.NoError(t, err)
require.NotNil(t, resp)
assert.Equal(t, "it-worked!", resp.HydratedSha)
})
}
func newServiceWithMocks(t *testing.T) (*Service, *mocks.RepoClientFactory) {
t.Helper()
metricsServer := metrics.NewMetricsServer()
mockCredsStore := git.NoopCredsStore{}
service := NewService(mockCredsStore, metricsServer)
mockRepoClientFactory := mocks.NewRepoClientFactory(t)
service.repoClientFactory = mockRepoClientFactory
return service, mockRepoClientFactory
}

View File

@@ -0,0 +1,23 @@
package commit
import "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
// getCredentialType returns the type of credential used by the repository.
func getCredentialType(repo *v1alpha1.Repository) string {
if repo == nil {
return ""
}
if repo.Password != "" {
return "https"
}
if repo.SSHPrivateKey != "" {
return "ssh"
}
if repo.GithubAppPrivateKey != "" && repo.GithubAppId != 0 && repo.GithubAppInstallationId != 0 {
return "github-app"
}
if repo.GCPServiceAccountKey != "" {
return "cloud-source-repositories"
}
return ""
}

View File

@@ -0,0 +1,62 @@
package commit
import (
"testing"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
)
func TestRepository_GetCredentialType(t *testing.T) {
tests := []struct {
name string
repo *v1alpha1.Repository
want string
}{
{
name: "Empty Repository",
repo: nil,
want: "",
},
{
name: "HTTPS Repository",
repo: &v1alpha1.Repository{
Repo: "foo",
Password: "some-password",
},
want: "https",
},
{
name: "SSH Repository",
repo: &v1alpha1.Repository{
Repo: "foo",
SSHPrivateKey: "some-key",
},
want: "ssh",
},
{
name: "GitHub App Repository",
repo: &v1alpha1.Repository{
Repo: "foo",
GithubAppPrivateKey: "some-key",
GithubAppId: 1,
GithubAppInstallationId: 1,
},
want: "github-app",
},
{
name: "Google Cloud Repository",
repo: &v1alpha1.Repository{
Repo: "foo",
GCPServiceAccountKey: "some-key",
},
want: "cloud-source-repositories",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := getCredentialType(tt.repo); got != tt.want {
t.Errorf("Repository.GetCredentialType() = %v, want %v", got, tt.want)
}
})
}
}

View File

@@ -0,0 +1,145 @@
package commit
import (
"encoding/json"
"fmt"
"os"
"path"
"text/template"
log "github.com/sirupsen/logrus"
"gopkg.in/yaml.v3"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/util/io/files"
)
// WriteForPaths writes the manifests, hydrator.metadata, and README.md files for each path in the provided paths. It
// also writes a root-level hydrator.metadata file containing the repo URL and dry SHA.
func WriteForPaths(rootPath string, repoUrl string, drySha string, paths []*apiclient.PathDetails) error {
// Write the top-level readme.
err := writeMetadata(rootPath, hydratorMetadataFile{DrySHA: drySha, RepoURL: repoUrl})
if err != nil {
return fmt.Errorf("failed to write top-level hydrator metadata: %w", err)
}
for _, p := range paths {
hydratePath := p.Path
if hydratePath == "." {
hydratePath = ""
}
var fullHydratePath string
fullHydratePath, err = files.SecureMkdirAll(rootPath, hydratePath, os.ModePerm)
if err != nil {
return fmt.Errorf("failed to create path: %w", err)
}
// Write the manifests
err = writeManifests(fullHydratePath, p.Manifests)
if err != nil {
return fmt.Errorf("failed to write manifests: %w", err)
}
// Write hydrator.metadata containing information about the hydration process.
hydratorMetadata := hydratorMetadataFile{
Commands: p.Commands,
DrySHA: drySha,
RepoURL: repoUrl,
}
err = writeMetadata(fullHydratePath, hydratorMetadata)
if err != nil {
return fmt.Errorf("failed to write hydrator metadata: %w", err)
}
// Write README
err = writeReadme(fullHydratePath, hydratorMetadata)
if err != nil {
return fmt.Errorf("failed to write readme: %w", err)
}
}
return nil
}
// writeMetadata writes the metadata to the hydrator.metadata file.
func writeMetadata(dirPath string, metadata hydratorMetadataFile) error {
hydratorMetadataJson, err := json.MarshalIndent(metadata, "", " ")
if err != nil {
return fmt.Errorf("failed to marshal hydrator metadata: %w", err)
}
// No need to use SecureJoin here, as the path is already sanitized.
hydratorMetadataPath := path.Join(dirPath, "hydrator.metadata")
err = os.WriteFile(hydratorMetadataPath, hydratorMetadataJson, os.ModePerm)
if err != nil {
return fmt.Errorf("failed to write hydrator metadata: %w", err)
}
return nil
}
// writeReadme writes the readme to the README.md file.
func writeReadme(dirPath string, metadata hydratorMetadataFile) error {
readmeTemplate := template.New("readme")
readmeTemplate, err := readmeTemplate.Parse(manifestHydrationReadmeTemplate)
if err != nil {
return fmt.Errorf("failed to parse readme template: %w", err)
}
// Create writer to template into
// No need to use SecureJoin here, as the path is already sanitized.
readmePath := path.Join(dirPath, "README.md")
readmeFile, err := os.Create(readmePath)
if err != nil && !os.IsExist(err) {
return fmt.Errorf("failed to create README file: %w", err)
}
err = readmeTemplate.Execute(readmeFile, metadata)
closeErr := readmeFile.Close()
if closeErr != nil {
log.WithError(closeErr).Error("failed to close README file")
}
if err != nil {
return fmt.Errorf("failed to execute readme template: %w", err)
}
return nil
}
// writeManifests writes the manifests to the manifest.yaml file, truncating the file if it exists and appending the
// manifests in the order they are provided.
func writeManifests(dirPath string, manifests []*apiclient.HydratedManifestDetails) error {
// If the file exists, truncate it.
// No need to use SecureJoin here, as the path is already sanitized.
manifestPath := path.Join(dirPath, "manifest.yaml")
file, err := os.OpenFile(manifestPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, os.ModePerm)
if err != nil {
return fmt.Errorf("failed to open manifest file: %w", err)
}
defer func() {
err := file.Close()
if err != nil {
log.WithError(err).Error("failed to close file")
}
}()
enc := yaml.NewEncoder(file)
defer func() {
err := enc.Close()
if err != nil {
log.WithError(err).Error("failed to close yaml encoder")
}
}()
enc.SetIndent(2)
for _, m := range manifests {
obj := &unstructured.Unstructured{}
err = json.Unmarshal([]byte(m.ManifestJSON), obj)
if err != nil {
return fmt.Errorf("failed to unmarshal manifest: %w", err)
}
err = enc.Encode(&obj.Object)
if err != nil {
return fmt.Errorf("failed to encode manifest: %w", err)
}
}
return nil
}

View File

@@ -0,0 +1,135 @@
package commit
import (
"encoding/json"
"os"
"path"
"testing"
securejoin "github.com/cyphar/filepath-securejoin"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/argoproj/argo-cd/v2/commitserver/apiclient"
)
func TestWriteForPaths(t *testing.T) {
dir := t.TempDir()
repoUrl := "https://github.com/example/repo"
drySha := "abc123"
paths := []*apiclient.PathDetails{
{
Path: "path1",
Manifests: []*apiclient.HydratedManifestDetails{
{ManifestJSON: `{"kind":"Pod","apiVersion":"v1"}`},
},
Commands: []string{"command1", "command2"},
},
{
Path: "path2",
Manifests: []*apiclient.HydratedManifestDetails{
{ManifestJSON: `{"kind":"Service","apiVersion":"v1"}`},
},
Commands: []string{"command3"},
},
}
err := WriteForPaths(dir, repoUrl, drySha, paths)
require.NoError(t, err)
// Check if the top-level hydrator.metadata exists and contains the repo URL and dry SHA
topMetadataPath := path.Join(dir, "hydrator.metadata")
topMetadataBytes, err := os.ReadFile(topMetadataPath)
require.NoError(t, err)
var topMetadata hydratorMetadataFile
err = json.Unmarshal(topMetadataBytes, &topMetadata)
require.NoError(t, err)
assert.Equal(t, repoUrl, topMetadata.RepoURL)
assert.Equal(t, drySha, topMetadata.DrySHA)
for _, p := range paths {
fullHydratePath, err := securejoin.SecureJoin(dir, p.Path)
require.NoError(t, err)
// Check if each path directory exists
assert.DirExists(t, fullHydratePath)
// Check if each path contains a hydrator.metadata file and contains the repo URL
metadataPath := path.Join(fullHydratePath, "hydrator.metadata")
metadataBytes, err := os.ReadFile(metadataPath)
require.NoError(t, err)
var readMetadata hydratorMetadataFile
err = json.Unmarshal(metadataBytes, &readMetadata)
require.NoError(t, err)
assert.Equal(t, repoUrl, readMetadata.RepoURL)
// Check if each path contains a README.md file and contains the repo URL
readmePath := path.Join(fullHydratePath, "README.md")
readmeBytes, err := os.ReadFile(readmePath)
require.NoError(t, err)
assert.Contains(t, string(readmeBytes), repoUrl)
// Check if each path contains a manifest.yaml file and contains the word Pod
manifestPath := path.Join(fullHydratePath, "manifest.yaml")
manifestBytes, err := os.ReadFile(manifestPath)
require.NoError(t, err)
assert.Contains(t, string(manifestBytes), "kind")
}
}
func TestWriteMetadata(t *testing.T) {
dir := t.TempDir()
metadata := hydratorMetadataFile{
RepoURL: "https://github.com/example/repo",
DrySHA: "abc123",
}
err := writeMetadata(dir, metadata)
require.NoError(t, err)
metadataPath := path.Join(dir, "hydrator.metadata")
metadataBytes, err := os.ReadFile(metadataPath)
require.NoError(t, err)
var readMetadata hydratorMetadataFile
err = json.Unmarshal(metadataBytes, &readMetadata)
require.NoError(t, err)
assert.Equal(t, metadata, readMetadata)
}
func TestWriteReadme(t *testing.T) {
dir := t.TempDir()
metadata := hydratorMetadataFile{
RepoURL: "https://github.com/example/repo",
DrySHA: "abc123",
}
err := writeReadme(dir, metadata)
require.NoError(t, err)
readmePath := path.Join(dir, "README.md")
readmeBytes, err := os.ReadFile(readmePath)
require.NoError(t, err)
assert.Contains(t, string(readmeBytes), metadata.RepoURL)
}
func TestWriteManifests(t *testing.T) {
dir := t.TempDir()
manifests := []*apiclient.HydratedManifestDetails{
{ManifestJSON: `{"kind":"Pod","apiVersion":"v1"}`},
}
err := writeManifests(dir, manifests)
require.NoError(t, err)
manifestPath := path.Join(dir, "manifest.yaml")
manifestBytes, err := os.ReadFile(manifestPath)
require.NoError(t, err)
assert.Contains(t, string(manifestBytes), "kind")
}

View File

@@ -0,0 +1,59 @@
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
git "github.com/argoproj/argo-cd/v2/util/git"
mock "github.com/stretchr/testify/mock"
v1alpha1 "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
)
// RepoClientFactory is an autogenerated mock type for the RepoClientFactory type
type RepoClientFactory struct {
mock.Mock
}
// NewClient provides a mock function with given fields: repo, rootPath
func (_m *RepoClientFactory) NewClient(repo *v1alpha1.Repository, rootPath string) (git.Client, error) {
ret := _m.Called(repo, rootPath)
if len(ret) == 0 {
panic("no return value specified for NewClient")
}
var r0 git.Client
var r1 error
if rf, ok := ret.Get(0).(func(*v1alpha1.Repository, string) (git.Client, error)); ok {
return rf(repo, rootPath)
}
if rf, ok := ret.Get(0).(func(*v1alpha1.Repository, string) git.Client); ok {
r0 = rf(repo, rootPath)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(git.Client)
}
}
if rf, ok := ret.Get(1).(func(*v1alpha1.Repository, string) error); ok {
r1 = rf(repo, rootPath)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// NewRepoClientFactory creates a new instance of RepoClientFactory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewRepoClientFactory(t interface {
mock.TestingT
Cleanup(func())
}) *RepoClientFactory {
mock := &RepoClientFactory{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}

View File

@@ -0,0 +1,32 @@
package commit
import (
"github.com/argoproj/argo-cd/v2/commitserver/metrics"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/util/git"
)
// RepoClientFactory is a factory for creating git clients for a repository.
type RepoClientFactory interface {
NewClient(repo *v1alpha1.Repository, rootPath string) (git.Client, error)
}
type repoClientFactory struct {
gitCredsStore git.CredsStore
metricsServer *metrics.Server
}
// NewRepoClientFactory returns a new instance of the repo client factory.
func NewRepoClientFactory(gitCredsStore git.CredsStore, metricsServer *metrics.Server) RepoClientFactory {
return &repoClientFactory{
gitCredsStore: gitCredsStore,
metricsServer: metricsServer,
}
}
// NewClient creates a new git client for the repository.
func (r *repoClientFactory) NewClient(repo *v1alpha1.Repository, rootPath string) (git.Client, error) {
gitCreds := repo.GetGitCreds(r.gitCredsStore)
opts := git.WithEventHandlers(metrics.NewGitClientEventHandlers(r.metricsServer))
return git.NewClientExt(repo.Repo, rootPath, gitCreds, repo.IsInsecure(), repo.IsLFSEnabled(), repo.Proxy, repo.NoProxy, opts)
}

View File

@@ -0,0 +1,34 @@
package metrics
import (
"time"
"github.com/argoproj/argo-cd/v2/util/git"
)
// NewGitClientEventHandlers creates event handlers that update Git related metrics
func NewGitClientEventHandlers(metricsServer *Server) git.EventHandlers {
return git.EventHandlers{
OnFetch: func(repo string) func() {
startTime := time.Now()
metricsServer.IncGitRequest(repo, GitRequestTypeFetch)
return func() {
metricsServer.ObserveGitRequestDuration(repo, GitRequestTypeFetch, time.Since(startTime))
}
},
OnLsRemote: func(repo string) func() {
startTime := time.Now()
metricsServer.IncGitRequest(repo, GitRequestTypeLsRemote)
return func() {
metricsServer.ObserveGitRequestDuration(repo, GitRequestTypeLsRemote, time.Since(startTime))
}
},
OnPush: func(repo string) func() {
startTime := time.Now()
metricsServer.IncGitRequest(repo, GitRequestTypePush)
return func() {
metricsServer.ObserveGitRequestDuration(repo, GitRequestTypePush, time.Since(startTime))
}
},
}
}

View File

@@ -0,0 +1,157 @@
package metrics
import (
"net/http"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/collectors"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
// Server is a prometheus server which collects application metrics.
type Server struct {
handler http.Handler
commitPendingRequestsGauge *prometheus.GaugeVec
gitRequestCounter *prometheus.CounterVec
gitRequestHistogram *prometheus.HistogramVec
commitRequestHistogram *prometheus.HistogramVec
userInfoRequestHistogram *prometheus.HistogramVec
commitRequestCounter *prometheus.CounterVec
}
// GitRequestType is the type of git request
type GitRequestType string
const (
// GitRequestTypeLsRemote is a request to list remote refs
GitRequestTypeLsRemote = "ls-remote"
// GitRequestTypeFetch is a request to fetch from remote
GitRequestTypeFetch = "fetch"
// GitRequestTypePush is a request to push to remote
GitRequestTypePush = "push"
)
// CommitResponseType is the type of response for a commit request
type CommitResponseType string
const (
// CommitResponseTypeSuccess is a successful commit request
CommitResponseTypeSuccess CommitResponseType = "success"
// CommitResponseTypeFailure is a failed commit request
CommitResponseTypeFailure CommitResponseType = "failure"
)
// NewMetricsServer returns a new prometheus server which collects application metrics.
func NewMetricsServer() *Server {
registry := prometheus.NewRegistry()
registry.MustRegister(collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}))
registry.MustRegister(collectors.NewGoCollector())
commitPendingRequestsGauge := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "argocd_commitserver_commit_pending_request_total",
Help: "Number of pending commit requests",
},
[]string{"repo"},
)
registry.MustRegister(commitPendingRequestsGauge)
gitRequestCounter := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "argocd_commitserver_git_request_total",
Help: "Number of git requests performed by repo server",
},
[]string{"repo", "request_type"},
)
registry.MustRegister(gitRequestCounter)
gitRequestHistogram := prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "argocd_commitserver_git_request_duration_seconds",
Help: "Git requests duration seconds.",
Buckets: []float64{0.1, 0.25, .5, 1, 2, 4, 10, 20},
},
[]string{"repo", "request_type"},
)
registry.MustRegister(gitRequestHistogram)
commitRequestHistogram := prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "argocd_commitserver_commit_request_duration_seconds",
Help: "Commit request duration seconds.",
Buckets: []float64{0.1, 0.25, .5, 1, 2, 4, 10, 20},
},
[]string{"repo", "response_type"},
)
registry.MustRegister(commitRequestHistogram)
userInfoRequestHistogram := prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "argocd_commitserver_userinfo_request_duration_seconds",
Help: "Userinfo request duration seconds.",
Buckets: []float64{0.1, 0.25, .5, 1, 2, 4, 10, 20},
},
[]string{"repo", "credential_type"},
)
registry.MustRegister(userInfoRequestHistogram)
commitRequestCounter := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "argocd_commitserver_commit_request_total",
Help: "Number of commit requests performed handled",
},
[]string{"repo", "response_type"},
)
registry.MustRegister(commitRequestCounter)
return &Server{
handler: promhttp.HandlerFor(registry, promhttp.HandlerOpts{}),
commitPendingRequestsGauge: commitPendingRequestsGauge,
gitRequestCounter: gitRequestCounter,
gitRequestHistogram: gitRequestHistogram,
commitRequestHistogram: commitRequestHistogram,
userInfoRequestHistogram: userInfoRequestHistogram,
commitRequestCounter: commitRequestCounter,
}
}
// GetHandler returns the http.Handler for the prometheus server
func (m *Server) GetHandler() http.Handler {
return m.handler
}
// IncPendingCommitRequest increments the pending commit requests gauge
func (m *Server) IncPendingCommitRequest(repo string) {
m.commitPendingRequestsGauge.WithLabelValues(repo).Inc()
}
// DecPendingCommitRequest decrements the pending commit requests gauge
func (m *Server) DecPendingCommitRequest(repo string) {
m.commitPendingRequestsGauge.WithLabelValues(repo).Dec()
}
// IncGitRequest increments the git requests counter
func (m *Server) IncGitRequest(repo string, requestType GitRequestType) {
m.gitRequestCounter.WithLabelValues(repo, string(requestType)).Inc()
}
// ObserveGitRequestDuration observes the duration of a git request
func (m *Server) ObserveGitRequestDuration(repo string, requestType GitRequestType, duration time.Duration) {
m.gitRequestHistogram.WithLabelValues(repo, string(requestType)).Observe(duration.Seconds())
}
// ObserveCommitRequestDuration observes the duration of a commit request
func (m *Server) ObserveCommitRequestDuration(repo string, rt CommitResponseType, duration time.Duration) {
m.commitRequestHistogram.WithLabelValues(repo, string(rt)).Observe(duration.Seconds())
}
// ObserveUserInfoRequestDuration observes the duration of a userinfo request
func (m *Server) ObserveUserInfoRequestDuration(repo string, credentialType string, duration time.Duration) {
m.userInfoRequestHistogram.WithLabelValues(repo, credentialType).Observe(duration.Seconds())
}
// IncCommitRequest increments the commit request counter
func (m *Server) IncCommitRequest(repo string, rt CommitResponseType) {
m.commitRequestCounter.WithLabelValues(repo, string(rt)).Inc()
}

38
commitserver/server.go Normal file
View File

@@ -0,0 +1,38 @@
package commitserver
import (
"google.golang.org/grpc"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
"github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/commitserver/commit"
"github.com/argoproj/argo-cd/v2/commitserver/metrics"
versionpkg "github.com/argoproj/argo-cd/v2/pkg/apiclient/version"
"github.com/argoproj/argo-cd/v2/server/version"
"github.com/argoproj/argo-cd/v2/util/git"
)
// ArgoCDCommitServer is the server that handles commit requests.
type ArgoCDCommitServer struct {
commitService *commit.Service
}
// NewServer returns a new instance of the commit server.
func NewServer(gitCredsStore git.CredsStore, metricsServer *metrics.Server) *ArgoCDCommitServer {
return &ArgoCDCommitServer{commitService: commit.NewService(gitCredsStore, metricsServer)}
}
// CreateGRPC creates a new gRPC server.
func (a *ArgoCDCommitServer) CreateGRPC() *grpc.Server {
server := grpc.NewServer()
versionpkg.RegisterVersionServiceServer(server, version.NewServer(nil, func() (bool, error) {
return true, nil
}))
apiclient.RegisterCommitServiceServer(server, a.commitService)
healthService := health.NewServer()
grpc_health_v1.RegisterHealthServer(server, healthService)
return server
}

View File

@@ -26,6 +26,8 @@ const (
const (
// DefaultRepoServerAddr is the gRPC address of the Argo CD repo server
DefaultRepoServerAddr = "argocd-repo-server:8081"
// DefaultCommitServerAddr is the gRPC address of the Argo CD commit server
DefaultCommitServerAddr = "argocd-commit-server:8086"
// DefaultDexServerAddr is the HTTP address of the Dex OIDC server, which we run a reverse proxy against
DefaultDexServerAddr = "argocd-dex-server:5556"
// DefaultRedisAddr is the default redis address
@@ -62,15 +64,19 @@ const (
DefaultPortArgoCDMetrics = 8082
DefaultPortArgoCDAPIServerMetrics = 8083
DefaultPortRepoServerMetrics = 8084
DefaultPortCommitServer = 8086
DefaultPortCommitServerMetrics = 8087
)
// DefaultAddressAPIServer for ArgoCD components
const (
DefaultAddressAdminDashboard = "localhost"
DefaultAddressAPIServer = "0.0.0.0"
DefaultAddressAPIServerMetrics = "0.0.0.0"
DefaultAddressRepoServer = "0.0.0.0"
DefaultAddressRepoServerMetrics = "0.0.0.0"
DefaultAddressAdminDashboard = "localhost"
DefaultAddressAPIServer = "0.0.0.0"
DefaultAddressAPIServerMetrics = "0.0.0.0"
DefaultAddressRepoServer = "0.0.0.0"
DefaultAddressRepoServerMetrics = "0.0.0.0"
DefaultAddressCommitServer = "0.0.0.0"
DefaultAddressCommitServerMetrics = "0.0.0.0"
)
// Default paths on the pod's file system
@@ -175,6 +181,8 @@ const (
LabelValueSecretTypeRepository = "repository"
// LabelValueSecretTypeRepoCreds indicates a secret type of repository credentials
LabelValueSecretTypeRepoCreds = "repo-creds"
// LabelValueSecretTypeRepositoryWrite indicates a secret type of repository credentials for writing
LabelValueSecretTypeRepositoryWrite = "repository-write"
// LabelValueSecretTypeSCMCreds indicates a secret type of SCM credentials
LabelValueSecretTypeSCMCreds = "scm-creds"

View File

@@ -92,7 +92,7 @@ func TestSetOptionalRedisPasswordFromKubeConfig(t *testing.T) {
t.Parallel()
var (
ctx = context.TODO()
kubeClient = kubefake.NewSimpleClientset()
kubeClient = kubefake.NewClientset()
redisOptions = &redis.Options{}
)
if tc.secret != nil {

View File

@@ -42,8 +42,10 @@ import (
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/util/workqueue"
commitclient "github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/common"
statecache "github.com/argoproj/argo-cd/v2/controller/cache"
"github.com/argoproj/argo-cd/v2/controller/hydrator"
"github.com/argoproj/argo-cd/v2/controller/metrics"
"github.com/argoproj/argo-cd/v2/controller/sharding"
"github.com/argoproj/argo-cd/v2/pkg/apis/application"
@@ -121,6 +123,8 @@ type ApplicationController struct {
appComparisonTypeRefreshQueue workqueue.TypedRateLimitingInterface[string]
appOperationQueue workqueue.TypedRateLimitingInterface[string]
projectRefreshQueue workqueue.TypedRateLimitingInterface[string]
appHydrateQueue workqueue.TypedRateLimitingInterface[string]
hydrationQueue workqueue.TypedRateLimitingInterface[hydrator.HydrationQueueKey]
appInformer cache.SharedIndexInformer
appLister applisters.ApplicationLister
projInformer cache.SharedIndexInformer
@@ -146,6 +150,8 @@ type ApplicationController struct {
// dynamicClusterDistributionEnabled if disabled deploymentInformer is never initialized
dynamicClusterDistributionEnabled bool
deploymentInformer informerv1.DeploymentInformer
hydrator *hydrator.Hydrator
}
// NewApplicationController creates new instance of ApplicationController.
@@ -155,6 +161,7 @@ func NewApplicationController(
kubeClientset kubernetes.Interface,
applicationClientset appclientset.Interface,
repoClientset apiclient.Clientset,
commitClientset commitclient.Clientset,
argoCache *appstatecache.Cache,
kubectl kube.Kubectl,
appResyncPeriod time.Duration,
@@ -177,6 +184,7 @@ func NewApplicationController(
dynamicClusterDistributionEnabled bool,
ignoreNormalizerOpts normalizers.IgnoreNormalizerOpts,
enableK8sEvent []string,
hydratorEnabled bool,
) (*ApplicationController, error) {
log.Infof("appResyncPeriod=%v, appHardResyncPeriod=%v, appResyncJitter=%v", appResyncPeriod, appHardResyncPeriod, appResyncJitter)
db := db.NewDB(namespace, settingsMgr, kubeClientset)
@@ -190,10 +198,12 @@ func NewApplicationController(
kubeClientset: kubeClientset,
kubectl: kubectl,
applicationClientset: applicationClientset,
appRefreshQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter(rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "app_reconciliation_queue"}),
appOperationQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter(rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "app_operation_processing_queue"}),
projectRefreshQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter(rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "project_reconciliation_queue"}),
appComparisonTypeRefreshQueue: workqueue.NewTypedRateLimitingQueue(ratelimiter.NewCustomAppControllerRateLimiter(rateLimiterConfig)),
appRefreshQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter[string](rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "app_reconciliation_queue"}),
appOperationQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter[string](rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "app_operation_processing_queue"}),
projectRefreshQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter[string](rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "project_reconciliation_queue"}),
appComparisonTypeRefreshQueue: workqueue.NewTypedRateLimitingQueue(ratelimiter.NewCustomAppControllerRateLimiter[string](rateLimiterConfig)),
appHydrateQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter[string](rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[string]{Name: "app_hydration_queue"}),
hydrationQueue: workqueue.NewTypedRateLimitingQueueWithConfig(ratelimiter.NewCustomAppControllerRateLimiter[hydrator.HydrationQueueKey](rateLimiterConfig), workqueue.TypedRateLimitingQueueConfig[hydrator.HydrationQueueKey]{Name: "manifest_hydration_queue"}),
db: db,
statusRefreshTimeout: appResyncPeriod,
statusHardRefreshTimeout: appHardResyncPeriod,
@@ -211,6 +221,9 @@ func NewApplicationController(
dynamicClusterDistributionEnabled: dynamicClusterDistributionEnabled,
ignoreNormalizerOpts: ignoreNormalizerOpts,
}
if hydratorEnabled {
ctrl.hydrator = hydrator.NewHydrator(&ctrl, appResyncPeriod, commitClientset)
}
if kubectlParallelismLimit > 0 {
ctrl.kubectlSemaphore = semaphore.NewWeighted(kubectlParallelismLimit)
}
@@ -296,7 +309,7 @@ func NewApplicationController(
}
}
stateCache := statecache.NewLiveStateCache(db, appInformer, ctrl.settingsMgr, kubectl, ctrl.metricsServer, ctrl.handleObjectUpdated, clusterSharding, argo.NewResourceTracking())
appStateManager := NewAppStateManager(db, applicationClientset, repoClientset, namespace, kubectl, ctrl.settingsMgr, stateCache, projInformer, ctrl.metricsServer, argoCache, ctrl.statusRefreshTimeout, argo.NewResourceTracking(), persistResourceHealth, repoErrorGracePeriod, serverSideDiff, ignoreNormalizerOpts)
appStateManager := NewAppStateManager(db, applicationClientset, repoClientset, namespace, kubectl, ctrl.onKubectlRun, ctrl.settingsMgr, stateCache, projInformer, ctrl.metricsServer, argoCache, ctrl.statusRefreshTimeout, argo.NewResourceTracking(), persistResourceHealth, repoErrorGracePeriod, serverSideDiff, ignoreNormalizerOpts)
ctrl.appInformer = appInformer
ctrl.appLister = appLister
ctrl.projInformer = projInformer
@@ -845,6 +858,8 @@ func (ctrl *ApplicationController) Run(ctx context.Context, statusProcessors int
defer ctrl.appComparisonTypeRefreshQueue.ShutDown()
defer ctrl.appOperationQueue.ShutDown()
defer ctrl.projectRefreshQueue.ShutDown()
defer ctrl.appHydrateQueue.ShutDown()
defer ctrl.hydrationQueue.ShutDown()
ctrl.metricsServer.RegisterClustersInfoSource(ctx, ctrl.stateCache)
ctrl.RegisterClusterSecretUpdater(ctx)
@@ -903,6 +918,19 @@ func (ctrl *ApplicationController) Run(ctx context.Context, statusProcessors int
for ctrl.processProjectQueueItem() {
}
}, time.Second, ctx.Done())
if ctrl.hydrator != nil {
go wait.Until(func() {
for ctrl.processAppHydrateQueueItem() {
}
}, time.Second, ctx.Done())
go wait.Until(func() {
for ctrl.processHydrationQueueItem() {
}
}, time.Second, ctx.Done())
}
<-ctx.Done()
}
@@ -912,7 +940,7 @@ func (ctrl *ApplicationController) requestAppRefresh(appName string, compareWith
key := ctrl.toAppKey(appName)
if compareWith != nil && after != nil {
ctrl.appComparisonTypeRefreshQueue.AddAfter(fmt.Sprintf("%s/%d", key, compareWith), *after)
ctrl.appComparisonTypeRefreshQueue.AddAfter(fmt.Sprintf("%s/%d", key, *compareWith), *after)
} else {
if compareWith != nil {
ctrl.refreshRequestedAppsMutex.Lock()
@@ -1774,6 +1802,68 @@ func (ctrl *ApplicationController) processAppRefreshQueueItem() (processNext boo
return
}
func (ctrl *ApplicationController) processAppHydrateQueueItem() (processNext bool) {
appKey, shutdown := ctrl.appHydrateQueue.Get()
if shutdown {
processNext = false
return
}
processNext = true
defer func() {
if r := recover(); r != nil {
log.Errorf("Recovered from panic: %+v\n%s", r, debug.Stack())
}
ctrl.appHydrateQueue.Done(appKey)
}()
obj, exists, err := ctrl.appInformer.GetIndexer().GetByKey(appKey)
if err != nil {
log.Errorf("Failed to get application '%s' from informer index: %+v", appKey, err)
return
}
if !exists {
// This happens after app was deleted, but the work queue still had an entry for it.
return
}
origApp, ok := obj.(*appv1.Application)
if !ok {
log.Warnf("Key '%s' in index is not an application", appKey)
return
}
ctrl.hydrator.ProcessAppHydrateQueueItem(origApp)
getAppLog(origApp).Debug("Successfully processed app hydrate queue item")
return
}
func (ctrl *ApplicationController) processHydrationQueueItem() (processNext bool) {
hydrationKey, shutdown := ctrl.hydrationQueue.Get()
if shutdown {
processNext = false
return
}
processNext = true
defer func() {
if r := recover(); r != nil {
log.Errorf("Recovered from panic: %+v\n%s", r, debug.Stack())
}
ctrl.hydrationQueue.Done(hydrationKey)
}()
logCtx := log.WithFields(log.Fields{
"sourceRepoURL": hydrationKey.SourceRepoURL,
"sourceTargetRevision": hydrationKey.SourceTargetRevision,
"destinationBranch": hydrationKey.DestinationBranch,
})
logCtx.Debug("Processing hydration queue item")
ctrl.hydrator.ProcessHydrationQueueItem(hydrationKey)
logCtx.Debug("Successfully processed hydration queue item")
return
}
func resourceStatusKey(res appv1.ResourceStatus) string {
return strings.Join([]string{res.Group, res.Kind, res.Namespace, res.Name}, "/")
}
@@ -1782,7 +1872,8 @@ func currentSourceEqualsSyncedSource(app *appv1.Application) bool {
if app.Spec.HasMultipleSources() {
return app.Spec.Sources.Equals(app.Status.Sync.ComparedTo.Sources)
}
return app.Spec.Source.Equals(&app.Status.Sync.ComparedTo.Source)
source := app.Spec.GetSource()
return source.Equals(&app.Status.Sync.ComparedTo.Source)
}
// needRefreshAppStatus answers if application status needs to be refreshed.
@@ -1918,6 +2009,7 @@ func (ctrl *ApplicationController) persistAppStatus(orig *appv1.Application, new
newAnnotations[k] = v
}
delete(newAnnotations, appv1.AnnotationKeyRefresh)
delete(newAnnotations, appv1.AnnotationKeyHydrate)
}
patch, modified, err := createMergePatch(
&appv1.Application{ObjectMeta: metav1.ObjectMeta{Annotations: orig.GetAnnotations()}, Status: orig.Status},
@@ -2325,6 +2417,9 @@ func (ctrl *ApplicationController) newApplicationInformerAndLister() (cache.Shar
if !newOK || (delay != nil && *delay != time.Duration(0)) {
ctrl.appOperationQueue.AddRateLimited(key)
}
if ctrl.hydrator != nil {
ctrl.appHydrateQueue.AddRateLimited(newApp.QualifiedName())
}
ctrl.clusterSharding.UpdateApp(newApp)
},
DeleteFunc: func(obj interface{}) {

View File

@@ -42,6 +42,7 @@ import (
dbmocks "github.com/argoproj/argo-cd/v2/util/db/mocks"
mockcommitclient "github.com/argoproj/argo-cd/v2/commitserver/apiclient/mocks"
mockstatecache "github.com/argoproj/argo-cd/v2/controller/cache/mocks"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
appclientset "github.com/argoproj/argo-cd/v2/pkg/client/clientset/versioned/fake"
@@ -126,6 +127,8 @@ func newFakeControllerWithResync(data *fakeData, appResyncPeriod time.Duration,
mockRepoClientset := mockrepoclient.Clientset{RepoServerServiceClient: &mockRepoClient}
mockCommitClientset := mockcommitclient.Clientset{}
secret := corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "argocd-secret",
@@ -148,7 +151,7 @@ func newFakeControllerWithResync(data *fakeData, appResyncPeriod time.Duration,
}
runtimeObjs := []runtime.Object{&clust, &secret, &cm}
runtimeObjs = append(runtimeObjs, data.additionalObjs...)
kubeClient := fake.NewSimpleClientset(runtimeObjs...)
kubeClient := fake.NewClientset(runtimeObjs...)
settingsMgr := settings.NewSettingsManager(context.Background(), kubeClient, test.FakeArgoCDNamespace)
kubectl := &MockKubectl{Kubectl: &kubetest.MockKubectlCmd{}}
ctrl, err := NewApplicationController(
@@ -157,6 +160,7 @@ func newFakeControllerWithResync(data *fakeData, appResyncPeriod time.Duration,
kubeClient,
appclientset.NewSimpleClientset(data.apps...),
&mockRepoClientset,
&mockCommitClientset,
appstatecache.NewCache(
cacheutil.NewCache(cacheutil.NewInMemoryCache(1*time.Minute)),
1*time.Minute,
@@ -182,6 +186,7 @@ func newFakeControllerWithResync(data *fakeData, appResyncPeriod time.Duration,
false,
normalizers.IgnoreNormalizerOpts{},
testEnableEventList,
false,
)
db := &dbmocks.ArgoDB{}
db.On("GetApplicationControllerReplicas").Return(1)
@@ -1404,6 +1409,25 @@ func TestNeedRefreshAppStatus(t *testing.T) {
assert.Equal(t, CompareWithRecent, compareWith)
})
t.Run("requesting refresh with delay gives correct compression level", func(t *testing.T) {
needRefresh, _, _ := ctrl.needRefreshAppStatus(app, 1*time.Hour, 2*time.Hour)
assert.False(t, needRefresh)
// use a one-off controller so other tests don't have a manual refresh request
ctrl := newFakeController(&fakeData{apps: []runtime.Object{}}, nil)
// refresh app with a non-nil delay
// use zero-second delay to test the add later logic without waiting in the test
delay := time.Duration(0)
ctrl.requestAppRefresh(app.Name, CompareWithRecent.Pointer(), &delay)
ctrl.processAppComparisonTypeQueueItem()
needRefresh, refreshType, compareWith := ctrl.needRefreshAppStatus(app, 1*time.Hour, 2*time.Hour)
assert.True(t, needRefresh)
assert.Equal(t, v1alpha1.RefreshTypeNormal, refreshType)
assert.Equal(t, CompareWithRecent, compareWith)
})
t.Run("refresh application which status is not reconciled using latest commit", func(t *testing.T) {
app := app.DeepCopy()
needRefresh, _, _ := ctrl.needRefreshAppStatus(app, 1*time.Hour, 2*time.Hour)

View File

@@ -69,6 +69,12 @@ const (
// EnvClusterCacheRetryUseBackoff is the env variable to control whether to use a backoff strategy with the retry during cluster cache sync
EnvClusterCacheRetryUseBackoff = "ARGOCD_CLUSTER_CACHE_RETRY_USE_BACKOFF"
// EnvClusterCacheBatchEventsProcessing is the env variable to control whether to enable batch events processing
EnvClusterCacheBatchEventsProcessing = "ARGOCD_CLUSTER_CACHE_BATCH_EVENTS_PROCESSING"
// EnvClusterCacheEventsProcessingInterval is the env variable to control the interval between processing events when BatchEventsProcessing is enabled
EnvClusterCacheEventsProcessingInterval = "ARGOCD_CLUSTER_CACHE_EVENTS_PROCESSING_INTERVAL"
// AnnotationIgnoreResourceUpdates when set to true on an untracked resource,
// argo will apply `ignoreResourceUpdates` configuration on it.
AnnotationIgnoreResourceUpdates = "argocd.argoproj.io/ignore-resource-updates"
@@ -103,6 +109,12 @@ var (
// clusterCacheRetryUseBackoff specifies whether to use a backoff strategy on cluster cache sync, if retry is enabled
clusterCacheRetryUseBackoff bool = false
// clusterCacheBatchEventsProcessing specifies whether to enable batch events processing
clusterCacheBatchEventsProcessing bool = false
// clusterCacheEventsProcessingInterval specifies the interval between processing events when BatchEventsProcessing is enabled
clusterCacheEventsProcessingInterval = 100 * time.Millisecond
)
func init() {
@@ -114,6 +126,8 @@ func init() {
clusterCacheListSemaphoreSize = env.ParseInt64FromEnv(EnvClusterCacheListSemaphore, clusterCacheListSemaphoreSize, 0, math.MaxInt64)
clusterCacheAttemptLimit = int32(env.ParseNumFromEnv(EnvClusterCacheAttemptLimit, int(clusterCacheAttemptLimit), 1, math.MaxInt32))
clusterCacheRetryUseBackoff = env.ParseBoolFromEnv(EnvClusterCacheRetryUseBackoff, false)
clusterCacheBatchEventsProcessing = env.ParseBoolFromEnv(EnvClusterCacheBatchEventsProcessing, false)
clusterCacheEventsProcessingInterval = env.ParseDurationFromEnv(EnvClusterCacheEventsProcessingInterval, clusterCacheEventsProcessingInterval, 0, math.MaxInt64)
}
type LiveStateCache interface {
@@ -554,6 +568,8 @@ func (c *liveStateCache) getCluster(server string) (clustercache.ClusterCache, e
clustercache.SetLogr(logutils.NewLogrusLogger(log.WithField("server", cluster.Server))),
clustercache.SetRetryOptions(clusterCacheAttemptLimit, clusterCacheRetryUseBackoff, isRetryableError),
clustercache.SetRespectRBAC(respectRBAC),
clustercache.SetBatchEventsProcessing(clusterCacheBatchEventsProcessing),
clustercache.SetEventProcessingInterval(clusterCacheEventsProcessingInterval),
}
clusterCache = clustercache.NewClusterCache(clusterCacheConfig, clusterCacheOpts...)
@@ -608,6 +624,10 @@ func (c *liveStateCache) getCluster(server string) (clustercache.ClusterCache, e
c.metricsServer.IncClusterEventsCount(cluster.Server, gvk.Group, gvk.Kind)
})
_ = clusterCache.OnProcessEventsHandler(func(duration time.Duration, processedEventsNumber int) {
c.metricsServer.ObserveResourceEventsProcessingDuration(cluster.Server, duration, processedEventsNumber)
})
c.clusters[server] = clusterCache
return clusterCache, nil

View File

@@ -140,7 +140,7 @@ func TestHandleDeleteEvent_CacheDeadlock(t *testing.T) {
}
db := &dbmocks.ArgoDB{}
db.On("GetApplicationControllerReplicas").Return(1)
fakeClient := fake.NewSimpleClientset()
fakeClient := fake.NewClientset()
settingsMgr := argosettings.NewSettingsManager(context.TODO(), fakeClient, "argocd")
liveStateCacheLock := sync.RWMutex{}
gitopsEngineClusterCache := &mocks.ClusterCache{}

View File

@@ -67,7 +67,7 @@ func TestClusterSecretUpdater(t *testing.T) {
"server.secretkey": nil,
},
}
kubeclientset := fake.NewSimpleClientset(emptyArgoCDConfigMap, argoCDSecret)
kubeclientset := fake.NewClientset(emptyArgoCDConfigMap, argoCDSecret)
appclientset := appsfake.NewSimpleClientset()
appInformer := appinformers.NewApplicationInformer(appclientset, "", time.Minute, cache.Indexers{})
settingsManager := settings.NewSettingsManager(context.Background(), kubeclientset, fakeNamespace)

View File

@@ -51,7 +51,7 @@ func (ctrl *ApplicationController) executePostDeleteHooks(app *v1alpha1.Applicat
revisions = append(revisions, src.TargetRevision)
}
targets, _, _, err := ctrl.appStateManager.GetRepoObjs(app, app.Spec.GetSources(), appLabelKey, revisions, false, false, false, proj, false)
targets, _, _, err := ctrl.appStateManager.GetRepoObjs(app, app.Spec.GetSources(), appLabelKey, revisions, false, false, false, proj, false, true)
if err != nil {
return false, err
}

View File

@@ -0,0 +1,355 @@
package hydrator
import (
"context"
"encoding/json"
"fmt"
"time"
log "github.com/sirupsen/logrus"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
commitclient "github.com/argoproj/argo-cd/v2/commitserver/apiclient"
"github.com/argoproj/argo-cd/v2/controller/utils"
appv1 "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/reposerver/apiclient"
argoio "github.com/argoproj/argo-cd/v2/util/io"
)
// Dependencies is the interface for the dependencies of the Hydrator. It serves two purposes: 1) it prevents the
// hydrator from having direct access to the app controller, and 2) it allows for easy mocking of dependencies in tests.
// If you add something here, be sure that it is something the app controller needs to provide to the hydrator.
type Dependencies interface {
// TODO: determine if we actually need to get the app, or if all the stuff we need the app for is done already on
// the app controller side.
GetProcessableAppProj(app *appv1.Application) (*appv1.AppProject, error)
GetProcessableApps() (*appv1.ApplicationList, error)
GetRepoObjs(app *appv1.Application, source appv1.ApplicationSource, revision string, project *appv1.AppProject) ([]*unstructured.Unstructured, *apiclient.ManifestResponse, error)
GetWriteCredentials(ctx context.Context, repoURL string, project string) (*appv1.Repository, error)
RequestAppRefresh(appName string, appNamespace string) error
// TODO: only allow access to the hydrator status
PersistAppHydratorStatus(orig *appv1.Application, newStatus *appv1.SourceHydratorStatus)
AddHydrationQueueItem(key HydrationQueueKey)
}
type Hydrator struct {
dependencies Dependencies
statusRefreshTimeout time.Duration
commitClientset commitclient.Clientset
}
func NewHydrator(dependencies Dependencies, statusRefreshTimeout time.Duration, commitClientset commitclient.Clientset) *Hydrator {
return &Hydrator{
dependencies: dependencies,
statusRefreshTimeout: statusRefreshTimeout,
commitClientset: commitClientset,
}
}
func (h *Hydrator) ProcessAppHydrateQueueItem(origApp *appv1.Application) {
origApp = origApp.DeepCopy()
app := origApp.DeepCopy()
if app.Spec.SourceHydrator == nil {
return
}
logCtx := utils.GetAppLog(app)
logCtx.Debug("Processing app hydrate queue item")
// TODO: don't reuse statusRefreshTimeout. Create a new timeout for hydration.
needsHydration, reason := appNeedsHydration(origApp, h.statusRefreshTimeout)
if !needsHydration {
return
}
logCtx.WithField("reason", reason).Info("Hydrating app")
app.Status.SourceHydrator.CurrentOperation = &appv1.HydrateOperation{
StartedAt: metav1.Now(),
FinishedAt: nil,
Phase: appv1.HydrateOperationPhaseHydrating,
SourceHydrator: *app.Spec.SourceHydrator,
}
h.dependencies.PersistAppHydratorStatus(origApp, &app.Status.SourceHydrator)
origApp.Status.SourceHydrator = app.Status.SourceHydrator
h.dependencies.AddHydrationQueueItem(getHydrationQueueKey(app))
logCtx.Debug("Successfully processed app hydrate queue item")
}
func getHydrationQueueKey(app *appv1.Application) HydrationQueueKey {
destinationBranch := app.Spec.SourceHydrator.SyncSource.TargetBranch
if app.Spec.SourceHydrator.HydrateTo != nil {
destinationBranch = app.Spec.SourceHydrator.HydrateTo.TargetBranch
}
key := HydrationQueueKey{
SourceRepoURL: app.Spec.SourceHydrator.DrySource.RepoURL,
SourceTargetRevision: app.Spec.SourceHydrator.DrySource.TargetRevision,
DestinationBranch: destinationBranch,
}
return key
}
type HydrationQueueKey struct {
SourceRepoURL string
SourceTargetRevision string
DestinationBranch string
}
// uniqueHydrationDestination is used to detect duplicate hydrate destinations.
type uniqueHydrationDestination struct {
sourceRepoURL string
sourceTargetRevision string
destinationBranch string
destinationPath string
}
func (h *Hydrator) ProcessHydrationQueueItem(hydrationKey HydrationQueueKey) (processNext bool) {
logCtx := log.WithFields(log.Fields{
"sourceRepoURL": hydrationKey.SourceRepoURL,
"sourceTargetRevision": hydrationKey.SourceTargetRevision,
"destinationBranch": hydrationKey.DestinationBranch,
})
relevantApps, drySHA, hydratedSHA, err := h.hydrateAppsLatestCommit(logCtx, hydrationKey)
if drySHA != "" {
logCtx = logCtx.WithField("drySHA", drySHA)
}
if err != nil {
logCtx.WithField("appCount", len(relevantApps)).WithError(err).Error("Failed to hydrate apps")
for _, app := range relevantApps {
origApp := app.DeepCopy()
app.Status.SourceHydrator.CurrentOperation.Phase = appv1.HydrateOperationPhaseFailed
failedAt := metav1.Now()
app.Status.SourceHydrator.CurrentOperation.FinishedAt = &failedAt
app.Status.SourceHydrator.CurrentOperation.Message = fmt.Sprintf("Failed to hydrate revision %q: %v", drySHA, err.Error())
// We may or may not have gotten far enough in the hydration process to get a non-empty SHA, but set it just
// in case we did.
app.Status.SourceHydrator.CurrentOperation.DrySHA = drySHA
h.dependencies.PersistAppHydratorStatus(origApp, &app.Status.SourceHydrator)
logCtx = logCtx.WithField("app", app.QualifiedName())
logCtx.Errorf("Failed to hydrate app: %v", err)
}
return
}
logCtx.WithField("appCount", len(relevantApps)).Debug("Successfully hydrated apps")
finishedAt := metav1.Now()
for _, app := range relevantApps {
origApp := app.DeepCopy()
operation := &appv1.HydrateOperation{
StartedAt: app.Status.SourceHydrator.CurrentOperation.StartedAt,
FinishedAt: &finishedAt,
Phase: appv1.HydrateOperationPhaseHydrated,
Message: "",
DrySHA: drySHA,
HydratedSHA: hydratedSHA,
SourceHydrator: app.Status.SourceHydrator.CurrentOperation.SourceHydrator,
}
app.Status.SourceHydrator.CurrentOperation = operation
app.Status.SourceHydrator.LastSuccessfulOperation = &appv1.SuccessfulHydrateOperation{
DrySHA: drySHA,
HydratedSHA: hydratedSHA,
SourceHydrator: app.Status.SourceHydrator.CurrentOperation.SourceHydrator,
}
h.dependencies.PersistAppHydratorStatus(origApp, &app.Status.SourceHydrator)
// Request a refresh since we pushed a new commit.
err := h.dependencies.RequestAppRefresh(app.Name, app.Namespace)
if err != nil {
logCtx.WithField("app", app.QualifiedName()).WithError(err).Error("Failed to request app refresh after hydration")
}
}
return
}
func (h *Hydrator) hydrateAppsLatestCommit(logCtx *log.Entry, hydrationKey HydrationQueueKey) ([]*appv1.Application, string, string, error) {
relevantApps, err := h.getRelevantAppsForHydration(logCtx, hydrationKey)
if err != nil {
return nil, "", "", fmt.Errorf("failed to get relevant apps for hydration: %w", err)
}
dryRevision, hydratedRevision, err := h.hydrate(logCtx, relevantApps)
if err != nil {
return relevantApps, dryRevision, "", fmt.Errorf("failed to hydrate apps: %w", err)
}
return relevantApps, dryRevision, hydratedRevision, nil
}
func (h *Hydrator) getRelevantAppsForHydration(logCtx *log.Entry, hydrationKey HydrationQueueKey) ([]*appv1.Application, error) {
// Get all apps
apps, err := h.dependencies.GetProcessableApps()
if err != nil {
return nil, fmt.Errorf("failed to list apps: %w", err)
}
var relevantApps []*appv1.Application
uniqueDestinations := make(map[uniqueHydrationDestination]bool, len(apps.Items))
for _, app := range apps.Items {
if app.Spec.SourceHydrator == nil {
continue
}
if app.Spec.SourceHydrator.DrySource.RepoURL != hydrationKey.SourceRepoURL ||
app.Spec.SourceHydrator.DrySource.TargetRevision != hydrationKey.SourceTargetRevision {
continue
}
destinationBranch := app.Spec.SourceHydrator.SyncSource.TargetBranch
if app.Spec.SourceHydrator.HydrateTo != nil {
destinationBranch = app.Spec.SourceHydrator.HydrateTo.TargetBranch
}
if destinationBranch != hydrationKey.DestinationBranch {
continue
}
var proj *appv1.AppProject
proj, err = h.dependencies.GetProcessableAppProj(&app)
if err != nil {
return nil, fmt.Errorf("failed to get project %q for app %q: %w", app.Spec.Project, app.QualifiedName(), err)
}
permitted := proj.IsSourcePermitted(app.Spec.GetSource())
if !permitted {
// Log and skip. We don't want to fail the entire operation because of one app.
logCtx.Warnf("App %q is not permitted to use source %q", app.QualifiedName(), app.Spec.Source.String())
continue
}
uniqueDestinationKey := uniqueHydrationDestination{
sourceRepoURL: app.Spec.SourceHydrator.DrySource.RepoURL,
sourceTargetRevision: app.Spec.SourceHydrator.DrySource.TargetRevision,
destinationBranch: destinationBranch,
destinationPath: app.Spec.SourceHydrator.SyncSource.Path,
}
// TODO: test the dupe detection
if _, ok := uniqueDestinations[uniqueDestinationKey]; ok {
return nil, fmt.Errorf("multiple app hydrators use the same destination: %v", uniqueDestinationKey)
}
uniqueDestinations[uniqueDestinationKey] = true
relevantApps = append(relevantApps, &app)
}
return relevantApps, nil
}
func (h *Hydrator) hydrate(logCtx *log.Entry, apps []*appv1.Application) (string, string, error) {
if len(apps) == 0 {
return "", "", nil
}
repoURL := apps[0].Spec.SourceHydrator.DrySource.RepoURL
syncBranch := apps[0].Spec.SourceHydrator.SyncSource.TargetBranch
targetBranch := apps[0].Spec.GetHydrateToSource().TargetRevision
var paths []*commitclient.PathDetails
projects := make(map[string]bool, len(apps))
var targetRevision string
// TODO: parallelize this loop
for _, app := range apps {
project, err := h.dependencies.GetProcessableAppProj(app)
if err != nil {
return "", "", fmt.Errorf("failed to get project: %w", err)
}
projects[project.Name] = true
drySource := appv1.ApplicationSource{
RepoURL: app.Spec.SourceHydrator.DrySource.RepoURL,
Path: app.Spec.SourceHydrator.DrySource.Path,
TargetRevision: app.Spec.SourceHydrator.DrySource.TargetRevision,
}
if targetRevision == "" {
targetRevision = app.Spec.SourceHydrator.DrySource.TargetRevision
}
// TODO: enable signature verification
objs, resp, err := h.dependencies.GetRepoObjs(app, drySource, targetRevision, project)
if err != nil {
return "", "", fmt.Errorf("failed to get repo objects: %w", err)
}
// This should be the DRY SHA. We set it here so that after processing the first app, all apps are hydrated
// using the same SHA.
targetRevision = resp.Revision
// Set up a ManifestsRequest
manifestDetails := make([]*commitclient.HydratedManifestDetails, len(objs))
for i, obj := range objs {
objJson, err := json.Marshal(obj)
if err != nil {
return "", "", fmt.Errorf("failed to marshal object: %w", err)
}
manifestDetails[i] = &commitclient.HydratedManifestDetails{ManifestJSON: string(objJson)}
}
paths = append(paths, &commitclient.PathDetails{
Path: app.Spec.SourceHydrator.SyncSource.Path,
Manifests: manifestDetails,
Commands: resp.Commands,
})
}
// If all the apps are under the same project, use that project. Otherwise, use an empty string to indicate that we
// need global creds.
project := ""
if len(projects) == 1 {
for p := range projects {
project = p
}
}
repo, err := h.dependencies.GetWriteCredentials(context.Background(), repoURL, project)
if err != nil {
return "", "", fmt.Errorf("failed to get hydrator credentials: %w", err)
}
if repo == nil {
// Try without credentials.
repo = &appv1.Repository{
Repo: repoURL,
}
logCtx.Warn("no credentials found for repo, continuing without credentials")
}
manifestsRequest := commitclient.CommitHydratedManifestsRequest{
Repo: repo,
SyncBranch: syncBranch,
TargetBranch: targetBranch,
DrySha: targetRevision,
CommitMessage: fmt.Sprintf("[Argo CD Bot] hydrate %s", targetRevision),
Paths: paths,
}
closer, commitService, err := h.commitClientset.NewCommitServerClient()
if err != nil {
return targetRevision, "", fmt.Errorf("failed to create commit service: %w", err)
}
defer argoio.Close(closer)
resp, err := commitService.CommitHydratedManifests(context.Background(), &manifestsRequest)
if err != nil {
return targetRevision, "", fmt.Errorf("failed to commit hydrated manifests: %w", err)
}
return targetRevision, resp.HydratedSha, nil
}
// appNeedsHydration answers if application needs manifests hydrated.
func appNeedsHydration(app *appv1.Application, statusHydrateTimeout time.Duration) (needsHydration bool, reason string) {
if app.Spec.SourceHydrator == nil {
return false, "source hydrator not configured"
}
var hydratedAt *metav1.Time
if app.Status.SourceHydrator.CurrentOperation != nil {
hydratedAt = &app.Status.SourceHydrator.CurrentOperation.StartedAt
}
if app.IsHydrateRequested() {
return true, "hydrate requested"
} else if app.Status.SourceHydrator.CurrentOperation == nil {
return true, "no previous hydrate operation"
} else if !app.Spec.SourceHydrator.DeepEquals(app.Status.SourceHydrator.CurrentOperation.SourceHydrator) {
return true, "spec.sourceHydrator differs"
} else if app.Status.SourceHydrator.CurrentOperation.Phase == appv1.HydrateOperationPhaseFailed && metav1.Now().Sub(app.Status.SourceHydrator.CurrentOperation.FinishedAt.Time) > 2*time.Minute {
return true, "previous hydrate operation failed more than 2 minutes ago"
} else if hydratedAt == nil || hydratedAt.Add(statusHydrateTimeout).Before(time.Now().UTC()) {
return true, "hydration expired"
}
return false, ""
}

View File

@@ -0,0 +1,103 @@
package hydrator
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
)
func Test_appNeedsHydration(t *testing.T) {
t.Parallel()
now := metav1.NewTime(time.Now())
oneHourAgo := metav1.NewTime(now.Add(-1 * time.Hour))
testCases := []struct {
name string
app *v1alpha1.Application
timeout time.Duration
expectedNeedsHydration bool
expectedMessage string
}{
{
name: "source hydrator not configured",
app: &v1alpha1.Application{},
expectedNeedsHydration: false,
expectedMessage: "source hydrator not configured",
},
{
name: "hydrate requested",
app: &v1alpha1.Application{
ObjectMeta: metav1.ObjectMeta{Annotations: map[string]string{v1alpha1.AnnotationKeyHydrate: "normal"}},
Spec: v1alpha1.ApplicationSpec{SourceHydrator: &v1alpha1.SourceHydrator{}},
},
timeout: 1 * time.Hour,
expectedNeedsHydration: true,
expectedMessage: "hydrate requested",
},
{
name: "no previous hydrate operation",
app: &v1alpha1.Application{
Spec: v1alpha1.ApplicationSpec{SourceHydrator: &v1alpha1.SourceHydrator{}},
},
timeout: 1 * time.Hour,
expectedNeedsHydration: true,
expectedMessage: "no previous hydrate operation",
},
{
name: "spec.sourceHydrator differs",
app: &v1alpha1.Application{
Spec: v1alpha1.ApplicationSpec{SourceHydrator: &v1alpha1.SourceHydrator{}},
Status: v1alpha1.ApplicationStatus{SourceHydrator: v1alpha1.SourceHydratorStatus{CurrentOperation: &v1alpha1.HydrateOperation{
SourceHydrator: v1alpha1.SourceHydrator{DrySource: v1alpha1.DrySource{RepoURL: "something new"}},
}}},
},
timeout: 1 * time.Hour,
expectedNeedsHydration: true,
expectedMessage: "spec.sourceHydrator differs",
},
{
name: "hydration failed more than two minutes ago",
app: &v1alpha1.Application{
Spec: v1alpha1.ApplicationSpec{SourceHydrator: &v1alpha1.SourceHydrator{}},
Status: v1alpha1.ApplicationStatus{SourceHydrator: v1alpha1.SourceHydratorStatus{CurrentOperation: &v1alpha1.HydrateOperation{DrySHA: "abc123", FinishedAt: &oneHourAgo, Phase: v1alpha1.HydrateOperationPhaseFailed}}},
},
timeout: 1 * time.Hour,
expectedNeedsHydration: true,
expectedMessage: "previous hydrate operation failed more than 2 minutes ago",
},
{
name: "timeout reached",
app: &v1alpha1.Application{
Spec: v1alpha1.ApplicationSpec{SourceHydrator: &v1alpha1.SourceHydrator{}},
Status: v1alpha1.ApplicationStatus{SourceHydrator: v1alpha1.SourceHydratorStatus{CurrentOperation: &v1alpha1.HydrateOperation{StartedAt: oneHourAgo}}},
},
timeout: 1 * time.Minute,
expectedNeedsHydration: true,
expectedMessage: "hydration expired",
},
{
name: "hydrate not needed",
app: &v1alpha1.Application{
Spec: v1alpha1.ApplicationSpec{SourceHydrator: &v1alpha1.SourceHydrator{}},
Status: v1alpha1.ApplicationStatus{SourceHydrator: v1alpha1.SourceHydratorStatus{CurrentOperation: &v1alpha1.HydrateOperation{DrySHA: "abc123", StartedAt: now, FinishedAt: &now, Phase: v1alpha1.HydrateOperationPhaseFailed}}},
},
timeout: 1 * time.Hour,
expectedNeedsHydration: false,
expectedMessage: "",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
needsHydration, message := appNeedsHydration(tc.app, tc.timeout)
assert.Equal(t, tc.expectedNeedsHydration, needsHydration)
assert.Equal(t, tc.expectedMessage, message)
})
}
}

View File

@@ -0,0 +1,90 @@
package controller
import (
"context"
"fmt"
"github.com/argoproj/argo-cd/v2/controller/hydrator"
appv1 "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/reposerver/apiclient"
argoutil "github.com/argoproj/argo-cd/v2/util/argo"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)
/**
This file implements the hydrator.Dependencies interface for the ApplicationController.
Hydration logic does not belong in this file. The methods here should be "bookkeeping" methods that keep hydration work
in the hydrator and app controller work in the app controller. The only purpose of this file is to provide the hydrator
safe, minimal access to certain app controller functionality to avoid duplicate code.
*/
func (ctrl *ApplicationController) GetProcessableAppProj(app *appv1.Application) (*appv1.AppProject, error) {
return ctrl.getAppProj(app)
}
// GetProcessableApps returns a list of applications that are processable by the controller.
func (ctrl *ApplicationController) GetProcessableApps() (*appv1.ApplicationList, error) {
// getAppList already filters out applications that are not processable by the controller.
return ctrl.getAppList(metav1.ListOptions{})
}
func (ctrl *ApplicationController) GetRepoObjs(origApp *appv1.Application, drySource appv1.ApplicationSource, revision string, project *appv1.AppProject) ([]*unstructured.Unstructured, *apiclient.ManifestResponse, error) {
drySources := []appv1.ApplicationSource{drySource}
dryRevisions := []string{revision}
appLabelKey, err := ctrl.settingsMgr.GetAppInstanceLabelKey()
if err != nil {
return nil, nil, fmt.Errorf("failed to get app instance label key: %w", err)
}
app := origApp.DeepCopy()
// Remove the manifest generate path annotation, because the feature will misbehave for apps using source hydrator.
// Setting this annotation causes GetRepoObjs to compare the dry source commit to the most recent synced commit. The
// problem is that the most recent synced commit is likely on the hydrated branch, not the dry branch. The
// comparison will throw an error and break hydration.
//
// The long-term solution will probably be to persist the synced _dry_ revision and use that for the comparison.
delete(app.Annotations, appv1.AnnotationKeyManifestGeneratePaths)
// FIXME: use cache and revision cache
objs, resp, _, err := ctrl.appStateManager.GetRepoObjs(app, drySources, appLabelKey, dryRevisions, true, true, false, project, false, false)
if err != nil {
return nil, nil, fmt.Errorf("failed to get repo objects: %w", err)
}
if len(resp) != 1 {
return nil, nil, fmt.Errorf("expected one manifest response, got %d", len(resp))
}
return objs, resp[0], nil
}
func (ctrl *ApplicationController) GetWriteCredentials(ctx context.Context, repoURL string, project string) (*appv1.Repository, error) {
return ctrl.db.GetWriteRepository(ctx, repoURL, project)
}
func (ctrl *ApplicationController) RequestAppRefresh(appName string, appNamespace string) error {
// We request a refresh by setting the annotation instead of by adding it to the refresh queue, because there is no
// guarantee that the hydrator is running on the same controller shard as is processing the application.
// This function is called for each app after a hydrate operation is completed so that the app controller can pick
// up the newly-hydrated changes. So we set hydrate=false to avoid a hydrate loop.
_, err := argoutil.RefreshApp(ctrl.applicationClientset.ArgoprojV1alpha1().Applications(appNamespace), appName, appv1.RefreshTypeNormal, false)
if err != nil {
return fmt.Errorf("failed to request app refresh: %w", err)
}
return nil
}
func (ctrl *ApplicationController) PersistAppHydratorStatus(orig *appv1.Application, newStatus *appv1.SourceHydratorStatus) {
status := orig.Status.DeepCopy()
status.SourceHydrator = *newStatus
ctrl.persistAppStatus(orig, status)
}
func (ctrl *ApplicationController) AddHydrationQueueItem(key hydrator.HydrationQueueKey) {
ctrl.hydrationQueue.AddRateLimited(key)
}

View File

@@ -30,18 +30,20 @@ import (
type MetricsServer struct {
*http.Server
syncCounter *prometheus.CounterVec
kubectlExecCounter *prometheus.CounterVec
kubectlExecPendingGauge *prometheus.GaugeVec
orphanedResourcesGauge *prometheus.GaugeVec
k8sRequestCounter *prometheus.CounterVec
clusterEventsCounter *prometheus.CounterVec
redisRequestCounter *prometheus.CounterVec
reconcileHistogram *prometheus.HistogramVec
redisRequestHistogram *prometheus.HistogramVec
registry *prometheus.Registry
hostname string
cron *cron.Cron
syncCounter *prometheus.CounterVec
kubectlExecCounter *prometheus.CounterVec
kubectlExecPendingGauge *prometheus.GaugeVec
orphanedResourcesGauge *prometheus.GaugeVec
k8sRequestCounter *prometheus.CounterVec
clusterEventsCounter *prometheus.CounterVec
redisRequestCounter *prometheus.CounterVec
reconcileHistogram *prometheus.HistogramVec
redisRequestHistogram *prometheus.HistogramVec
resourceEventsProcessingHistogram *prometheus.HistogramVec
resourceEventsNumberGauge *prometheus.GaugeVec
registry *prometheus.Registry
hostname string
cron *cron.Cron
}
const (
@@ -153,6 +155,20 @@ var (
},
descAppDefaultLabels,
)
resourceEventsProcessingHistogram = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "argocd_resource_events_processing",
Help: "Time to process resource events in seconds.",
Buckets: []float64{0.25, .5, 1, 2, 4, 8, 16},
},
[]string{"server"},
)
resourceEventsNumberGauge = prometheus.NewGaugeVec(prometheus.GaugeOpts{
Name: "argocd_resource_events_processed_in_batch",
Help: "Number of resource events processed in batch",
}, []string{"server"})
)
// NewMetricsServer returns a new prometheus server which collects application metrics
@@ -202,6 +218,8 @@ func NewMetricsServer(addr string, appLister applister.ApplicationLister, appFil
registry.MustRegister(clusterEventsCounter)
registry.MustRegister(redisRequestCounter)
registry.MustRegister(redisRequestHistogram)
registry.MustRegister(resourceEventsProcessingHistogram)
registry.MustRegister(resourceEventsNumberGauge)
return &MetricsServer{
registry: registry,
@@ -209,16 +227,18 @@ func NewMetricsServer(addr string, appLister applister.ApplicationLister, appFil
Addr: addr,
Handler: mux,
},
syncCounter: syncCounter,
k8sRequestCounter: k8sRequestCounter,
kubectlExecCounter: kubectlExecCounter,
kubectlExecPendingGauge: kubectlExecPendingGauge,
orphanedResourcesGauge: orphanedResourcesGauge,
reconcileHistogram: reconcileHistogram,
clusterEventsCounter: clusterEventsCounter,
redisRequestCounter: redisRequestCounter,
redisRequestHistogram: redisRequestHistogram,
hostname: hostname,
syncCounter: syncCounter,
k8sRequestCounter: k8sRequestCounter,
kubectlExecCounter: kubectlExecCounter,
kubectlExecPendingGauge: kubectlExecPendingGauge,
orphanedResourcesGauge: orphanedResourcesGauge,
reconcileHistogram: reconcileHistogram,
clusterEventsCounter: clusterEventsCounter,
redisRequestCounter: redisRequestCounter,
redisRequestHistogram: redisRequestHistogram,
resourceEventsProcessingHistogram: resourceEventsProcessingHistogram,
resourceEventsNumberGauge: resourceEventsNumberGauge,
hostname: hostname,
// This cron is used to expire the metrics cache.
// Currently clearing the metrics cache is logging and deleting from the map
// so there is no possibility of panic, but we will add a chain to keep robfig/cron v1 behavior.
@@ -284,6 +304,12 @@ func (m *MetricsServer) ObserveRedisRequestDuration(duration time.Duration) {
m.redisRequestHistogram.WithLabelValues(m.hostname, common.ApplicationController).Observe(duration.Seconds())
}
// ObserveResourceEventsProcessingDuration observes resource events processing duration
func (m *MetricsServer) ObserveResourceEventsProcessingDuration(server string, duration time.Duration, processedEventsNumber int) {
m.resourceEventsProcessingHistogram.WithLabelValues(server).Observe(duration.Seconds())
m.resourceEventsNumberGauge.WithLabelValues(server).Set(float64(processedEventsNumber))
}
// IncReconcile increments the reconcile counter for an application
func (m *MetricsServer) IncReconcile(app *argoappv1.Application, duration time.Duration) {
m.reconcileHistogram.WithLabelValues(app.Namespace, app.Spec.Destination.Server).Observe(duration.Seconds())
@@ -311,6 +337,8 @@ func (m *MetricsServer) SetExpiration(cacheExpiration time.Duration) error {
m.redisRequestCounter.Reset()
m.reconcileHistogram.Reset()
m.redisRequestHistogram.Reset()
m.resourceEventsProcessingHistogram.Reset()
m.resourceEventsNumberGauge.Reset()
})
if err != nil {
return err

View File

@@ -71,7 +71,7 @@ type managedResource struct {
type AppStateManager interface {
CompareAppState(app *v1alpha1.Application, project *v1alpha1.AppProject, revisions []string, sources []v1alpha1.ApplicationSource, noCache bool, noRevisionCache bool, localObjects []string, hasMultipleSources bool, rollback bool) (*comparisonResult, error)
SyncAppState(app *v1alpha1.Application, state *v1alpha1.OperationState)
GetRepoObjs(app *v1alpha1.Application, sources []v1alpha1.ApplicationSource, appLabelKey string, revisions []string, noCache, noRevisionCache, verifySignature bool, proj *v1alpha1.AppProject, rollback bool) ([]*unstructured.Unstructured, []*apiclient.ManifestResponse, bool, error)
GetRepoObjs(app *v1alpha1.Application, sources []v1alpha1.ApplicationSource, appLabelKey string, revisions []string, noCache, noRevisionCache, verifySignature bool, proj *v1alpha1.AppProject, rollback, sendRuntimeState bool) ([]*unstructured.Unstructured, []*apiclient.ManifestResponse, bool, error)
}
// comparisonResult holds the state of an application after the reconciliation
@@ -108,6 +108,7 @@ type appStateManager struct {
appclientset appclientset.Interface
projInformer cache.SharedIndexInformer
kubectl kubeutil.Kubectl
onKubectlRun kubeutil.OnKubectlRunFunc
repoClientset apiclient.Clientset
liveStateCache statecache.LiveStateCache
cache *appstatecache.Cache
@@ -125,7 +126,7 @@ type appStateManager struct {
// task to the repo-server. It returns the list of generated manifests as unstructured
// objects. It also returns the full response from all calls to the repo server as the
// second argument.
func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alpha1.ApplicationSource, appLabelKey string, revisions []string, noCache, noRevisionCache, verifySignature bool, proj *v1alpha1.AppProject, rollback bool) ([]*unstructured.Unstructured, []*apiclient.ManifestResponse, bool, error) {
func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alpha1.ApplicationSource, appLabelKey string, revisions []string, noCache, noRevisionCache, verifySignature bool, proj *v1alpha1.AppProject, rollback, sendRuntimeState bool) ([]*unstructured.Unstructured, []*apiclient.ManifestResponse, bool, error) {
ts := stats.NewTimingStats()
helmRepos, err := m.db.ListHelmRepositories(context.Background())
if err != nil {
@@ -168,9 +169,13 @@ func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alp
}
ts.AddCheckpoint("build_options_ms")
serverVersion, apiResources, err := m.liveStateCache.GetVersionsInfo(app.Spec.Destination.Server)
if err != nil {
return nil, nil, false, fmt.Errorf("failed to get cluster version for cluster %q: %w", app.Spec.Destination.Server, err)
var serverVersion string
var apiResources []kubeutil.APIResourceInfo
if sendRuntimeState {
serverVersion, apiResources, err = m.liveStateCache.GetVersionsInfo(app.Spec.Destination.Server)
if err != nil {
return nil, nil, false, fmt.Errorf("failed to get cluster version for cluster %q: %w", app.Spec.Destination.Server, err)
}
}
conn, repoClient, err := m.repoClientset.NewRepoServerClient()
if err != nil {
@@ -219,6 +224,12 @@ func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alp
revision := revisions[i]
appNamespace := app.Spec.Destination.Namespace
apiVersions := argo.APIResourcesToStrings(apiResources, true)
if !sendRuntimeState {
appNamespace = ""
}
if !source.IsHelm() && syncedRevision != "" && keyManifestGenerateAnnotationExists && keyManifestGenerateAnnotationVal != "" {
// Validate the manifest-generate-path annotation to avoid generating manifests if it has not changed.
updateRevisionResult, err := repoClient.UpdateRevisionForPaths(context.Background(), &apiclient.UpdateRevisionForPathsRequest{
@@ -229,10 +240,10 @@ func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alp
Paths: path.GetAppRefreshPaths(app),
AppLabelKey: appLabelKey,
AppName: app.InstanceName(m.namespace),
Namespace: app.Spec.Destination.Namespace,
Namespace: appNamespace,
ApplicationSource: &source,
KubeVersion: serverVersion,
ApiVersions: argo.APIResourcesToStrings(apiResources, true),
ApiVersions: apiVersions,
TrackingMethod: string(argo.GetTrackingMethod(m.settingsMgr)),
RefSources: refSources,
HasMultipleSources: app.Spec.HasMultipleSources(),
@@ -263,11 +274,11 @@ func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alp
NoRevisionCache: noRevisionCache,
AppLabelKey: appLabelKey,
AppName: app.InstanceName(m.namespace),
Namespace: app.Spec.Destination.Namespace,
Namespace: appNamespace,
ApplicationSource: &source,
KustomizeOptions: kustomizeOptions,
KubeVersion: serverVersion,
ApiVersions: argo.APIResourcesToStrings(apiResources, true),
ApiVersions: apiVersions,
VerifySignature: verifySignature,
HelmRepoCreds: permittedHelmCredentials,
TrackingMethod: string(argo.GetTrackingMethod(m.settingsMgr)),
@@ -309,6 +320,39 @@ func (m *appStateManager) GetRepoObjs(app *v1alpha1.Application, sources []v1alp
return targetObjs, manifestInfos, revisionUpdated, nil
}
// ResolveGitRevision will resolve the given revision to a full commit SHA. Only works for git.
func (m *appStateManager) ResolveGitRevision(repoURL string, revision string) (string, error) {
conn, repoClient, err := m.repoClientset.NewRepoServerClient()
if err != nil {
return "", fmt.Errorf("failed to connect to repo server: %w", err)
}
defer io.Close(conn)
repo, err := m.db.GetRepository(context.Background(), repoURL, "")
if err != nil {
return "", fmt.Errorf("failed to get repo %q: %w", repoURL, err)
}
// Mock the app. The repo-server only needs to know whether the "chart" field is populated.
app := &v1alpha1.Application{
Spec: v1alpha1.ApplicationSpec{
Source: &v1alpha1.ApplicationSource{
RepoURL: repoURL,
TargetRevision: revision,
},
},
}
resp, err := repoClient.ResolveRevision(context.Background(), &apiclient.ResolveRevisionRequest{
Repo: repo,
App: app,
AmbiguousRevision: revision,
})
if err != nil {
return "", fmt.Errorf("failed to determine whether the dry source has changed: %w", err)
}
return resp.Revision, nil
}
func unmarshalManifests(manifests []string) ([]*unstructured.Unstructured, error) {
targetObjs := make([]*unstructured.Unstructured, 0)
for _, manifest := range manifests {
@@ -441,7 +485,7 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *v1
if hasMultipleSources {
return &comparisonResult{
syncStatus: &v1alpha1.SyncStatus{
ComparedTo: v1alpha1.ComparedTo{Destination: app.Spec.Destination, Sources: sources, IgnoreDifferences: app.Spec.IgnoreDifferences},
ComparedTo: app.Spec.BuildComparedToStatus(),
Status: v1alpha1.SyncStatusCodeUnknown,
Revisions: revisions,
},
@@ -450,7 +494,7 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *v1
} else {
return &comparisonResult{
syncStatus: &v1alpha1.SyncStatus{
ComparedTo: v1alpha1.ComparedTo{Source: sources[0], Destination: app.Spec.Destination, IgnoreDifferences: app.Spec.IgnoreDifferences},
ComparedTo: app.Spec.BuildComparedToStatus(),
Status: v1alpha1.SyncStatusCodeUnknown,
Revision: revisions[0],
},
@@ -490,7 +534,7 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *v1
}
}
targetObjs, manifestInfos, revisionUpdated, err = m.GetRepoObjs(app, sources, appLabelKey, revisions, noCache, noRevisionCache, verifySignature, project, rollback)
targetObjs, manifestInfos, revisionUpdated, err = m.GetRepoObjs(app, sources, appLabelKey, revisions, noCache, noRevisionCache, verifySignature, project, rollback, true)
if err != nil {
targetObjs = make([]*unstructured.Unstructured, 0)
msg := fmt.Sprintf("Failed to load target state: %s", err.Error())
@@ -698,13 +742,13 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *v1
diffConfigBuilder.WithServerSideDiff(serverSideDiff)
if serverSideDiff {
resourceOps, cleanup, err := m.getResourceOperations(app.Spec.Destination.Server)
applier, cleanup, err := m.getServerSideDiffDryRunApplier(app.Spec.Destination.Server)
if err != nil {
log.Errorf("CompareAppState error getting resource operations: %s", err)
log.Errorf("CompareAppState error getting server side diff dry run applier: %s", err)
conditions = append(conditions, v1alpha1.ApplicationCondition{Type: v1alpha1.ApplicationConditionUnknownError, Message: err.Error(), LastTransitionTime: &now})
}
defer cleanup()
diffConfigBuilder.WithServerSideDryRunner(diff.NewK8sServerSideDryRunner(resourceOps))
diffConfigBuilder.WithServerSideDryRunner(diff.NewK8sServerSideDryRunner(applier))
}
// enable structured merge diff if application syncs with server-side apply
@@ -1029,6 +1073,7 @@ func NewAppStateManager(
repoClientset apiclient.Clientset,
namespace string,
kubectl kubeutil.Kubectl,
onKubectlRun kubeutil.OnKubectlRunFunc,
settingsMgr *settings.SettingsManager,
liveStateCache statecache.LiveStateCache,
projInformer cache.SharedIndexInformer,
@@ -1047,6 +1092,7 @@ func NewAppStateManager(
db: db,
appclientset: appclientset,
kubectl: kubectl,
onKubectlRun: onKubectlRun,
repoClientset: repoClientset,
namespace: namespace,
settingsMgr: settingsMgr,

View File

@@ -14,6 +14,7 @@ import (
cdcommon "github.com/argoproj/argo-cd/v2/common"
gitopsDiff "github.com/argoproj/gitops-engine/pkg/diff"
"github.com/argoproj/gitops-engine/pkg/sync"
"github.com/argoproj/gitops-engine/pkg/sync/common"
"github.com/argoproj/gitops-engine/pkg/utils/kube"
@@ -33,6 +34,7 @@ import (
"github.com/argoproj/argo-cd/v2/util/argo"
"github.com/argoproj/argo-cd/v2/util/argo/diff"
"github.com/argoproj/argo-cd/v2/util/glob"
kubeutil "github.com/argoproj/argo-cd/v2/util/kube"
logutils "github.com/argoproj/argo-cd/v2/util/log"
"github.com/argoproj/argo-cd/v2/util/lua"
"github.com/argoproj/argo-cd/v2/util/rand"
@@ -66,11 +68,11 @@ func (m *appStateManager) getGVKParser(server string) (*managedfields.GvkParser,
return cluster.GetGVKParser(), nil
}
// getResourceOperations will return the kubectl implementation of the ResourceOperations
// interface that provides functionality to manage kubernetes resources. Returns a
// getServerSideDiffDryRunApplier will return the kubectl implementation of the KubeApplier
// interface that provides functionality to dry run apply kubernetes resources. Returns a
// cleanup function that must be called to remove the generated kube config for this
// server.
func (m *appStateManager) getResourceOperations(server string) (kube.ResourceOperations, func(), error) {
func (m *appStateManager) getServerSideDiffDryRunApplier(server string) (gitopsDiff.KubeApplier, func(), error) {
clusterCache, err := m.liveStateCache.GetClusterCache(server)
if err != nil {
return nil, nil, fmt.Errorf("error getting cluster cache: %w", err)
@@ -85,7 +87,7 @@ func (m *appStateManager) getResourceOperations(server string) (kube.ResourceOpe
if err != nil {
return nil, nil, fmt.Errorf("error getting cluster REST config: %w", err)
}
ops, cleanup, err := m.kubectl.ManageResources(rawConfig, clusterCache.GetOpenAPISchema())
ops, cleanup, err := kubeutil.ManageServerSideDiffDryRuns(rawConfig, clusterCache.GetOpenAPISchema(), m.onKubectlRun)
if err != nil {
return nil, nil, fmt.Errorf("error creating kubectl ResourceOperations: %w", err)
}

View File

@@ -56,7 +56,7 @@ func TestPersistRevisionHistory(t *testing.T) {
updatedApp, err := ctrl.applicationClientset.ArgoprojV1alpha1().Applications(app.Namespace).Get(context.Background(), app.Name, v1.GetOptions{})
require.NoError(t, err)
assert.Len(t, updatedApp.Status.History, 1)
require.Len(t, updatedApp.Status.History, 1)
assert.Equal(t, app.Spec.GetSource(), updatedApp.Status.History[0].Source)
assert.Equal(t, "abc123", updatedApp.Status.History[0].Revision)
}

17
controller/utils/log.go Normal file
View File

@@ -0,0 +1,17 @@
package utils
import (
"github.com/sirupsen/logrus"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
)
// GetAppLog returns a logrus entry with fields set for the given application.
func GetAppLog(app *v1alpha1.Application) *logrus.Entry {
return logrus.WithFields(logrus.Fields{
"application": app.Name,
"app-namespace": app.Namespace,
"app-qualified-name": app.QualifiedName(),
"project": app.Spec.Project,
})
}

View File

@@ -184,12 +184,11 @@ the argocd-secret with key 'some.argocd.secret.key'.
If provided, and multiple services are configured, will have to match
the application destination name or server to have requests properly
forwarded to this service URL. If there are multiple backends for the
same extension this field is required. In this case at least one of
the two will be required: name or server. It is better to provide both
values to avoid problems with applications unable to send requests to
the proper backend service. If only one backend service is
configured, this field is ignored, and all requests are forwarded to
the configured one.
same extension this field is required. In this case, it is necessary
to provide both values to avoid problems with applications unable to
send requests to the proper backend service. If only one backend
service is configured, this field is ignored, and all requests are
forwarded to the configured one.
#### `extensions.backend.services.cluster.name` (*string*)
(optional)

View File

@@ -9,6 +9,9 @@ data:
# Repo server address. (default "argocd-repo-server:8081")
repo.server: "argocd-repo-server:8081"
# Commit server address. (default "argocd-commit-server:8086")
commit.server: "argocd-commit-server:8086"
# Redis server hostname and port (e.g. argocd-redis:6379)
redis.server: "argocd-redis:6379"
# Enable compression for data sent to Redis with the required compression algorithm. (default 'gzip')
@@ -16,6 +19,9 @@ data:
# Redis database
redis.db:
# Enables the alpha "manifest hydrator" feature. (default "false")
hydrator.enabled: "false"
# Open-Telemetry collector address: (e.g. "otel-collector:4317")
otlp.address: ""
# Open-Telemetry collector insecure: (e.g. "true")
@@ -79,6 +85,13 @@ data:
controller.diff.server.side: "false"
# Enables profile endpoint on the internal metrics port
controller.profile.enabled: "false"
# Enables batch-processing mode in the controller's cluster cache. This can help improve performance for clusters that
# have high "churn," i.e. lots of resource modifications.
controller.cluster.cache.batch.events.processing: "false"
# This sets the interval at which the controller's cluster cache processes a batch of cluster events. A lower value
# will increase the speed at which Argo CD becomes aware of external cluster state. A higher value will reduce cluster
# cache lock contention and better handle high-churn clusters.
controller.cluster.cache.events.processing.interval: "100ms"
## Server properties
# Listen on given address for incoming connections (default "0.0.0.0")
@@ -195,6 +208,15 @@ data:
# Include hidden directories from Git
reposerver.include.hidden.directories: "false"
## Commit-server properties
# Listen on given address for incoming connections (default "0.0.0.0")
commitserver.listen.address: "0.0.0.0"
# Set the logging format. One of: text|json (default "text")
commitserver.log.format: "text"
# Set the logging level. One of: debug|info|warn|error (default "info")
commitserver.log.level: "info"
# Listen on given address for metrics (default "0.0.0.0")
commitserver.metrics.listen.address: "0.0.0.0"
# Set the logging format. One of: text|json (default "text")
dexserver.log.format: "text"

View File

@@ -15,7 +15,7 @@ to indicate their stability and maturity. These are the statuses of non-stable f
## Overview
| Feature | Introduced | Status |
| ----------------------------------------- | ---------- | ------ |
|-------------------------------------------|------------|--------|
| [AppSet Progressive Syncs][2] | v2.6.0 | Alpha |
| [Proxy Extensions][3] | v2.7.0 | Alpha |
| [Skip Application Reconcile][4] | v2.7.0 | Alpha |
@@ -25,6 +25,7 @@ to indicate their stability and maturity. These are the statuses of non-stable f
| [Server Side Diff][8] | v2.10.0 | Beta |
| [Cluster Sharding: consistent-hashing][9] | v2.12.0 | Alpha |
| [Service Account Impersonation][10] | v2.13.0 | Alpha |
| [Source Hydrator][11] | v2.14.0 | Alpha |
## Unstable Configurations
@@ -83,3 +84,4 @@ to indicate their stability and maturity. These are the statuses of non-stable f
[8]: ../user-guide/diff-strategies.md#server-side-diff
[9]: ./high_availability.md#argocd-application-controller
[10]: app-sync-using-impersonation.md
[11]: ../user-guide/source-hydrator.md

View File

@@ -130,6 +130,15 @@ stringData:
count (grouped by k8s api version, the granule of parallelism for list operations). In this case, all resources will
be buffered in memory -- no api server request will be blocked by processing.
* `ARGOCD_CLUSTER_CACHE_BATCH_EVENTS_PROCESSING` - environment variable that enables the controller to collect events
for Kubernetes resources and process them in a batch. This is useful when the cluster contains a large number of resources,
and the controller is overwhelmed by the number of events. The default value is `false`, which means that the controller
processes events one by one.
* `ARGOCD_CLUSTER_CACHE_EVENTS_PROCESSING_INTERVAL` - environment variable controlling the interval for processing events in a batch.
The valid value is in the format of Go time duration string, e.g. `1ms`, `1s`, `1m`, `1h`. The default value is `100ms`.
The variable is used only when `ARGOCD_CLUSTER_CACHE_BATCH_EVENTS_PROCESSING` is set to `true`.
* `ARGOCD_APPLICATION_TREE_SHARD_SIZE` - environment variable controlling the max number of resources stored in one Redis
key. Splitting application tree into multiple keys helps to reduce the amount of traffic between the controller and Redis.
The default value is 0, which means that the application tree is stored in a single Redis key. The reasonable value is 100.

View File

@@ -24,6 +24,8 @@ Metrics about applications. Scraped at the `argocd-metrics:8082/metrics` endpoin
| `argocd_kubectl_exec_total` | counter | Number of kubectl executions |
| `argocd_redis_request_duration` | histogram | Redis requests duration. |
| `argocd_redis_request_total` | counter | Number of redis requests executed during application reconciliation |
| `argocd_resource_events_processing` | histogram | Time to process resource events in batch in seconds |
| `argocd_resource_events_processed_in_batch` | gauge | Number of resource events processed in batch |
If you use Argo CD with many application and project creation and deletion,
the metrics page will keep in cache your application and project's history.
@@ -129,6 +131,20 @@ Scraped at the `argocd-repo-server:8084/metrics` endpoint.
| `argocd_redis_request_total` | counter | Number of Kubernetes requests executed during application reconciliation. |
| `argocd_repo_pending_request_total` | gauge | Number of pending requests requiring repository lock |
## Commit Server Metrics
Metrics about the Commit Server.
Scraped at the `argocd-commit-server:8087/metrics` endpoint.
| Metric | Type | Description |
|---------------------------------------------------------|:---------:|------------------------------------------------------|
| `argocd_commitserver_commit_pending_request_total` | guage | Number of pending commit requests. |
| `argocd_commitserver_git_request_duration_seconds` | histogram | Git requests duration seconds. |
| `argocd_commitserver_git_request_total` | counter | Number of git requests performed by commit server |
| `argocd_commitserver_commit_request_duration_seconds` | histogram | Commit requests duration seconds. |
| `argocd_commitserver_userinfo_request_duration_seconds` | histogram | Userinfo requests duration seconds. |
| `argocd_commitserver_commit_request_total` | counter | Number of commit requests performed by commit server |
## Prometheus Operator
If using Prometheus Operator, the following ServiceMonitor example manifests can be used.

View File

@@ -57,7 +57,7 @@ kind: Secret
metadata:
name: argocd-notifications-secret
stringData:
sampleWebhookToken: secret-token
sampleWebhookToken: secret-token
type: Opaque
```
@@ -112,7 +112,7 @@ You can change the timezone to show in notifications as follows.
## Functions
Templates have access to the set of built-in functions:
Templates have access to the set of built-in functions such as the functions of the [Sprig](https://masterminds.github.io/sprig/) package
```yaml
apiVersion: v1

View File

@@ -130,9 +130,9 @@ p, example-user, applications, delete/*/Pod/*/*, default/prod-app, allow
Argo CD RBAC does not use `/` as a separator when evaluating glob patterns. So the pattern `delete/*/kind/*`
will match `delete/<group>/kind/<namespace>/<name>` but also `delete/<group>/<kind>/kind/<name>`.
The fact that both of these match will generally not be a problem, because resource kinds generally contain capital
letters, and namespaces cannot contain capital letters. However, it is possible for a resource kind to be lowercase.
So it is better to just always include all the parts of the resource in the pattern (in other words, always use four
The fact that both of these match will generally not be a problem, because resource kinds generally contain capital
letters, and namespaces cannot contain capital letters. However, it is possible for a resource kind to be lowercase.
So it is better to just always include all the parts of the resource in the pattern (in other words, always use four
slashes).
If we want to grant access to the user to update all resources of an application, but not the application itself:
@@ -148,9 +148,9 @@ p, example-user, applications, delete, default/prod-app, deny
p, example-user, applications, delete/*/Pod/*/*, default/prod-app, allow
```
!!! note
!!! note "Disable Application permission Inheritance"
It is not possible to deny fine-grained permissions for a sub-resource if the action was **explicitly allowed on the application**.
By default, it is not possible to deny fine-grained permissions for a sub-resource if the action was **explicitly allowed on the application**.
For instance, the following policies will **allow** a user to delete the Pod and any other resources in the application:
```csv
@@ -158,6 +158,20 @@ p, example-user, applications, delete/*/Pod/*/*, default/prod-app, allow
p, example-user, applications, delete/*/Pod/*/*, default/prod-app, deny
```
To change this behavior, you can set the config value
`server.rbac.disableApplicationFineGrainedRBACInheritance` to `true` in
the Argo CD ConfigMap `argocd-cm`.
When inheritance is disabled, it is now possible to deny fine-grained permissions for a sub-resource
if the action was **explicitly allowed on the application**.
For instance, if we want to explicitly allow updates to the application, but deny updates to any sub-resources:
```csv
p, example-user, applications, update, default/prod-app, allow
p, example-user, applications, update/*, default/prod-app, deny
```
#### The `action` action
The `action` action corresponds to either built-in resource customizations defined

View File

@@ -27,6 +27,7 @@ argocd-application-controller [flags]
--client-certificate string Path to a client certificate file for TLS
--client-key string Path to a client key file for TLS
--cluster string The name of the kubeconfig cluster to use
--commit-server string Commit server address. (default "argocd-commit-server:8086")
--context string The name of the kubeconfig context to use
--default-cache-expiration duration Cache expiration default (default 24h0m0s)
--disable-compression If true, opt-out of response compression for all requests to the server
@@ -34,6 +35,7 @@ argocd-application-controller [flags]
--enable-k8s-event none Enable ArgoCD to use k8s event. For disabling all events, set the value as none. (e.g --enable-k8s-event=none), For enabling specific events, set the value as `event reason`. (e.g --enable-k8s-event=StatusRefreshed,ResourceCreated) (default [all])
--gloglevel int Set the glog logging level
-h, --help help for argocd-application-controller
--hydrator-enabled Feature flag to enable Hydrator. Default ("false")
--ignore-normalizer-jq-execution-timeout-seconds duration Set ignore normalizer JQ execution timeout
--insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
--kubeconfig string Path to a kube config. Only required if out-of-cluster

View File

@@ -55,6 +55,7 @@ argocd-server [flags]
--enable-proxy-extension Enable Proxy Extension feature
--gloglevel int Set the glog logging level
-h, --help help for argocd-server
--hydrator-enabled Feature flag to enable Hydrator. Default ("false")
--insecure Run server without TLS
--insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
--kubeconfig string Path to a kube config. Only required if out-of-cluster

View File

@@ -1,2 +1,5 @@
This page is populated for released Argo CD versions. Use the version selector to view this table for a specific
version.
| Argo CD version | Kubernetes versions |
|-----------------|---------------------|
| 2.14 | v1.31, v1.30, v1.29, v1.28 |
| 2.13 | v1.30, v1.29, v1.28, v1.27 |
| 2.12 | v1.29, v1.28, v1.27, v1.26 |

View File

@@ -0,0 +1,6 @@
# v2.13 to 2.14
## Upgraded Helm Version
Helm was upgraded to 3.16.2 and the skipSchemaValidation Flag was added to
the [CLI and Application CR](https://argo-cd.readthedocs.io/en/latest/user-guide/helm/#helm-skip-schema-validation).

View File

@@ -0,0 +1,538 @@
---
title: Manifest Hydrator
authors:
- "@crenshaw-dev"
- "@zachaller"
sponsors:
- TBD # List all interested parties here.
reviewers:
- TBD
approvers:
- TBD
creation-date: 2024-03-26
last-updated: 2024-03-26
---
# Manifest Hydrator
This proposal describes a feature to make manifest hydration (i.e. the "rendered manifest pattern") a first-class feature of Argo CD.
## Terms
* dry manifests: DRY or Don't Repeat Yourself - things like Kustomize overlays and Helm charts that produce Kubernetes manifests but are not themselves Kubernetes Manifests
* hydrated manifests: the output from dry manifest tools, i.e. plain Kubernetes manifests
## Summary
Manifest hydration tools like Helm and Kustomize are indispensable in GitOps. These tools transform "dry" (Don't Repeat Yourself) sources into plain Kubernetes manifests. The effects of a change to dry sources are not always obvious. So storing only dry sources in git leaves the user with an incomplete and confusing history of their application. This undercuts some of the main benefits of GitOps.
The "rendered manifests" pattern has emerged as a way to mitigate the downsides of using hydration tools in GitOps. Today, developers use CI tools to automatically hydrate manifests and push to separate branches. They then configure Argo CD to deploy from the hydrated branches. (For more information, see the awesome [blog post](https://akuity.io/blog/the-rendered-manifests-pattern/) and [ArgoCon talk](https://www.youtube.com/watch?v=TonN-369Qfo) by Nicholas Morey.)
This proposal describes manifest hydration and pushing to git as a first-class feature of Argo CD.
It offers two modes of operation: push-to-deploy and push-to-stage. In push-to-deploy, hydrated manifests are pushed to the same branch from which Argo CD deploys. In push-to-stage, manifests are pushed to a different branch, and Argo CD relies on some external system to move changes to the deployment branch; this provides an integration point for automated environment promotion systems.
### Opinions
This proposal is opinionated. It is based on the belief that, in order to reap the full benefits of GitOps, every change to an application's desired state must originate from a commit to a single GitOps repository. In other words, the full history of the application's desired state must be visible as the commit history on a git repository.
This requirement is incompatible with tooling which injects nondeterministic configuration into the desired state before it is deployed by the GitOps controller. Examples of nondeterministic external configuration are:
1) Helm chart dependencies on unpinned chart versions
2) Kustomize remote bases to unpinned git revisions
3) Config tool parameter overrides in the Argo CD Application `spec.source` fields
4) Multiple sources referenced in the same application (knowledge of combination of source versions is held externally to git)
Injecting nondeterministic configuration makes it impossible to know the complete history of an application by looking at a git branch history. Even if the nondeterministic output is databased (for example, in a hydrated source branch in git), it is impossible for developers to confidently make changes to desired state, because they cannot know ahead of time what other configuration will be injected at deploy time.
We believe that the problems of injecting external configuration are best solved by asking these two questions:
1) Does the configuration belong in the developer's interface (i.e. the dry manifests)?
2) Does the configuration need to be mutable at runtime, or only at deploy time?
If the configuration belongs in the developer's interface, write a tool to push the information to git. Image tags are a good example of such configuration, and the Argo CD Image Updater is a good example of such tooling.
If the configuration doesn't belong in the developer's interface, and it needs to be updated at runtime, write a controller. The developer shouldn't be expected to maintain configuration which is not an immediate part of their desired state. An example would be an auto-sizing controller which eliminates the need for the developer to manage their own autoscaler config.
If the configuration doesn't belong in the developer's interface and doesn't need to be updated at runtime (only at deploy time), write a mutating webhook. This is a great option for injecting cluster-specific configuration that the developer doesn't need to directly control.
With these three options available (git-pushers, controllers, and mutating webhooks), we believe that it is not generally necessary to inject nondeterministic configuration into the manifest hydration process. Instead, we can have a full history of the developer's minimal intent (dry branch) and the full expression of that intent (hydrated branch) completely recorded in a series of commits on a git branch.
By respecting these limitations, we unlock the ability to manage change promotion/reversion entirely via git. Change lineage is fully represented as a series of dry commit hashes. This makes it possible to write reliable rules around how these hashes are promoted to different environments and how they are reverted (i.e. we can meaningfully say "`prod` may never be more than one dry hash ahead of `test`"). If information about the lineage of an application is scattered among multiple sources, it is difficult or even impossible to meaningfully define rules about how one environment's lineage must relate to that of another environment.
Being opinionated unlocks the full benefits of GitOps as well as the ability to build a reasonable, reliable preview/promotion/reversion system.
These opinions will lock out use cases where configuration injection cannot be avoided by writing git-pushers, controllers, or mutating webhooks. We believe that the benefits of making an opinionated system outweigh the costs of compromising those opinions.
## Motivation
Many organizations have implemented their own manifest hydration system. By implementing it in Argo CD, we can lower the cost to our users of maintaining those systems, and we can encourage best practices related to the pattern.
### Goals
1) Make manifest hydration easy and intuitive for Argo CD users
2) Make it possible to implement a promotion system which relies on the manifest hydration's push-to-stage mode
3) Emphasize maintaining as much of the system's state as possible in git rather than in the Application CR (e.g. source hydrator config values, such as Helm values)
4) Every deployed change must have a corresponding dry commit - i.e. git is always the source of any changes
5) Developers should be able to easily reproduce the manifest hydration process locally, i.e. by running some commands
#### Hydration Reproducibility
One goal of this proposal is to make hydration reproducibility easy. Reproducibility brings a couple benefits: easy iteration/debugging and reliable previews.
##### Easy Iteration/Debugging
The hydration system should enable developers to easily reproduce the hydration process locally. The developer should be able to run a short series of commands and perform the exact same tasks that Argo CD would take to hydrate their manifests. This allows the developer to verify that Argo CD is behaving as expected and to quickly tweak inputs and see the results. This lets them iterate quickly and improves developer satisfaction and change velocity.
To provide this experience, the hydrator needs to provide the developer with a few pieces of information:
1) The input repo URL, path, and commit SHA
2) The hydration tool CLI version(s) (for example, the version of the Helm CLI used for hydration)
3) A series of commands and arguments which the developer can run locally
Equipped with this information, the developer can perform the exact same steps as Argo CD and be confident that their dry manifest changes will produce the desired output.
Ensuring that hydration is deterministic assures the developer that the output for a given dry state will be the same next week as it is today.
###### Avoiding Esoteric Behavior
We should avoid the developer needing to know Argo CD-specific behavior in order to reproduce hydration. Tools like Helm, Kustimize, etc. have excellent public-facing documentation which the developer should be able to take advantage of without needing to know quirks of Argo CD.
##### Reliable Previews
Deterministic hydration output allows Argo CD to produce a reliable change preview when a developer proposes a change to the dry manifests via a PR.
If output is not deterministic, then a preview generated today might not be valid/correct a week, day, or even hour later. Non-determinism makes it so that developers can't trust that the change they review will be the change actually applied.
### Non-Goals
1) Implementing a change promotion system
## Open Questions
* The `sourceHydrator` field is mutually exclusive with the `source` and the `sources` field. Should we throw an error if they're both configured, or should we just pick one and ignore the others?
* How will/should this feature relate to the image updater? Is there an opportunity to share code, since both tools involve pushing to git?
* Should we enforce a naming convention for hydrated manifest branches, e.g. `argo/...`? This would make it easier to recommend branch protection rules, for example, only allow pushes to `argo/*` from the argo bot.
* Should we enforce setting a `sourceHydrator.syncSource.path` to something besides `.`? Setting a path makes it easier to add/remove other apps later if desired.
## Proposal
Today, Argo CD watches one or more git repositories (configured in the `spec.source` or `spec.sources` field). When a new commit appears, Argo CD updates the desired state by rendering the manifests with the configured manifest hydration tool. If auto-sync is enabled, Argo CD applies the new manifests to the cluster.
With the introduction of this change, Argo CD will watch two revisions in the same git repository: the first is the "dry source", i.e. the git repo/revision where the un-rendered manifests reside, and the second is the "hydrated source," where the rendered manifests are places and retrieved for syncing to the cluster.
### New `spec.sourceHydrator` Application Field
A `sourceHydrator` field will be added to the Argo CD Application spec:
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: example
spec:
# The sourceHydrator field is mutually-exclusive with `source` and with `sources`. If this field is configured, we
# should either throw an error or ignore the other two.
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
# This assumes the Application's environments are modeled as directories.
path: environments/e2e
syncSource:
targetBranch: environments/e2e
path: .
# The hydrateTo field is optional. If specified, Argo CD will write hydrated manifests to this branch instead of the
# syncSource.targetBranch. This allows the user to "stage" a hydrated commit before actually deploying the changes
# by merging them into the syncSource branch. A complete change promotion system can be built around this feature.
hydrateTo:
targetBranch: environments/e2e-next
# The path is assumed to be the same as that in syncSource.
```
When the Argo CD application controller detects a new commit on the `drySource`, it queue up the hydration process.
When the application controller detects a new (hydrated) commit on the `syncSource.targetBranch`, it will sync the manifests.
### Processing a New Dry Commit
On noticing a new dry commit, Argo CD will first collect all Applications which have the same `drySource` repo and targetRevision.
Argo CD will then group those sources by the configured `syncSource` targetBranch.
```go
package hydrator
import "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
type DrySource struct {
repoURL string
targetRevision string
}
type SyncSource struct {
targetBranch string
}
var appGroups map[DrySource]map[SyncSource][]v1alpha1.Application
```
Then Argo CD will loop over the apps in each group. For each group, it will run manifest hydration on the configured `drySource.path` and write the result to the configured `syncSource.path`. After looping over all apps in the group and writing all their manifests, it will commit the changes to the configured `syncSource` repoURL and targetBranch (or, if configured, the `hydratedTo` targetBranch). Finally, it will push those changes to git. Then it will repeat this process for the remaining groups.
The actual push operation should be delegated to the [commit server](./manifest-hydrator/commit-server/README.md).
To understand how this would work for a simple dev/test/prod setup with two regions, consider this example:
```yaml
### DEV APPS ###
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: dev-west
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
path: environments/dev/west
syncSource:
targetBranch: environments/dev
path: west
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: dev-east
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
path: environments/dev/east
syncSource:
targetBranch: environments/dev
path: east
---
### TEST APPS ###
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: test-west
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
path: environments/test/west
syncSource:
targetBranch: environments/test
path: west
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: test-east
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
path: environments/test/east
syncSource:
targetBranch: environments/prod
path: east
---
### PROD APPS ###
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: prod-west
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
path: environments/prod/west
syncSource:
targetBranch: environments/prod
path: west
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: prod-east
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/argoproj/argocd-example-apps
targetRevision: main
path: environments/prod/east
syncSource:
targetBranch: environments/prod
path: east
---
```
Each commit to the dry branch will result in a commit to up to three branches. Each commit to an environment branch will contain changes for west, east, or both (depending on which is affected). Changes originating from a single dry commit are always grouped into a single hydrated commit.
### Handling External Values Files
Since only one source may be used in as the dry source, the multi-source approach to external Helm values files will not work here. Instead, we'll recommend that users use the umbrella chart approach. The main reasons for multi-source as an alternative were convenience (no need to maintain the parent chart) and resolving issues with authentication to dependency charts. We believe the simplification is worth the cost of convenience, and we can address the auth issues as standalone bugs.
An earlier iteration of this proposal attempted to preserve the multi-source style of external value file inclusion by introducing a "magic" `.argocd-hydrator.yaml` file containing `additionalSources` to reference the Helm chart. In the end, it felt like we were re-implementing Helm's dependencies feature or git submodules. It's better to just rely on one of those existing tools.
### `.argocd-source.yaml` Support
The `spec.sourceHydrator.drySource` field contains only three fields: `repoURL`, `targetRevision`, and `path`.
`spec.source` contains a number of fields for configuring manifest hydration tools (`helm`, `kustomize`, and `directory`). That functionality is still available for `spec.sourceHydrator`. But instead of being configured in the Application CR, those values are set in `.argocd-source.yaml`, an existing "override" mechanism for `spec.source`. By requiring that this configuration be set in `.argocd-source.yaml`, we respect the principle that all changes must be made in git instead of in the Application CR.
### `spec.destination.namespace` Behavior
The Application `spec.destination.namespace` field is used to set the `metadata.namespace` field of any namespace resources for which that field is not set in the manifests.
The hydrator will not inject `metadata.namespace` into the hydrated manifests pushed to git. Instead, Argo CD's behavior of injecting that value immediately before applying to the cluster will continue to be used with the `spec.sourceHydrator.syncSource`.
### Build Environment Support
For sources specified in `spec.source` or `spec.sources`, Argo CD [sets certain environment variables](https://argo-cd.readthedocs.io/en/stable/user-guide/build-environment/) before running the manifest hydration tool.
Some of these environment variables may change independently of the dry source and therefore break the reproducibility of manifest hydration (see the [Opinions](#opinions) section). Therefore, only some environment variables will be populated for the `spec.sourceHydrator` source.
These environment variables will **not** be set:
* `ARGOCD_APP_NAME`
* `ARGOCD_APP_NAMESPACE`
* `KUBE_VERSION`
* `KUBE_API_VERSIONS`
These environment variables will be set because they are commit SHAs and are directly and immutably tied to the dry manifest commit:
* `ARGOCD_APP_REVISION`
* `ARGOCD_APP_REVISION_SHORT`
These environment variables will be set because they are inherently tied to the manifest hydrator configuration. If these fields set in `spec.sourceHydrator.drySource` change, we are breaking the connection to the original hydrator configuration anyway.
* `ARGOCD_APP_SOURCE_PATH`
* `ARGOCD_APP_SOURCE_REPO_URL`
* `ARGOCD_APP_SOURCE_TARGET_REVISION`
### Support for Helm-Specific Features
#### App Name / Release Name
By default, Argo CD's `source` and `sources` fields use the Application's name as the release name when hydrating Helm manifests.
To centralize the source of truth when using `spec.sourceHydrator`, the default release name will be an empty string, and any different release name should be specified in the `helm.releaseName` field in `.argocd-source.yaml`.
#### Kube API Versions
`helm install` supports dynamically reading Kube API versions from the destination cluster to adjust manifest output. `helm template` accepts a list of Kube API versions to simulate the same behavior, and Argo CD's `spec.source` and `spec.sources` fields set those API versions when running `helm template`.
To centralize the source of truth when using `spec.sourceHydrator`, the Kube API versions will not be populated by default.
Instead, a new field will be added to the Application's `spec.source.helm` field:
```yaml
kind: Application
spec:
source:
helm:
apiVersions:
- admissionregistration.k8s.io/v1/MutatingWebhookConfiguration
- admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration
- ... etc.
```
That field will also be available in `.argocd-source.yaml`:
```yaml
helm:
apiVersions:
- admissionregistration.k8s.io/v1/MutatingWebhookConfiguration
- admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration
- ... etc.
```
So the appropriate way to set Kube API versions for the source hydrator will be to populate the `.argocd-source.yaml` file.
#### Hydrated Environment Branches
Representing the dry manifests of environments as branches has well-documented downsides for developer experience. Specifically, it's toilsome for developers to manage moving changes from one branch to another and avoid drift.
So environments-as-directories has emerged as the standard for good GitOps practices. Change management across directories in a single branch is much easier to perform and reason about.
**This proposal does not suggest using branches to represent the dry manifests of environments.** As a matter of fact, this proposal codifies the current best practice of representing the dry manifests as directories in a single branch.
This proposal recommends using different branches for the _hydrated_ representation of environments only. Using different branches has some benefits:
1) Intuitive grouping of "changes to ship at once" - for example, if you have app-1-east and app-1-west, it makes sense to merge a single hydrated PR to deploy to both of those apps at once
2) Easy-to-read history of a single environment via the commits history
3) Easy comparison between environments using the SCMs' "compare" interfaces
In other words, branches make a very nice _read_ interface for _hydrated_ manifests while preserving the best-practice of using _directories_ for the _write_ interface.
### Commit Metadata
Each output directory should contain two files: manifest.yaml and README.md. manifest.yaml should contain the plain hydrated manifests. The resources should be sorted by namespace, name, group, and kind (in that order).
The README will be built using the following template:
````gotemplate
{{ if eq (len .applications) 1 }}
{{ $appName := (index .applications 0).metadata.name }}
# {{ $appName }} Manifests
[manifest.yaml](./manifest.yaml) contains the hydrated manifests for the {{ $appName }} application.
{{ end }}
{{ if gt (len .applications) 1 }}
{{ $appName := (index .applications 0).metadata.name }}
# Manifests for {{ len .applications }} Applications
[manifest.yaml](./manifest.yaml) contains the hydrated manifests for these applications:
{{ range $i, $app := .applications }}
- {{ $app.name }}
{{ end }}
{{ end }}
These are the details of the most recent change;
* Author: {{ .commitAuthor }}
* Message: {{ .commitMessage }}
* Time: {{ .commitTime }}
To reproduce the manifest hydration, do the following:
```
git clone {{ .repoURL }}
cd {{ .repoName }}
git checkout {{ .dryShortSHA }}
{{ range $i, $command := .commands }}
{{ $command }}
{{ end }}
```
````
This template should be admin-configurable.
Example output might look like this:
````markdown
# dev-west Manifests
[manifest.yaml](./manifest.yaml) contains the hydrated manifests for the dev-west application.
These are the details of the most recent change;
* Author: Michael Crenshaw <michael@example.com>
* Message: chore: bumped image tag to v0.0.2
* Time: 2024-03-27 10:32:04 UTC
To reproduce the manifest hydration, do the following:
```
git clone https://github.com/argoproj/argocd-example-apps
cd argocd-example-apps
git checkout ab2382f
kustomize edit set image my-app:v0.0.2
kustomize build environments/dev/west
```
````
The hydrator will also write a `hydrator.metadata` file containing a JSON representation of all the values available for README templating. This metadata can be used by external systems (e.g. a PR-based promoter system) to generate contextual information about the hydrated manifest's provenance.
```json
{
"commands": ["kustomize edit set image my-app:v0.0.2", "kustomize build ."],
"drySHA": "ab2382f",
"commitAuthor": "Michael Crenshaw <michael@example.com>",
"commitMessage": "chore: bump Helm dependency chart to 32.1.12",
"repoURL": "https://github.com/argoproj/argocd-example-apps"
}
```
To request a commit to the hydrated branch, the application controller will make a call to the CommitManifests service.
A single call will bundle all the changes destined for a given targetBranch.
It's the application controller's job to ensure that the user has write access to the repo before making the call.
```protobuf
// CommitManifests represents the caller's request for some Kubernetes manifests to be pushed to a git repository.
message CommitManifests {
// repoURL is the URL of the repo we're pushing to. HTTPS or SSH URLs are acceptable.
required string repoURL = 1;
// targetBranch is the name of the branch we're pushing to.
required string targetBranch = 2;
// drySHA is the full SHA256 hash of the "dry commit" from which the manifests were hydrated.
required string drySHA = 3;
// commitAuthor is the name of the author of the dry commit.
required string commitAuthor = 4;
// commitMessage is the short commit message from the dry commit.
required string commitMessage = 5;
// commitTime is the dry commit timestamp.
required string commitTime = 6;
// details holds the information about the actual hydrated manifests.
repeated CommitPathDetails details = 7;
}
// CommitManifestDetails represents the details about a
message CommitPathDetails {
// path is the path to the directory to which these manifests should be written.
required string path = 1;
// manifests is a list of JSON documents representing the Kubernetes manifests.
repeated string manifests = 2;
// readme is a string which will be written to a README.md alongside the manifest.yaml.
required string readme = 3;
}
message CommitManifestsResponse {
}
```
### Push access
The hydrator will need to push to the git repository. This will require a secret containing the git credentials.
Write access will be configured via a Kubernetes secret with the following structure:
```yaml
apiVersion: v1
kind: Secret
metadata:
labels:
argocd.argoproj.io/secret-type: repository-write
stringData:
url: 'https://github.com/argoproj/argocd-example-apps'
githubAppID: '123456'
githubInstallationID: '123456'
githubAppPrivateKey: |
-----
```
### Use cases
#### Use case 1:
An organization with strong requirements around change auditing might enable manifest hydration in order to generate a full history of changes.
#### Use case 2:
### Implementation Details/Notes/Constraints
### Detailed examples
### Security Considerations
This proposal would involve introducing a component capable of pushing to git.
We'll need to consider what git permissions setup to recommend, what security features we should recommend enabling (e.g. branch protection), etc.
We'll also need to consider how to store the git push secrets. It's probable that they'll need to be stored in a namespace separate from the other Argo CD components to provide a bit extra protection.
### Risks and Mitigations
### Upgrade / Downgrade Strategy
## Drawbacks
## Alternatives

View File

@@ -0,0 +1,44 @@
# Argo CD Manifest Hydrator
Most Argo CD Applications don't directly use plain Kubernetes manifests. They reference a Helm chart or some Kustomize manifests, and then Argo CD transforms those sources into their final form (plain Kubernetes manifests).
Having Argo CD quietly do this transformation behind the scenes is convenient. But it can make it harder for developers to understand the full state of their application, both current and past. Hydrating (also known as "rendering") the sources and pushing the hydrated manifests to git is a common technique to preserve a full history of an Application's state.
Argo CD provides first-class tooling to hydrate manifests and push them to git. This document explains how to take advantage of that tooling.
## Setting up git Push Access
To use Argo CD's source hydration tooling, you have to grant Argo CD push access to all the repositories for apps using the source hydrator.
### Security Considerations
Argo CD stores git push secrets separately from the main Argo CD components and separately from git pull credentials to minimize the possibility of a malicious actor stealing the secrets or hijacking Argo CD components to push malicious changes.
Pushing hydrated manifests to git can improve security by ensuring that all state changes are stored and auditable. If a malicious actor does manage to produce malicious changes in manifests, those changes will be discoverable in git instead of living only in the live cluster state.
You should use your SCM's security mechanisms to ensure that Argo CD can only push to the allowed repositories and branches.
### Adding the Access Credentials
To set up push access, add a secret to the `argocd-push` namespace with the following format:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: argocd-example-apps
labels:
# Note that this is "repository-push" instead of "repository". The same secret should never be used for both push and pull access.
argocd.argoproj.io/secret-type: repository-push
type: Opaque
stringData:
url: https://github.com/argoproj/argocd-example-apps.git
username: '****'
password: '****'
```
Once the secret is available, any Application which has pull access to a given repo will be able to use the source hydration tooling to also push to that repo.
## Using the `sourceHydrator` Field
## Migrating from the `source` or `sources` Field

View File

@@ -0,0 +1,38 @@
# Commit Server
The Argo CD Commit Server provides push access to git repositories for hydrated manifests.
The server exposes a gRPC service which accepts requests to push hydrated manifests to a git repository. This is the interface:
```protobuf
// CommitManifests represents the caller's request for some Kubernetes manifests to be pushed to a git repository.
message CommitManifests {
// repoURL is the URL of the repo we're pushing to. HTTPS or SSH URLs are acceptable.
required string repoURL = 1;
// targetBranch is the name of the branch we're pushing to.
required string targetBranch = 2;
// drySHA is the full SHA256 hash of the "dry commit" from which the manifests were hydrated.
required string drySHA = 3;
// commitAuthor is the name of the author of the dry commit.
required string commitAuthor = 4;
// commitMessage is the short commit message from the dry commit.
required string commitMessage = 5;
// commitTime is the dry commit timestamp.
required string commitTime = 6;
// details holds the information about the actual hydrated manifests.
repeated CommitPathDetails details = 7;
}
// CommitManifestDetails represents the details about a
message CommitPathDetails {
// path is the path to the directory to which these manifests should be written.
required string path = 1;
// manifests is a list of JSON documents representing the Kubernetes manifests.
repeated string manifests = 2;
// readme is a string which will be written to a README.md alongside the manifest.yaml.
required string readme = 3;
}
message CommitManifestsResponse {
}
```

View File

@@ -23,39 +23,39 @@ recent minor releases.
| [install.yaml](master/argocd-iac-install.html) | - | - | - | - |
| [namespace-install.yaml](master/argocd-iac-namespace-install.html) | - | - | - | - |
### v2.13.1
### v2.13.2
| | Critical | High | Medium | Low |
|---:|:--------:|:----:|:------:|:---:|
| [go.mod](v2.13.1/argocd-test.html) | 0 | 0 | 7 | 2 |
| [ui/yarn.lock](v2.13.1/argocd-test.html) | 0 | 0 | 1 | 0 |
| [dex:v2.41.1](v2.13.1/ghcr.io_dexidp_dex_v2.41.1.html) | 0 | 0 | 0 | 2 |
| [haproxy:2.6.17-alpine](v2.13.1/public.ecr.aws_docker_library_haproxy_2.6.17-alpine.html) | 0 | 0 | 2 | 4 |
| [redis:7.0.15-alpine](v2.13.1/public.ecr.aws_docker_library_redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [argocd:v2.13.1](v2.13.1/quay.io_argoproj_argocd_v2.13.1.html) | 0 | 0 | 3 | 10 |
| [redis:7.0.15-alpine](v2.13.1/redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [install.yaml](v2.13.1/argocd-iac-install.html) | - | - | - | - |
| [namespace-install.yaml](v2.13.1/argocd-iac-namespace-install.html) | - | - | - | - |
| [go.mod](v2.13.2/argocd-test.html) | 1 | 0 | 7 | 2 |
| [ui/yarn.lock](v2.13.2/argocd-test.html) | 0 | 0 | 1 | 0 |
| [dex:v2.41.1](v2.13.2/ghcr.io_dexidp_dex_v2.41.1.html) | 0 | 0 | 0 | 2 |
| [haproxy:2.6.17-alpine](v2.13.2/public.ecr.aws_docker_library_haproxy_2.6.17-alpine.html) | 0 | 0 | 2 | 4 |
| [redis:7.0.15-alpine](v2.13.2/public.ecr.aws_docker_library_redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [argocd:v2.13.2](v2.13.2/quay.io_argoproj_argocd_v2.13.2.html) | 0 | 0 | 3 | 10 |
| [redis:7.0.15-alpine](v2.13.2/redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [install.yaml](v2.13.2/argocd-iac-install.html) | - | - | - | - |
| [namespace-install.yaml](v2.13.2/argocd-iac-namespace-install.html) | - | - | - | - |
### v2.12.7
### v2.12.8
| | Critical | High | Medium | Low |
|---:|:--------:|:----:|:------:|:---:|
| [go.mod](v2.12.7/argocd-test.html) | 0 | 0 | 8 | 2 |
| [ui/yarn.lock](v2.12.7/argocd-test.html) | 0 | 0 | 1 | 0 |
| [dex:v2.38.0](v2.12.7/ghcr.io_dexidp_dex_v2.38.0.html) | 0 | 0 | 6 | 7 |
| [haproxy:2.6.17-alpine](v2.12.7/public.ecr.aws_docker_library_haproxy_2.6.17-alpine.html) | 0 | 0 | 2 | 4 |
| [redis:7.0.15-alpine](v2.12.7/public.ecr.aws_docker_library_redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [argocd:v2.12.7](v2.12.7/quay.io_argoproj_argocd_v2.12.7.html) | 0 | 0 | 3 | 11 |
| [redis:7.0.15-alpine](v2.12.7/redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [install.yaml](v2.12.7/argocd-iac-install.html) | - | - | - | - |
| [namespace-install.yaml](v2.12.7/argocd-iac-namespace-install.html) | - | - | - | - |
| [go.mod](v2.12.8/argocd-test.html) | 1 | 0 | 8 | 2 |
| [ui/yarn.lock](v2.12.8/argocd-test.html) | 0 | 0 | 1 | 0 |
| [dex:v2.38.0](v2.12.8/ghcr.io_dexidp_dex_v2.38.0.html) | 0 | 0 | 6 | 7 |
| [haproxy:2.6.17-alpine](v2.12.8/public.ecr.aws_docker_library_haproxy_2.6.17-alpine.html) | 0 | 0 | 2 | 4 |
| [redis:7.0.15-alpine](v2.12.8/public.ecr.aws_docker_library_redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [argocd:v2.12.8](v2.12.8/quay.io_argoproj_argocd_v2.12.8.html) | 0 | 0 | 3 | 10 |
| [redis:7.0.15-alpine](v2.12.8/redis_7.0.15-alpine.html) | 0 | 0 | 0 | 1 |
| [install.yaml](v2.12.8/argocd-iac-install.html) | - | - | - | - |
| [namespace-install.yaml](v2.12.8/argocd-iac-namespace-install.html) | - | - | - | - |
### v2.11.12
| | Critical | High | Medium | Low |
|---:|:--------:|:----:|:------:|:---:|
| [go.mod](v2.11.12/argocd-test.html) | 0 | 2 | 9 | 2 |
| [go.mod](v2.11.12/argocd-test.html) | 1 | 2 | 9 | 2 |
| [ui/yarn.lock](v2.11.12/argocd-test.html) | 0 | 0 | 1 | 0 |
| [dex:v2.38.0](v2.11.12/ghcr.io_dexidp_dex_v2.38.0.html) | 0 | 0 | 6 | 7 |
| [haproxy:2.6.14-alpine](v2.11.12/haproxy_2.6.14-alpine.html) | 0 | 1 | 7 | 7 |

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:23:04 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:23:55 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following path:</span>
@@ -2861,7 +2861,7 @@
</li>
<li class="card__meta__item">
Line number: 24840
Line number: 24846
</li>
</ul>

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:23:14 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:24:05 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following path:</span>
@@ -2815,7 +2815,7 @@
</li>
<li class="card__meta__item">
Line number: 2163
Line number: 2169
</li>
</ul>

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:20:56 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:21:36 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following paths:</span>
@@ -470,7 +470,7 @@
<div class="meta-counts">
<div class="meta-count"><span>7</span> <span>known vulnerabilities</span></div>
<div class="meta-count"><span>26 vulnerable dependency paths</span></div>
<div class="meta-count"><span>2158</span> <span>dependencies</span></div>
<div class="meta-count"><span>2160</span> <span>dependencies</span></div>
</div><!-- .meta-counts -->
</div><!-- .layout-container--short -->
</header><!-- .project__header -->

View File

@@ -7,7 +7,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>Snyk test report</title>
<meta name="description" content="22 known vulnerabilities found in 43 vulnerable dependency paths.">
<meta name="description" content="23 known vulnerabilities found in 44 vulnerable dependency paths.">
<base target="_blank">
<link rel="icon" type="image/png" href="https://res.cloudinary.com/snyk/image/upload/v1468845142/favicon/favicon.png"
sizes="194x194">
@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:21:06 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:21:47 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following paths:</span>
@@ -469,8 +469,8 @@
</div>
<div class="meta-counts">
<div class="meta-count"><span>22</span> <span>known vulnerabilities</span></div>
<div class="meta-count"><span>43 vulnerable dependency paths</span></div>
<div class="meta-count"><span>23</span> <span>known vulnerabilities</span></div>
<div class="meta-count"><span>44 vulnerable dependency paths</span></div>
<div class="meta-count"><span>969</span> <span>dependencies</span></div>
</div><!-- .meta-counts -->
</div><!-- .layout-container--short -->
@@ -479,6 +479,80 @@
<div class="layout-container" style="padding-top: 35px;">
<div class="cards--vuln filter--patch filter--ignore">
<div class="card card--vuln disclosure--not-new severity--critical" data-snyk-test="critical">
<h2 class="card__title">Incorrect Implementation of Authentication Algorithm</h2>
<div class="card__section">
<div class="label label--critical">
<span class="label__text">critical severity</span>
</div>
<hr/>
<ul class="card__meta">
<li class="card__meta__item">
Manifest file: ghcr.io/dexidp/dex:v2.41.1/hairyhenderson/gomplate/v4 <span class="list-paths__item__arrow"></span> /usr/local/bin/gomplate
</li>
<li class="card__meta__item">
Package Manager: golang
</li>
<li class="card__meta__item">
Vulnerable module:
golang.org/x/crypto/ssh
</li>
<li class="card__meta__item">Introduced through:
github.com/hairyhenderson/gomplate/v4@* and golang.org/x/crypto/ssh@v0.24.0
</li>
</ul>
<hr/>
<h3 class="card__section__title">Detailed paths</h3>
<ul class="card__meta__paths">
<li>
<span class="list-paths__item__introduced"><em>Introduced through</em>:
github.com/hairyhenderson/gomplate/v4@*
<span class="list-paths__item__arrow"></span>
golang.org/x/crypto/ssh@v0.24.0
</span>
</li>
</ul><!-- .list-paths -->
</div><!-- .card__section -->
<hr/>
<!-- Overview -->
<h2 id="overview">Overview</h2>
<p><a href="https://pkg.go.dev/golang.org/x/crypto/ssh?tab=doc">golang.org/x/crypto/ssh</a> is a SSH client and server</p>
<p>Affected versions of this package are vulnerable to Incorrect Implementation of Authentication Algorithm when the key passed in the last call before a connection is established is assumed to be the key used for authentication. It is not necessarily the authentication key in use, and this allows attackers who can control the key cache by making their own carefully-timed connections to bypass authorization with subsequent legitimate <code>ServerConfig.PublicKeyCallback</code> callbacks.</p>
<p><strong>Note:</strong> The assumed caching behavior of this callback is not documented and is therefore considered human error, but the project maintainers have observed reliance on it for authorization decisions in production. In fact, the assumption is negated in the documentation, which states &quot;A call to this function does not guarantee that the key offered is in fact used to authenticate.&quot; The behavior after upgrading still allows the possibility of an attacker forcing their own key to be the one in the cache when the callback is invoked if the client is using a different authentication method such as <code>PasswordCallback</code>, <code>KeyboardInteractiveCallback</code>, or <code>NoClientAuth</code>. It is therefore recommended to rely on the return values of the connection itself, found in <code>ServerConn.Permissions</code> for further authorization steps.</p>
<h2 id="remediation">Remediation</h2>
<p>Upgrade <code>golang.org/x/crypto/ssh</code> to version 0.31.0 or higher.</p>
<h2 id="references">References</h2>
<ul>
<li><a href="https://github.com/golang/crypto/commit/b4f1988a35dee11ec3e05d6bf3e90b695fbd8909">GitHub Commit</a></li>
<li><a href="https://github.com/golang/go/issues/20094">GitHub Issue</a></li>
<li><a href="https://go.dev/cl/635315">go.dev Commit</a></li>
<li><a href="https://go.dev/issue/70779">go.dev Issue</a></li>
<li><a href="https://groups.google.com/g/golang-announce/c/-nPEi39gI4Q/m/cGVPJCqdAQAJ">Google Groups Forum</a></li>
<li><a href="https://pkg.go.dev/vuln/GO-2024-3321">Go Vulnerability Database</a></li>
</ul>
<hr/>
<div class="cta card__cta">
<p><a href="https://snyk.io/vuln/SNYK-GOLANG-GOLANGORGXCRYPTOSSH-8496611">More about this vulnerability</a></p>
</div>
</div><!-- .card -->
<div class="card card--vuln disclosure--not-new severity--medium" data-snyk-test="medium">
<h2 class="card__title">Insertion of Sensitive Information into Log File</h2>
<div class="card__section">

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:21:11 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:21:52 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following path:</span>

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:21:15 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:22:00 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following paths:</span>

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:21:33 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:22:20 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following paths:</span>
@@ -472,7 +472,7 @@
<div class="meta-counts">
<div class="meta-count"><span>20</span> <span>known vulnerabilities</span></div>
<div class="meta-count"><span>100 vulnerable dependency paths</span></div>
<div class="meta-count"><span>2378</span> <span>dependencies</span></div>
<div class="meta-count"><span>2380</span> <span>dependencies</span></div>
</div><!-- .meta-counts -->
</div><!-- .layout-container--short -->
</header><!-- .project__header -->

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:21:38 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:22:25 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following paths:</span>

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:30:13 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:31:10 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following path:</span>

View File

@@ -456,7 +456,7 @@
<div class="header-wrap">
<h1 class="project__header__title">Snyk test report</h1>
<p class="timestamp">December 8th 2024, 12:30:22 am (UTC+00:00)</p>
<p class="timestamp">December 15th 2024, 12:31:19 am (UTC+00:00)</p>
</div>
<div class="source-panel">
<span>Scanned the following path:</span>

Some files were not shown because too many files have changed in this diff Show More