Compare commits

...

65 Commits

Author SHA1 Message Date
argo-bot
a48bca03c7 Bump version to 2.4.6 2022-07-12 22:19:23 +00:00
argo-bot
6930ceb414 Bump version to 2.4.6 2022-07-12 22:19:14 +00:00
CI
01b7a73922 chore: fix build error
Signed-off-by: CI <michael@crenshaw.dev>
2022-07-12 16:46:39 -04:00
taksenov
ff2d9b918d docs: fix typo in Generators-Git.md (#9949)
`ApplictionSet` --> `ApplicationSet`
Signed-off-by: CI <michael@crenshaw.dev>
2022-07-12 16:29:03 -04:00
Jake
cea91ce935 docs: add terminal documentation (#9948)
Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>
2022-07-12 16:28:24 -04:00
Michael Crenshaw
51b73096f5 fix: 'unexpected reserved bits' breaking web terminal (#9605) (#9895)
* fix: 'unexpected reserved bits' breaking web terminal (#9605)

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* make things more like they were originally, since the mutex fixes the problem

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fix typo, don't pass around a pointer when it isn't necessary

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* apply suggestions

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-07-12 16:28:01 -04:00
Yuan Tang
2c166dac97 feat: Treat connection reset as a retryable error (#9739)
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
2022-07-12 16:25:08 -04:00
jannfis
16c015f0cb test: Use dedicated multi-arch workloads in e2e tests (#9921)
* test: Use dedicated multi-arch workloads in e2e tests

Signed-off-by: jannfis <jann@mistrust.net>

* Use correct tag

Signed-off-by: jannfis <jann@mistrust.net>
2022-07-12 16:24:41 -04:00
Xiao Yang
7ca60cb957 fix: argocd login just hangs on 2.4.0 #9679 (#9935)
Signed-off-by: Xiao Yang <muma.378@163.com>

Co-authored-by: Michael Crenshaw <michael@crenshaw.dev>
Signed-off-by: CI <michael@crenshaw.dev>
2022-07-12 16:24:23 -04:00
Jake
d0c80ee0bb fix: CMP manifest generation fails with ENHANCE_YOUR_CALM if over 40s (#9922)
* fix: CMP manifest generation fails with ENHANCE_YOUR_CALM if over 40s

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* fix timeouts across all gRPC servers

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* use common consts

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>
2022-07-12 16:24:00 -04:00
yongguangl
9f1a33865a fix: NotAfter is not set when ValidFor is set (#9911)
Signed-off-by: yongguangl <1363186473@qq.com>
2022-07-12 16:21:21 -04:00
Hyeonmin Park
f09c84f30a fix: add missing download CLI tool link for ppc64le, s390x (#9649)
Signed-off-by: Hyeonmin Park <hyeonmin.park@kennysoft.kr>
2022-07-12 16:21:03 -04:00
jannfis
f7f7493e9f fix: Check tracking annotation for being self-referencing (#9791)
* fix: Check tracking annotation for being self-referencing

Signed-off-by: jannfis <jann@mistrust.net>

* Tweak isManagedLiveObj() logic

Signed-off-by: jannfis <jann@mistrust.net>

* Rename isManagedLiveResource to isSelfReferencedObj

Signed-off-by: jannfis <jann@mistrust.net>

* Add e2e test

Signed-off-by: jannfis <jann@mistrust.net>
2022-07-12 16:20:21 -04:00
jannfis
c1ddb53d29 fix: Make change of tracking method work at runtime (#9820)
* fix: Make change of tracking method work at runtime

Signed-off-by: jannfis <jann@mistrust.net>

* GetAppName() will figure tracking label or annotation on its own

Signed-off-by: jannfis <jann@mistrust.net>

* Correct test comments and add another test

Signed-off-by: jannfis <jann@mistrust.net>

* Add a read lock before getting cache settings

Signed-off-by: jannfis <jann@mistrust.net>
2022-07-12 16:19:41 -04:00
Shunki
e04288482d fix: argo-cd git submodule is using SSH auth instead of HTTPs (#3118) (#9821)
* fix: argo-cd git submodule is using SSH auth instead of HTTPs (#3118)

Signed-off-by: shunki-fujita <shunki-fujita@cybozu.co.jp>

* Add submodule functions and unit tests
Signed-off-by: shunki-fujita <shunki-fujita@cybozu.co.jp>
2022-07-12 16:16:48 -04:00
YaytayAtWork
1f3e1ec803 #9429: Adding blank line so list is formatted correctly. (#9880)
Signed-off-by: CI <michael@crenshaw.dev>
2022-07-12 16:14:05 -04:00
Jake
c3423e8df3 docs: small fix for plugin stream filtering (#9871)
Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>
2022-07-12 16:13:45 -04:00
Didrik Finnøy
5ef48c1123 docs: Document the possibility of rendering Helm charts with Kustomize (#9841)
* Update kustomize.md

Resolves  #7835.

Signed-off-by: Didrik Finnøy <djfinnoy@protonmail.com>

* Removed unnecessary command flag from example. Minor text edits.

Signed-off-by: Didrik Finnøy <djfinnoy@protonmail.com>

* spelling

Signed-off-by: Didrik Finnøy <djfinnoy@protonmail.com>
2022-07-12 16:13:22 -04:00
YaytayAtWork
e3ae286e2d docs: getting started notes on self-signed cert (#9429) (#9784)
* Fix #9429: A couple of notes in the docs to explain that the default certificate is insecure.

Signed-off-by: Jim Talbut <jim.talbut@groupgti.com>

* Fixes #9429: More verbose, but complete, text for Getting Started.

Signed-off-by: Jim Talbut <jim.talbut@groupgti.com>
2022-07-12 16:12:41 -04:00
Michael Crenshaw
7abf6713b4 test: check for error messages from CI env (#9953)
test: check for error messages from CI env (#9953)

Signed-off-by: CI <michael@crenshaw.dev>
2022-07-12 15:31:44 -04:00
argo-bot
0232073ccf Bump version to 2.4.5 2022-07-12 16:08:04 +00:00
argo-bot
860d8fd7e1 Bump version to 2.4.5 2022-07-12 16:07:57 +00:00
Michael Crenshaw
147ff80543 Merge pull request from GHSA-7943-82jg-wmw5
* add tests to demonstrate issue

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

more

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

docs

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

settings tests

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

tests for OIDC handlers, consolidating test helpers

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

consolidate

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

consolidate

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

docs

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fix log message

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-07-12 08:46:13 -04:00
Michael Crenshaw
3800a1e49d Merge pull request from GHSA-pmjg-52h9-72qv
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

formatting

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fixes from comments

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix test

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-07-12 08:45:22 -04:00
argo-bot
fe80bdcfdc Bump version to 2.4.4 2022-07-07 06:57:43 +00:00
argo-bot
94c09dff59 Bump version to 2.4.4 2022-07-07 06:57:37 +00:00
jannfis
3d9e4d439e test: Remove circular symlinks from testdata (#9886)
* test: Remove circular symlinks from testdata

Signed-off-by: jannfis <jann@mistrust.net>

* Another test case

Signed-off-by: jannfis <jann@mistrust.net>

* Use defer for changing back to original workdir

Signed-off-by: jannfis <jann@mistrust.net>

* Abort the test on error in defer

Signed-off-by: jannfis <jann@mistrust.net>
2022-07-06 19:18:51 +00:00
noah
4a955e25a0 docs: custom secret must be labeled (#9835)
As far as I can tell, this isn't explicitly documented anywhere (some docs mentioned this label in reference to ConfigMaps) I only figure it out by looking at the code.
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-07-01 15:55:34 -04:00
noah
4f99f251bf fix: missing path segments for git file generator (#9839)
* fix: missing path segments for git file generator

given the path:

  /one/two/file.yaml

This change adds the params:

path[1]=two
path.filename=file.yaml
path.filenameNormalized

The use case is for symmetry with other generators (e.g., merge with a directory generator using the key path[1]). As no existing parameter values are changed it should be non-breaking.

Signed-off-by: Noah Perks Sloan <noah@hackedu.io>

* fix: expose all path elements for directory generator

This makes it consistent with the files generator and removes un-intuitive behavior without any breaking change.

docs: clarify - can use either baseName or nth path element
docs: use "directory", like the name of the generator, where "folder" was mentioned
Signed-off-by: Noah Perks Sloan <noah@hackedu.io>
2022-07-01 15:54:15 -04:00
Alexander Matyushentsev
fd418cec0d fix: make sure api server informer does not stop after setting change (#9842)
Signed-off-by: Alexander Matyushentsev <AMatyushentsev@gmail.com>
2022-06-30 15:30:33 -07:00
Xiao Yang
dc291a629f fix: support resource logs and exec (#9833)
Signed-off-by: Xiao Yang <muma.378@163.com>
2022-06-30 09:07:00 -04:00
Paulo Coelho
e23a1c6026 docs: update archlinux install with official package (#9718)
Signed-off-by: Paulo Coelho <prscoelho@protonmail.com>
2022-06-29 16:38:21 -04:00
Chris Reilly
aee88c6452 docs: explain rightmost git generator path parameter behavior (#9799)
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-29 16:31:15 -04:00
KevinSnyderCodes
3f111cc640 git: prune any deleted refs before fetching (#9504)
* git: prune any deleted refers before fetching

This commit modifies `nativeGitClient.Fetch()` to call `git remote prune origin` before fetching refs.

In some cases, an old branch may exist that conflicts with the name of a new branch. The old branch will have been deleted from `origin` but still exist locally in the `argocd-repo-server`.

Example: an old branch `feature/foo` conflicts with a new branch `feature/foo/bar`

In these cases, syncing an application results in the error:

```
rpc error: code = Internal desc = Failed to fetch default: `git fetch origin --tags --force` failed exit status 1: error: cannot lock ref 'refs/remotes/origin/feature/foo/bar': 'refs/remotes/origin/feature/foo' exists; cannot create 'refs/remotes/origin/feature/foo/bar' From https://github.com/org/repo ! [new branch] feature/foo/bar -> origin/feature/foo/bar (unable to update local ref) error: some local refs could not be updated; try running 'git remote prune origin' to remove any old, conflicting branches
```

Adding `git remote prune origin` before fetching, as recommended by the error message, should fix this issue.

The current workaround is to restart the `argocd-repo-server` which should flush the local repository folder. This works when Argo CD is installed using the Helm chart.

Signed-off-by: Kevin Snyder <kevin.snyder.codes@gmail.com>

* fix: added extra protection to syncing app with replace (#9187)

* fix: added extra protection to syncing app with replace

Signed-off-by: ciiay <yicai@redhat.com>

* Code clean up

Signed-off-by: ciiay <yicai@redhat.com>

* Updated logic for isAppOfAppsPattern

Signed-off-by: ciiay <yicai@redhat.com>

* Updated text strings as per comment

Signed-off-by: ciiay <yicai@redhat.com>

* Fixed lint issue

Signed-off-by: ciiay <yicai@redhat.com>
Signed-off-by: Kevin Snyder <kevin.snyder.codes@gmail.com>

* chore: Simplified GetRepoHTTPClient function (#9396)

* chore: Simplified GetRepoHTTPClient function

Signed-off-by: ls0f <lovedboy.tk@qq.com>

* simplified code and improve unit test coverage

Signed-off-by: ls0f <lovedboy.tk@qq.com>
Signed-off-by: Kevin Snyder <kevin.snyder.codes@gmail.com>

* Only prune if fetch error message indicates that it is worthwhile, add unit tests

Confirmed that `Test_nativeGitClient_Fetch_Prune` fails without the bug fix, succeeds with it.

Signed-off-by: Kevin Snyder <kevin.snyder.codes@gmail.com>

* fix: avoid k8s call before authorization for terminal endpoint (#9434)

* fix: avoid k8s API call before authorization in k8s endpoint

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* check for bad project

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* lint

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* more logging

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* handle 404, return 500 instead of 400 for other errors

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* use user input

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* refactor validation

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fix tests

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fixes, tests

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
Signed-off-by: Kevin Snyder <kevin.snyder.codes@gmail.com>

* Match against "try running 'git remote prune origin'"

Signed-off-by: Kevin Snyder <kevin.snyder.codes@gmail.com>

Co-authored-by: Yi Cai <yicai@redhat.com>
Co-authored-by: ls0f <lovedboy.tk@qq.com>
Co-authored-by: Michael Crenshaw <michael@crenshaw.dev>
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-29 16:28:51 -04:00
Jake
596038fc0f fix: configurable CMP tar exclusions (#9675) (#9789)
* feat: configurable CMP tar exclusions

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* rename cmp to plugin

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* add test for tar stream exclusions

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* fix tests

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* update cmp guide

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>

* fully parameterize

Signed-off-by: notfromstatefarm <86763948+notfromstatefarm@users.noreply.github.com>
2022-06-29 16:27:41 -04:00
argo-bot
471685feae Bump version to 2.4.3 2022-06-27 20:53:14 +00:00
argo-bot
42e1f29117 Bump version to 2.4.3 2022-06-27 20:53:07 +00:00
Michael Crenshaw
c089f9a5e5 fix: respect OIDC providers' supported token signing algorithms (#9433) (#9761)
* fix: respect OIDC providers' supported token signing algorithms (#9433)

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* go mod tidy

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-27 16:32:35 -04:00
TomHellier
1dc5e6aaf9 fix: websockets for terminal not working on subPath (#9795)
Signed-off-by: Tom Hellier <me@tomhellier.com>
2022-06-27 13:26:11 -07:00
Alexander Matyushentsev
82726fcbcf fix: avoid closing and re-opening port of api server settings change (#9778)
Signed-off-by: Alexander Matyushentsev <AMatyushentsev@gmail.com>
2022-06-24 10:17:20 -07:00
Nicolas Fillot
e715e085aa [ArgoCD] Fixing webhook typo in case of error in GetManifests (#9671)
Signed-off-by: Nicolas Fillot <nfillot@weborama.com>

Co-authored-by: Nicolas Fillot <nfillot@weborama.com>
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-23 09:27:15 -04:00
Michael Crenshaw
7db5ab71b0 fix: overrides should not appear in the manifest cache key (#9601)
* fix: overrides should not appear in the manifest cache key

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fix Helm regression

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fix test

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* fix test again

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-22 09:24:29 -04:00
argo-bot
c6d0c8baaa Bump version to 2.4.2 2022-06-21 20:32:13 +00:00
argo-bot
2115bf7746 Bump version to 2.4.2 2022-06-21 20:32:08 +00:00
Yaakov Selkowitz
0a2520f511 test/remote: Allow override of base image (#9734)
ubuntu:22.04 bumped the ppc64el baseline to POWER9, rendering this
unusable on existing POWER8 systems.  This allows customization similar
to the top-level Dockerfile.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2022-06-21 15:24:13 -04:00
Soumya Ghosh Dastidar
211e9f6127 fix: updated config file permission requirements for windows (#9732)
* fix: reduced config file permission restriction on windows

Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com>

* fix: updated localconfig tests to check error

Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com>
2022-06-21 15:09:48 -04:00
hritikch24
792e278080 Update sync-options.md (#9687)
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 14:49:10 -04:00
Michael Crenshaw
93bf321002 fix: project filter (#9651) (#9709)
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 14:39:34 -04:00
Joren Zandstra
48c2e0bf21 fix broken symlink in Dockerfile (#9674)
Signed-off-by: Joren Zandstra <jorenzandstra@yahoo.com>
2022-06-21 14:39:26 -04:00
Soumya Ghosh Dastidar
b4b893ca1c fix: updated baseHRefRegex to perform lazy match (#9724)
* fix: updated baseHRefRegex to perform lazy match

Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com>

* feat: added unit tests for replaceBaseHRef

Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com>
2022-06-21 10:39:12 -07:00
argo-bot
52e6025f8b Bump version to 2.4.1 2022-06-21 16:27:47 +00:00
argo-bot
b121b89c81 Bump version to 2.4.1 2022-06-21 16:27:40 +00:00
Michael Crenshaw
12149c0710 chore: fix docs gen
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 10:34:03 -04:00
Michael Crenshaw
9d67469428 Merge pull request from GHSA-jhqp-vf4w-rpwq
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

defer instead of multiple close calls

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

oops

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

don't count jsonnet against max

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix codegen

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

add caveat about 300x ratio

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix versions

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix tests/lint

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 09:40:36 -04:00
Michael Crenshaw
04c3053964 Merge pull request from GHSA-q4w5-4gq2-98vm
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix tests/lint

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 09:39:56 -04:00
Michael Crenshaw
17f7f4f462 Merge pull request from GHSA-2m7h-86qq-fp4v
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix references

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

use long enough state param for oauth2

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

typo

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

more entropy

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix tests

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix tests/lint

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 09:39:01 -04:00
Michael Crenshaw
8bc3ef690d Merge pull request from GHSA-h4w9-6x78-8vrj
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

fix tests/lint

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-21 09:36:38 -04:00
Michael Crenshaw
8fe9a58c21 test: directory app manifest generation (#9503)
* test: directory app manifest generation

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>

* git doesn't support empty dirs

Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-14 16:12:12 -04:00
Leonardo Luz Almeida
269c61a9c8 chore: Implement tests to validate aws auth retry (#9627)
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
2022-06-14 10:11:38 -04:00
Leonardo Luz Almeida
9a08c123f9 chore: Implement a retry in aws auth command (#9618)
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
2022-06-14 10:10:43 -04:00
jannfis
971f0f1ff1 test: Remove temp directories from repo server tests (#9501)
Signed-off-by: jannfis <jann@mistrust.net>
2022-06-13 15:42:29 -04:00
jannfis
1ae2f97b05 test: Make context tests idempodent (#9502)
Signed-off-by: jannfis <jann@mistrust.net>
2022-06-13 15:41:55 -04:00
Michael Crenshaw
bb26cc207b test: fix plugin var test for OSX (#9590)
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-13 15:37:00 -04:00
Christian Hernandez
77d3dcdc62 Update Generators-Git.md (#9632)
Document how to deploy from the root of the git repository

Signed-off-by: Christian Hernandez <christian@codefresh.io>
2022-06-13 10:57:28 -04:00
Hari Sekhon
f414a8e985 docs: added environment variables documentation (#8680)
* added environment variables page

Signed-off-by: Hari Sekhon <harisekhon@gmail.com>

* added Environment Variables page to User Guide

Signed-off-by: Hari Sekhon <harisekhon@gmail.com>

* reformatted table cells as multiline for clarity

Signed-off-by: Hari Sekhon <harisekhon@gmail.com>

* fixed typo

Co-authored-by: Michael Crenshaw <michael@crenshaw.dev>
Signed-off-by: Michael Crenshaw <michael@crenshaw.dev>
2022-06-13 10:38:39 -04:00
167 changed files with 3712 additions and 606 deletions

View File

@@ -63,7 +63,7 @@ RUN ln -s /usr/local/bin/entrypoint.sh /usr/local/bin/uid_entrypoint.sh
# support for mounting configuration from a configmap
WORKDIR /app/config/ssh
RUN touch ssh_known_hosts && \
ln -s ssh_known_hosts /etc/ssh/ssh_known_hosts
ln -s /app/config/ssh/ssh_known_hosts /etc/ssh/ssh_known_hosts
WORKDIR /app/config
RUN mkdir -p tls && \

View File

@@ -37,6 +37,7 @@ Currently, the following organizations are **officially** using Argo CD:
1. [Chargetrip](https://chargetrip.com)
1. [Chime](https://www.chime.com)
1. [Cisco ET&I](https://eti.cisco.com/)
1. [Cobalt](https://www.cobalt.io/)
1. [Codefresh](https://www.codefresh.io/)
1. [Codility](https://www.codility.com/)
1. [Commonbond](https://commonbond.co/)

View File

@@ -1 +1 @@
2.4.0
2.4.6

View File

@@ -162,8 +162,10 @@ func (g *GitGenerator) generateParamsFromGitFile(filePath string, fileContent []
}
params["path"] = path.Dir(filePath)
params["path.basename"] = path.Base(params["path"])
params["path.filename"] = path.Base(filePath)
params["path.basenameNormalized"] = sanitizeName(path.Base(params["path"]))
for k, v := range strings.Split(strings.TrimSuffix(params["path"], params["path.basename"]), "/") {
params["path.filenameNormalized"] = sanitizeName(path.Base(params["path.filename"]))
for k, v := range strings.Split(params["path"], "/") {
if len(v) > 0 {
params["path["+strconv.Itoa(k)+"]"] = v
}
@@ -213,7 +215,7 @@ func (g *GitGenerator) generateParamsFromApps(requestedApps []string, _ *argopro
params["path"] = a
params["path.basename"] = path.Base(a)
params["path.basenameNormalized"] = sanitizeName(path.Base(a))
for k, v := range strings.Split(strings.TrimSuffix(params["path"], params["path.basename"]), "/") {
for k, v := range strings.Split(params["path"], "/") {
if len(v) > 0 {
params["path["+strconv.Itoa(k)+"]"] = v
}

View File

@@ -47,6 +47,28 @@ func (a argoCDServiceMock) GetDirectories(ctx context.Context, repoURL string, r
return args.Get(0).([]string), args.Error(1)
}
func Test_generateParamsFromGitFile(t *testing.T) {
params, err := (*GitGenerator)(nil).generateParamsFromGitFile("path/dir/file_name.yaml", []byte(`
foo:
bar: baz
`))
if err != nil {
t.Fatal(err)
}
assert.Equal(t, []map[string]string{
{
"foo.bar": "baz",
"path": "path/dir",
"path.basename": "dir",
"path.filename": "file_name.yaml",
"path.basenameNormalized": "dir",
"path.filenameNormalized": "file-name.yaml",
"path[0]": "path",
"path[1]": "dir",
},
}, params)
}
func TestGitGenerateParamsFromDirectories(t *testing.T) {
cases := []struct {
@@ -68,9 +90,9 @@ func TestGitGenerateParamsFromDirectories(t *testing.T) {
},
repoError: nil,
expected: []map[string]string{
{"path": "app1", "path.basename": "app1", "path.basenameNormalized": "app1"},
{"path": "app2", "path.basename": "app2", "path.basenameNormalized": "app2"},
{"path": "app_3", "path.basename": "app_3", "path.basenameNormalized": "app-3"},
{"path": "app1", "path.basename": "app1", "path.basenameNormalized": "app1", "path[0]": "app1"},
{"path": "app2", "path.basename": "app2", "path.basenameNormalized": "app2", "path[0]": "app2"},
{"path": "app_3", "path.basename": "app_3", "path.basenameNormalized": "app-3", "path[0]": "app_3"},
},
expectedError: nil,
},
@@ -85,8 +107,8 @@ func TestGitGenerateParamsFromDirectories(t *testing.T) {
},
repoError: nil,
expected: []map[string]string{
{"path": "p1/app2", "path.basename": "app2", "path[0]": "p1", "path.basenameNormalized": "app2"},
{"path": "p1/p2/app3", "path.basename": "app3", "path[0]": "p1", "path[1]": "p2", "path.basenameNormalized": "app3"},
{"path": "p1/app2", "path.basename": "app2", "path[0]": "p1", "path[1]": "app2", "path.basenameNormalized": "app2"},
{"path": "p1/p2/app3", "path.basename": "app3", "path[0]": "p1", "path[1]": "p2", "path[2]": "app3", "path.basenameNormalized": "app3"},
},
expectedError: nil,
},
@@ -102,9 +124,9 @@ func TestGitGenerateParamsFromDirectories(t *testing.T) {
},
repoError: nil,
expected: []map[string]string{
{"path": "app1", "path.basename": "app1", "path.basenameNormalized": "app1"},
{"path": "app2", "path.basename": "app2", "path.basenameNormalized": "app2"},
{"path": "p2/app3", "path.basename": "app3", "path[0]": "p2", "path.basenameNormalized": "app3"},
{"path": "app1", "path.basename": "app1", "path[0]": "app1", "path.basenameNormalized": "app1"},
{"path": "app2", "path.basename": "app2", "path[0]": "app2", "path.basenameNormalized": "app2"},
{"path": "p2/app3", "path.basename": "app3", "path[0]": "p2", "path[1]": "app3", "path.basenameNormalized": "app3"},
},
expectedError: nil,
},
@@ -120,9 +142,9 @@ func TestGitGenerateParamsFromDirectories(t *testing.T) {
},
repoError: nil,
expected: []map[string]string{
{"path": "app1", "path.basename": "app1", "path.basenameNormalized": "app1"},
{"path": "app2", "path.basename": "app2", "path.basenameNormalized": "app2"},
{"path": "p2/app3", "path.basename": "app3", "path[0]": "p2", "path.basenameNormalized": "app3"},
{"path": "app1", "path.basename": "app1", "path[0]": "app1", "path.basenameNormalized": "app1"},
{"path": "app2", "path.basename": "app2", "path[0]": "app2", "path.basenameNormalized": "app2"},
{"path": "p2/app3", "path.basename": "app3", "path[0]": "p2", "path[1]": "app3", "path.basenameNormalized": "app3"},
},
expectedError: nil,
},
@@ -238,7 +260,10 @@ func TestGitGenerateParamsFromFiles(t *testing.T) {
"path": "cluster-config/production",
"path.basename": "production",
"path[0]": "cluster-config",
"path[1]": "production",
"path.basenameNormalized": "production",
"path.filename": "config.json",
"path.filenameNormalized": "config.json",
},
{
"cluster.owner": "foo.bar@example.com",
@@ -247,7 +272,10 @@ func TestGitGenerateParamsFromFiles(t *testing.T) {
"path": "cluster-config/staging",
"path.basename": "staging",
"path[0]": "cluster-config",
"path[1]": "staging",
"path.basenameNormalized": "staging",
"path.filename": "config.json",
"path.filenameNormalized": "config.json",
},
},
expectedError: nil,
@@ -305,7 +333,10 @@ func TestGitGenerateParamsFromFiles(t *testing.T) {
"path": "cluster-config/production",
"path.basename": "production",
"path[0]": "cluster-config",
"path[1]": "production",
"path.basenameNormalized": "production",
"path.filename": "config.json",
"path.filenameNormalized": "config.json",
},
{
"cluster.owner": "john.doe@example.com",
@@ -314,7 +345,10 @@ func TestGitGenerateParamsFromFiles(t *testing.T) {
"path": "cluster-config/production",
"path.basename": "production",
"path[0]": "cluster-config",
"path[1]": "production",
"path.basenameNormalized": "production",
"path.filename": "config.json",
"path.filenameNormalized": "config.json",
},
},
expectedError: nil,
@@ -353,7 +387,10 @@ cluster:
"path": "cluster-config/production",
"path.basename": "production",
"path[0]": "cluster-config",
"path[1]": "production",
"path.basenameNormalized": "production",
"path.filename": "config.yaml",
"path.filenameNormalized": "config.yaml",
},
{
"cluster.owner": "foo.bar@example.com",
@@ -362,7 +399,10 @@ cluster:
"path": "cluster-config/staging",
"path.basename": "staging",
"path[0]": "cluster-config",
"path[1]": "staging",
"path.basenameNormalized": "staging",
"path.filename": "config.yaml",
"path.filenameNormalized": "config.yaml",
},
},
expectedError: nil,
@@ -393,7 +433,10 @@ cluster:
"path": "cluster-config/production",
"path.basename": "production",
"path[0]": "cluster-config",
"path[1]": "production",
"path.basenameNormalized": "production",
"path.filename": "config.yaml",
"path.filenameNormalized": "config.yaml",
},
{
"cluster.owner": "john.doe@example.com",
@@ -402,7 +445,10 @@ cluster:
"path": "cluster-config/production",
"path.basename": "production",
"path[0]": "cluster-config",
"path[1]": "production",
"path.basenameNormalized": "production",
"path.filename": "config.yaml",
"path.filenameNormalized": "config.yaml",
},
},
expectedError: nil,

View File

@@ -1,6 +1,7 @@
package commands
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
@@ -40,16 +41,7 @@ func newAWSCommand() *cobra.Command {
var command = &cobra.Command{
Use: "aws",
Run: func(c *cobra.Command, args []string) {
sess, err := session.NewSession()
errors.CheckError(err)
stsAPI := sts.New(sess)
if roleARN != "" {
creds := stscreds.NewCredentials(sess, roleARN)
stsAPI = sts.New(sess, &aws.Config{Credentials: creds})
}
request, _ := stsAPI.GetCallerIdentityRequest(&sts.GetCallerIdentityInput{})
request.HTTPRequest.Header.Add(clusterIDHeader, clusterName)
presignedURLString, err := request.Presign(requestPresignParam)
presignedURLString, err := getSignedRequestWithRetry(time.Minute, 5*time.Second, clusterName, roleARN, getSignedRequest)
errors.CheckError(err)
token := v1Prefix + base64.RawURLEncoding.EncodeToString([]byte(presignedURLString))
// Set token expiration to 1 minute before the presigned URL expires for some cushion
@@ -62,6 +54,43 @@ func newAWSCommand() *cobra.Command {
return command
}
type getSignedRequestFunc func(clusterName, roleARN string) (string, error)
func getSignedRequestWithRetry(timeout, interval time.Duration, clusterName, roleARN string, fn getSignedRequestFunc) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
for {
signed, err := fn(clusterName, roleARN)
if err == nil {
return signed, nil
}
select {
case <-ctx.Done():
return "", fmt.Errorf("timeout while trying to get signed aws request: last error: %s", err)
case <-time.After(interval):
}
}
}
func getSignedRequest(clusterName, roleARN string) (string, error) {
sess, err := session.NewSession()
if err != nil {
return "", fmt.Errorf("error creating new AWS session: %s", err)
}
stsAPI := sts.New(sess)
if roleARN != "" {
creds := stscreds.NewCredentials(sess, roleARN)
stsAPI = sts.New(sess, &aws.Config{Credentials: creds})
}
request, _ := stsAPI.GetCallerIdentityRequest(&sts.GetCallerIdentityInput{})
request.HTTPRequest.Header.Add(clusterIDHeader, clusterName)
signed, err := request.Presign(requestPresignParam)
if err != nil {
return "", fmt.Errorf("error presigning AWS request: %s", err)
}
return signed, nil
}
func formatJSON(token string, expiration time.Time) string {
expirationTimestamp := metav1.NewTime(expiration)
execInput := &clientauthv1beta1.ExecCredential{

View File

@@ -0,0 +1,73 @@
package commands
import (
"fmt"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestGetSignedRequestWithRetry(t *testing.T) {
t.Run("will return signed request on first attempt", func(t *testing.T) {
// given
t.Parallel()
mock := &signedRequestMock{
returnFunc: func(m *signedRequestMock) (string, error) {
return "token", nil
},
}
// when
signed, err := getSignedRequestWithRetry(time.Second, time.Millisecond, "cluster-name", "", mock.getSignedRequestMock)
// then
assert.NoError(t, err)
assert.Equal(t, "token", signed)
})
t.Run("will return signed request on third attempt", func(t *testing.T) {
// given
t.Parallel()
mock := &signedRequestMock{
returnFunc: func(m *signedRequestMock) (string, error) {
if m.getSignedRequestCalls < 3 {
return "", fmt.Errorf("some error")
}
return "token", nil
},
}
// when
signed, err := getSignedRequestWithRetry(time.Second, time.Millisecond, "cluster-name", "", mock.getSignedRequestMock)
// then
assert.NoError(t, err)
assert.Equal(t, "token", signed)
})
t.Run("will return error on timeout", func(t *testing.T) {
// given
t.Parallel()
mock := &signedRequestMock{
returnFunc: func(m *signedRequestMock) (string, error) {
return "", fmt.Errorf("some error")
},
}
// when
signed, err := getSignedRequestWithRetry(time.Second, time.Millisecond, "cluster-name", "", mock.getSignedRequestMock)
// then
assert.Error(t, err)
assert.Equal(t, "", signed)
})
}
type signedRequestMock struct {
getSignedRequestCalls int
returnFunc func(m *signedRequestMock) (string, error)
}
func (m *signedRequestMock) getSignedRequestMock(clusterName, roleARN string) (string, error) {
m.getSignedRequestCalls++
return m.returnFunc(m)
}

View File

@@ -14,6 +14,7 @@ import (
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"google.golang.org/grpc/health/grpc_health_v1"
"k8s.io/apimachinery/pkg/api/resource"
cmdutil "github.com/argoproj/argo-cd/v2/cmd/util"
"github.com/argoproj/argo-cd/v2/common"
@@ -70,15 +71,17 @@ func getSubmoduleEnabled() bool {
func NewCommand() *cobra.Command {
var (
parallelismLimit int64
listenPort int
metricsPort int
otlpAddress string
cacheSrc func() (*reposervercache.Cache, error)
tlsConfigCustomizer tls.ConfigCustomizer
tlsConfigCustomizerSrc func() (tls.ConfigCustomizer, error)
redisClient *redis.Client
disableTLS bool
parallelismLimit int64
listenPort int
metricsPort int
otlpAddress string
cacheSrc func() (*reposervercache.Cache, error)
tlsConfigCustomizer tls.ConfigCustomizer
tlsConfigCustomizerSrc func() (tls.ConfigCustomizer, error)
redisClient *redis.Client
disableTLS bool
maxCombinedDirectoryManifestsSize string
cmpTarExcludedGlobs []string
)
var command = cobra.Command{
Use: cliName,
@@ -98,15 +101,20 @@ func NewCommand() *cobra.Command {
cache, err := cacheSrc()
errors.CheckError(err)
maxCombinedDirectoryManifestsQuantity, err := resource.ParseQuantity(maxCombinedDirectoryManifestsSize)
errors.CheckError(err)
askPassServer := askpass.NewServer()
metricsServer := metrics.NewMetricsServer()
cacheutil.CollectMetrics(redisClient, metricsServer)
server, err := reposerver.NewServer(metricsServer, cache, tlsConfigCustomizer, repository.RepoServerInitConstants{
ParallelismLimit: parallelismLimit,
ParallelismLimit: parallelismLimit,
PauseGenerationAfterFailedGenerationAttempts: getPauseGenerationAfterFailedGenerationAttempts(),
PauseGenerationOnFailureForMinutes: getPauseGenerationOnFailureForMinutes(),
PauseGenerationOnFailureForRequests: getPauseGenerationOnFailureForRequests(),
SubmoduleEnabled: getSubmoduleEnabled(),
MaxCombinedDirectoryManifestsSize: maxCombinedDirectoryManifestsQuantity,
CMPTarExcludedGlobs: cmpTarExcludedGlobs,
}, askPassServer)
errors.CheckError(err)
@@ -182,6 +190,8 @@ func NewCommand() *cobra.Command {
command.Flags().IntVar(&metricsPort, "metrics-port", common.DefaultPortRepoServerMetrics, "Start metrics server on given port")
command.Flags().StringVar(&otlpAddress, "otlp-address", env.StringFromEnv("ARGOCD_REPO_SERVER_OTLP_ADDRESS", ""), "OpenTelemetry collector address to send traces to")
command.Flags().BoolVar(&disableTLS, "disable-tls", env.ParseBoolFromEnv("ARGOCD_REPO_SERVER_DISABLE_TLS", false), "Disable TLS on the gRPC endpoint")
command.Flags().StringVar(&maxCombinedDirectoryManifestsSize, "max-combined-directory-manifests-size", env.StringFromEnv("ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE", "10M"), "Max combined size of manifest files in a directory-type Application")
command.Flags().StringArrayVar(&cmpTarExcludedGlobs, "plugin-tar-exclude", env.StringsFromEnv("ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS", []string{}, ";"), "Globs to filter when sending tarballs to plugins.")
tlsConfigCustomizerSrc = tls.AddTLSFlagsToCmd(&command)
cacheSrc = reposervercache.AddCacheFlagsToCmd(&command, func(client *redis.Client) {

View File

@@ -152,7 +152,10 @@ func NewCommand() *cobra.Command {
stats.RegisterStackDumper()
stats.StartStatsTicker(10 * time.Minute)
stats.RegisterHeapDumper("memprofile")
argocd := server.NewServer(context.Background(), argoCDOpts)
argocd.Init(context.Background())
lns, err := argocd.Listen()
errors.CheckError(err)
for {
var closer func()
ctx := context.Background()
@@ -163,8 +166,7 @@ func NewCommand() *cobra.Command {
log.Fatalf("failed to initialize tracing: %v", err)
}
}
argocd := server.NewServer(ctx, argoCDOpts)
argocd.Run(ctx, listenPort, metricsPort)
argocd.Run(ctx, lns)
cancel()
if closer != nil {
closer()

View File

@@ -25,6 +25,7 @@ import (
"github.com/spf13/cobra"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"k8s.io/apimachinery/pkg/api/resource"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/types"
@@ -778,7 +779,7 @@ func getLocalObjectsString(app *argoappv1.Application, local, localRepoRoot, app
ApiVersions: apiVersions,
Plugins: configManagementPlugins,
TrackingMethod: trackingMethod,
}, true, &git.NoopCredsStore{})
}, true, &git.NoopCredsStore{}, resource.MustParse("0"))
errors.CheckError(err)
return res.Manifests

View File

@@ -36,13 +36,13 @@ users:
- auth-token: vErrYS3c3tReFRe$hToken
name: localhost:8080`
const testConfigFilePath = "./testdata/config"
const testConfigFilePath = "./testdata/local.config"
func TestContextDelete(t *testing.T) {
// Write the test config file
err := ioutil.WriteFile(testConfigFilePath, []byte(testConfig), os.ModePerm)
assert.NoError(t, err)
defer os.Remove(testConfigFilePath)
err = os.Chmod(testConfigFilePath, 0600)
require.NoError(t, err, "Could not change the file permission to 0600 %v", err)
@@ -75,9 +75,4 @@ func TestContextDelete(t *testing.T) {
assert.NotContains(t, localConfig.Servers, localconfig.Server{PlainText: true, Server: "localhost:8080"})
assert.NotContains(t, localConfig.Users, localconfig.User{AuthToken: "vErrYS3c3tReFRe$hToken", Name: "localhost:8080"})
assert.Contains(t, localConfig.Contexts, localconfig.ContextRef{Name: "argocd2.example.com:443", Server: "argocd2.example.com:443", User: "argocd2.example.com:443"})
// Write the file again so that no conflicts are made in git
err = ioutil.WriteFile(testConfigFilePath, []byte(testConfig), os.ModePerm)
assert.NoError(t, err)
}

View File

@@ -215,8 +215,13 @@ func StartLocalServer(clientOpts *apiclient.ClientOptions, ctxStr string, port *
ListenHost: *address,
RepoClientset: &forwardRepoClientset{namespace: namespace, context: ctxStr},
})
srv.Init(ctx)
go srv.Run(ctx, *port, 0)
lns, err := srv.Listen()
if err != nil {
return err
}
go srv.Run(ctx, lns)
clientOpts.ServerAddr = fmt.Sprintf("%s:%d", *address, *port)
clientOpts.PlainText = true
if !cache2.WaitForCacheSync(ctx.Done(), srv.Initialized) {

View File

@@ -68,7 +68,8 @@ argocd login cd.argoproj.io --core`,
server = "kubernetes"
} else {
server = args[0]
tlsTestResult, err := grpc_util.TestTLS(server)
dialTime := 30 * time.Second
tlsTestResult, err := grpc_util.TestTLS(server, dialTime)
errors.CheckError(err)
if !tlsTestResult.TLS {
if !globalClientOpts.PlainText {
@@ -202,7 +203,10 @@ func oauth2Login(ctx context.Context, port int, oidcSettings *settingspkg.OIDCCo
// completionChan is to signal flow completed. Non-empty string indicates error
completionChan := make(chan string)
// stateNonce is an OAuth2 state nonce
stateNonce := rand.RandString(10)
// According to the spec (https://www.rfc-editor.org/rfc/rfc6749#section-10.10), this must be guessable with
// probability <= 2^(-128). The following call generates one of 52^24 random strings, ~= 2^136 possibilities.
stateNonce, err := rand.String(24)
errors.CheckError(err)
var tokenString string
var refreshToken string
@@ -212,7 +216,8 @@ func oauth2Login(ctx context.Context, port int, oidcSettings *settingspkg.OIDCCo
}
// PKCE implementation of https://tools.ietf.org/html/rfc7636
codeVerifier := rand.RandStringCharset(43, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~")
codeVerifier, err := rand.StringFromCharset(43, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~")
errors.CheckError(err)
codeChallengeHash := sha256.Sum256([]byte(codeVerifier))
codeChallenge := base64.RawURLEncoding.EncodeToString(codeChallengeHash[:])
@@ -296,7 +301,8 @@ func oauth2Login(ctx context.Context, port int, oidcSettings *settingspkg.OIDCCo
opts = append(opts, oauth2.SetAuthURLParam("code_challenge_method", "S256"))
url = oauth2conf.AuthCodeURL(stateNonce, opts...)
case oidcutil.GrantTypeImplicit:
url = oidcutil.ImplicitFlowURL(oauth2conf, stateNonce, opts...)
url, err = oidcutil.ImplicitFlowURL(oauth2conf, stateNonce, opts...)
errors.CheckError(err)
default:
log.Fatalf("Unsupported grant type: %v", grantType)
}

View File

@@ -6,6 +6,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
argocdclient "github.com/argoproj/argo-cd/v2/pkg/apiclient"
"github.com/argoproj/argo-cd/v2/util/localconfig"
@@ -16,6 +17,10 @@ func TestLogout(t *testing.T) {
// Write the test config file
err := ioutil.WriteFile(testConfigFilePath, []byte(testConfig), os.ModePerm)
assert.NoError(t, err)
defer os.Remove(testConfigFilePath)
err = os.Chmod(testConfigFilePath, 0600)
require.NoError(t, err)
localConfig, err := localconfig.ReadLocalConfig(testConfigFilePath)
assert.NoError(t, err)
@@ -32,9 +37,4 @@ func TestLogout(t *testing.T) {
assert.Contains(t, localConfig.Contexts, localconfig.ContextRef{Name: "argocd1.example.com:443", Server: "argocd1.example.com:443", User: "argocd1.example.com:443"})
assert.Contains(t, localConfig.Contexts, localconfig.ContextRef{Name: "argocd2.example.com:443", Server: "argocd2.example.com:443", User: "argocd2.example.com:443"})
assert.Contains(t, localConfig.Contexts, localconfig.ContextRef{Name: "localhost:8080", Server: "localhost:8080", User: "localhost:8080"})
// Write the file again so that no conflicts are made in git
err = ioutil.WriteFile(testConfigFilePath, []byte(testConfig), os.ModePerm)
assert.NoError(t, err)
}

View File

@@ -195,7 +195,8 @@ func TestMatchRepository(t *testing.T) {
d := Discover{
Find: Find{
Command: Command{
Command: []string{"sh", "-c", "echo -n $ENV_NO_VAR"},
// Use printf instead of echo since OSX prints the "-n" when there's no additional arg.
Command: []string{"sh", "-c", `printf "%s" "$ENV_NO_VAR"`},
},
},
}

View File

@@ -2,12 +2,13 @@ package cmpserver
import (
"fmt"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"net"
"os"
"os/signal"
"syscall"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
grpc_middleware "github.com/grpc-ecosystem/go-grpc-middleware"
grpc_logrus "github.com/grpc-ecosystem/go-grpc-middleware/logging/logrus"
grpc_prometheus "github.com/grpc-ecosystem/go-grpc-prometheus"
@@ -24,6 +25,7 @@ import (
"github.com/argoproj/argo-cd/v2/server/version"
"github.com/argoproj/argo-cd/v2/util/errors"
grpc_util "github.com/argoproj/argo-cd/v2/util/grpc"
"google.golang.org/grpc/keepalive"
)
// ArgoCDCMPServer is the config management plugin server implementation
@@ -61,6 +63,11 @@ func NewServer(initConstants plugin.CMPServerInitConstants) (*ArgoCDCMPServer, e
grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(streamInterceptors...)),
grpc.MaxRecvMsgSize(apiclient.MaxGRPCMessageSize),
grpc.MaxSendMsgSize(apiclient.MaxGRPCMessageSize),
grpc.KeepaliveEnforcementPolicy(
keepalive.EnforcementPolicy{
MinTime: common.GRPCKeepAliveEnforcementMinimum,
},
),
}
return &ArgoCDCMPServer{

View File

@@ -286,3 +286,10 @@ const (
// AnnotationApplicationRefresh is an annotation that is added when an ApplicationSet is requested to be refreshed by a webhook. The ApplicationSet controller will remove this annotation at the end of reconcilation.
AnnotationApplicationSetRefresh = "argocd.argoproj.io/application-set-refresh"
)
// gRPC settings
const (
GRPCKeepAliveEnforcementMinimum = 10 * time.Second
// Keep alive is 2x enforcement minimum to ensure network jitter does not introduce ENHANCE_YOUR_CALM errors
GRPCKeepAliveTime = 2 * GRPCKeepAliveEnforcementMinimum
)

View File

@@ -176,6 +176,7 @@ func NewLiveStateCache(
type cacheSettings struct {
clusterSettings clustercache.Settings
appInstanceLabelKey string
trackingMethod appv1.TrackingMethod
}
type liveStateCache struct {
@@ -210,7 +211,7 @@ func (c *liveStateCache) loadCacheSettings() (*cacheSettings, error) {
ResourceHealthOverride: lua.ResourceHealthOverrides(resourceOverrides),
ResourcesFilter: resourcesFilter,
}
return &cacheSettings{clusterSettings, appInstanceLabelKey}, nil
return &cacheSettings{clusterSettings, appInstanceLabelKey, argo.GetTrackingMethod(c.settingsMgr)}, nil
}
func asResourceNode(r *clustercache.Resource) appv1.ResourceNode {
@@ -354,7 +355,8 @@ func isTransientNetworkErr(err error) bool {
}
if strings.Contains(errorString, "net/http: TLS handshake timeout") ||
strings.Contains(errorString, "i/o timeout") ||
strings.Contains(errorString, "connection timed out") {
strings.Contains(errorString, "connection timed out") ||
strings.Contains(errorString, "connection reset by peer") {
return true
}
return false
@@ -387,7 +389,6 @@ func (c *liveStateCache) getCluster(server string) (clustercache.ClusterCache, e
return nil, fmt.Errorf("controller is configured to ignore cluster %s", cluster.Server)
}
trackingMethod := argo.GetTrackingMethod(c.settingsMgr)
clusterCacheOpts := []clustercache.UpdateSettingsFunc{
clustercache.SetListSemaphore(semaphore.NewWeighted(clusterCacheListSemaphoreSize)),
clustercache.SetListPageSize(clusterCacheListPageSize),
@@ -400,9 +401,12 @@ func (c *liveStateCache) getCluster(server string) (clustercache.ClusterCache, e
clustercache.SetPopulateResourceInfoHandler(func(un *unstructured.Unstructured, isRoot bool) (interface{}, bool) {
res := &ResourceInfo{}
populateNodeInfo(un, res)
c.lock.RLock()
cacheSettings := c.cacheSettings
c.lock.RUnlock()
res.Health, _ = health.GetResourceHealth(un, cacheSettings.clusterSettings.ResourceHealthOverride)
appName := c.resourceTracking.GetAppName(un, cacheSettings.appInstanceLabelKey, trackingMethod)
appName := c.resourceTracking.GetAppName(un, cacheSettings.appInstanceLabelKey, cacheSettings.trackingMethod)
if isRoot && appName != "" {
res.AppName = appName
}

View File

@@ -111,6 +111,7 @@ func TestIsRetryableError(t *testing.T) {
tlsHandshakeTimeoutErr net.Error = netError("net/http: TLS handshake timeout")
ioTimeoutErr net.Error = netError("i/o timeout")
connectionTimedout net.Error = netError("connection timed out")
connectionReset net.Error = netError("connection reset by peer")
)
t.Run("Nil", func(t *testing.T) {
assert.False(t, isRetryableError(nil))
@@ -148,4 +149,7 @@ func TestIsRetryableError(t *testing.T) {
t.Run("ConnectionTimeout", func(t *testing.T) {
assert.True(t, isRetryableError(connectionTimedout))
})
t.Run("ConnectionReset", func(t *testing.T) {
assert.True(t, isRetryableError(connectionReset))
})
}

View File

@@ -494,6 +494,8 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *ap
}
gvk := obj.GroupVersionKind()
isSelfReferencedObj := m.isSelfReferencedObj(liveObj, appLabelKey, trackingMethod)
resState := v1alpha1.ResourceStatus{
Namespace: obj.GetNamespace(),
Name: obj.GetName(),
@@ -501,7 +503,7 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *ap
Version: gvk.Version,
Group: gvk.Group,
Hook: hookutil.IsHook(obj),
RequiresPruning: targetObj == nil && liveObj != nil,
RequiresPruning: targetObj == nil && liveObj != nil && isSelfReferencedObj,
}
var diffResult diff.DiffResult
@@ -510,8 +512,11 @@ func (m *appStateManager) CompareAppState(app *v1alpha1.Application, project *ap
} else {
diffResult = diff.DiffResult{Modified: false, NormalizedLive: []byte("{}"), PredictedLive: []byte("{}")}
}
if resState.Hook || ignore.Ignore(obj) || (targetObj != nil && hookutil.Skip(targetObj)) {
// For resource hooks or skipped resources, don't store sync status, and do not affect overall sync status
if resState.Hook || ignore.Ignore(obj) || (targetObj != nil && hookutil.Skip(targetObj)) || !isSelfReferencedObj {
// For resource hooks, skipped resources or objects that may have
// been created by another controller with annotations copied from
// the source object, don't store sync status, and do not affect
// overall sync status
} else if diffResult.Modified || targetObj == nil || liveObj == nil {
// Set resource state to OutOfSync since one of the following is true:
// * target and live resource are different
@@ -667,3 +672,33 @@ func NewAppStateManager(
resourceTracking: resourceTracking,
}
}
// isSelfReferencedObj returns whether the given obj is managed by the application
// according to the values in the tracking annotation. It returns true when all
// of the properties in the annotation (name, namespace, group and kind) match
// the properties of the inspected object, or if the tracking method used does
// not provide the required properties for matching.
func (m *appStateManager) isSelfReferencedObj(obj *unstructured.Unstructured, appLabelKey string, trackingMethod v1alpha1.TrackingMethod) bool {
if obj == nil {
return true
}
// If tracking method doesn't contain required metadata for this check,
// we are not able to determine and just assume the object to be managed.
if trackingMethod == argo.TrackingMethodLabel {
return true
}
// In order for us to assume obj to be managed by this application, the
// values from the annotation have to match the properties from the live
// object.
appInstance := m.resourceTracking.GetAppInstance(obj, appLabelKey, trackingMethod)
if appInstance != nil {
return obj.GetNamespace() == appInstance.Namespace &&
obj.GetName() == appInstance.Name &&
obj.GetObjectKind().GroupVersionKind().Group == appInstance.Group &&
obj.GetObjectKind().GroupVersionKind().Kind == appInstance.Kind
}
return true
}

View File

@@ -13,6 +13,7 @@ import (
. "github.com/argoproj/gitops-engine/pkg/utils/testing"
"github.com/stretchr/testify/assert"
v1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
@@ -21,6 +22,7 @@ import (
argoappv1 "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/reposerver/apiclient"
"github.com/argoproj/argo-cd/v2/test"
"github.com/argoproj/argo-cd/v2/util/argo"
)
// TestCompareAppStateEmpty tests comparison when both git and live have no objects
@@ -770,3 +772,128 @@ func TestComparisonResult_GetSyncStatus(t *testing.T) {
assert.Equal(t, status, res.GetSyncStatus())
}
func TestIsLiveResourceManaged(t *testing.T) {
managedObj := kube.MustToUnstructured(&corev1.ConfigMap{
TypeMeta: metav1.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
ObjectMeta: metav1.ObjectMeta{
Name: "configmap1",
Namespace: "default",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: "guestbook:/ConfigMap:default/configmap1",
},
},
})
managedObjWithLabel := kube.MustToUnstructured(&corev1.ConfigMap{
TypeMeta: metav1.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
ObjectMeta: metav1.ObjectMeta{
Name: "configmap1",
Namespace: "default",
Labels: map[string]string{
common.LabelKeyAppInstance: "guestbook",
},
},
})
unmanagedObjWrongName := kube.MustToUnstructured(&corev1.ConfigMap{
TypeMeta: metav1.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
ObjectMeta: metav1.ObjectMeta{
Name: "configmap2",
Namespace: "default",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: "guestbook:/ConfigMap:default/configmap1",
},
},
})
unmanagedObjWrongKind := kube.MustToUnstructured(&corev1.ConfigMap{
TypeMeta: metav1.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
ObjectMeta: metav1.ObjectMeta{
Name: "configmap2",
Namespace: "default",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: "guestbook:/Service:default/configmap2",
},
},
})
unmanagedObjWrongGroup := kube.MustToUnstructured(&corev1.ConfigMap{
TypeMeta: metav1.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
ObjectMeta: metav1.ObjectMeta{
Name: "configmap2",
Namespace: "default",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: "guestbook:apps/ConfigMap:default/configmap2",
},
},
})
unmanagedObjWrongNamespace := kube.MustToUnstructured(&corev1.ConfigMap{
TypeMeta: metav1.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
ObjectMeta: metav1.ObjectMeta{
Name: "configmap2",
Namespace: "default",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: "guestbook:/ConfigMap:fakens/configmap2",
},
},
})
ctrl := newFakeController(&fakeData{
apps: []runtime.Object{app, &defaultProj},
manifestResponse: &apiclient.ManifestResponse{
Manifests: []string{},
Namespace: test.FakeDestNamespace,
Server: test.FakeClusterURL,
Revision: "abc123",
},
managedLiveObjs: map[kube.ResourceKey]*unstructured.Unstructured{
kube.GetResourceKey(managedObj): managedObj,
kube.GetResourceKey(unmanagedObjWrongName): unmanagedObjWrongName,
kube.GetResourceKey(unmanagedObjWrongKind): unmanagedObjWrongKind,
kube.GetResourceKey(unmanagedObjWrongGroup): unmanagedObjWrongGroup,
kube.GetResourceKey(unmanagedObjWrongNamespace): unmanagedObjWrongNamespace,
},
})
manager := ctrl.appStateManager.(*appStateManager)
// Managed resource w/ annotations
assert.True(t, manager.isSelfReferencedObj(managedObj, common.AnnotationKeyAppInstance, argo.TrackingMethodLabel))
assert.True(t, manager.isSelfReferencedObj(managedObj, common.AnnotationKeyAppInstance, argo.TrackingMethodAnnotation))
// Managed resource w/ label
assert.True(t, manager.isSelfReferencedObj(managedObjWithLabel, common.AnnotationKeyAppInstance, argo.TrackingMethodLabel))
// Wrong resource name
assert.True(t, manager.isSelfReferencedObj(unmanagedObjWrongName, common.AnnotationKeyAppInstance, argo.TrackingMethodLabel))
assert.False(t, manager.isSelfReferencedObj(unmanagedObjWrongName, common.AnnotationKeyAppInstance, argo.TrackingMethodAnnotation))
// Wrong resource group
assert.True(t, manager.isSelfReferencedObj(unmanagedObjWrongGroup, common.AnnotationKeyAppInstance, argo.TrackingMethodLabel))
assert.False(t, manager.isSelfReferencedObj(unmanagedObjWrongGroup, common.AnnotationKeyAppInstance, argo.TrackingMethodAnnotation))
// Wrong resource kind
assert.True(t, manager.isSelfReferencedObj(unmanagedObjWrongKind, common.AnnotationKeyAppInstance, argo.TrackingMethodLabel))
assert.False(t, manager.isSelfReferencedObj(unmanagedObjWrongKind, common.AnnotationKeyAppInstance, argo.TrackingMethodAnnotation))
// Wrong resource namespace
assert.True(t, manager.isSelfReferencedObj(unmanagedObjWrongNamespace, common.AnnotationKeyAppInstance, argo.TrackingMethodLabel))
assert.False(t, manager.isSelfReferencedObj(unmanagedObjWrongNamespace, common.AnnotationKeyAppInstance, argo.TrackingMethodAnnotationAndLabel))
// Nil resource
assert.True(t, manager.isSelfReferencedObj(nil, common.AnnotationKeyAppInstance, argo.TrackingMethodAnnotation))
}

View File

@@ -149,7 +149,13 @@ func (m *appStateManager) SyncAppState(app *v1alpha1.Application, state *v1alpha
}
atomic.AddUint64(&syncIdPrefix, 1)
syncId := fmt.Sprintf("%05d-%s", syncIdPrefix, rand.RandString(5))
randSuffix, err := rand.String(5)
if err != nil {
state.Phase = common.OperationError
state.Message = fmt.Sprintf("Failed generate random sync ID: %v", err)
return
}
syncId := fmt.Sprintf("%05d-%s", syncIdPrefix, randSuffix)
logEntry := log.WithFields(log.Fields{"application": app.Name, "syncId": syncId})
initialResourcesRes := make([]common.ResourceSyncResult, 0)

BIN
docs/assets/terminal.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 145 KiB

View File

@@ -4,10 +4,10 @@ You can download the latest Argo CD version from [the latest release page of thi
## Linux and WSL
### ArchLinux User Repository ([AUR](https://aur.archlinux.org/packages/))
### ArchLinux
```bash
yay -Sy argocd-bin
pacman -S argocd
```
### Homebrew

View File

@@ -174,7 +174,9 @@ argocd ... --grpc-web
## Why Am I Getting `x509: certificate signed by unknown authority` When Using The CLI?
Your not running your server with correct certs.
The certificate created by default by Argo CD is not automatically recognised by the Argo CD CLI, in order
to create a secure system you must follow the instructions to [install a certificate](/operator-manual/tls/)
and configure your client OS to trust that certificate.
If you're not running in a production system (e.g. you're testing Argo CD out), try the `--insecure` flag:

View File

@@ -29,6 +29,13 @@ kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/core-install.yaml
```
This default installation will have a self-signed certificate and cannot be accessed without a bit of extra work.
Do one of:
* Follow the [instructions to configure a certificate](./operator-manual/tls) (and ensure that the client OS trusts it).
* Configure the client OS to trust the self signed certificate.
* Use the --insecure flag on all Argo CD CLI operations in this guide.
Use `argocd login --core` to [configure](./user-guide/commands/argocd_login.md) CLI access and skip steps 3-5.
## 2. Download Argo CD CLI

View File

@@ -65,13 +65,15 @@ The generator parameters are:
- `{{path.basename}}`: For any directory path within the Git repository that matches the `path` wildcard, the right-most path name is extracted (e.g. `/directory/directory2` would produce `directory2`).
- `{{path.basenameNormalized}}`: This field is the same as `path.basename` with unsupported characters replaced with `-` (e.g. a `path` of `/directory/directory_2`, and `path.basename` of `directory_2` would produce `directory-2` here).
Whenever a new Helm chart/Kustomize YAML/Application/plain subfolder is added to the Git repository, the ApplicationSet controller will detect this change and automatically deploy the resulting manifests within new `Application` resources.
**Note**: The right-most path name always becomes `{{path.basename}}`. For example, for `- path: /one/two/three/four`, `{{path.basename}}` is `four`.
Whenever a new Helm chart/Kustomize YAML/Application/plain subdirectory is added to the Git repository, the ApplicationSet controller will detect this change and automatically deploy the resulting manifests within new `Application` resources.
As with other generators, clusters *must* already be defined within Argo CD, in order to generate Applications for them.
### Exclude directories
The Git directory generator will automatically exclude folders that begin with `.` (such as `.git`).
The Git directory generator will automatically exclude directories that begin with `.` (such as `.git`).
The Git directory generator also supports an `exclude` option in order to exclude directories in the repository from being scanned by the ApplicationSet controller:
@@ -105,7 +107,7 @@ spec:
```
(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/examples/applicationset/git-generator-directory/excludes).*)
This example excludes the `exclude-helm-guestbook` directory from the list of directories scanned for this `ApplictionSet` resource.
This example excludes the `exclude-helm-guestbook` directory from the list of directories scanned for this `ApplicationSet` resource.
!!! note "Exclude rules have higher priority than include rules"
@@ -150,6 +152,41 @@ Or, a shorter way (using [path.Match](https://golang.org/pkg/path/#Match) syntax
exclude: true
```
### Root Of Git Repo
The Git directory generator can be configured to deploy from the root of the git repository by providing `'*'` as the `path`.
To exclude directories, you only need to put the name/[path.Match](https://golang.org/pkg/path/#Match) of the directory you do not want to deploy.
```yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: cluster-addons
namespace: argocd
spec:
generators:
- git:
repoURL: https://github.com/example/example-repo.git
revision: HEAD
directories:
- path: '*'
- path: donotdeploy
exclude: true
template:
metadata:
name: '{{path.basename}}'
spec:
project: "my-project"
source:
repoURL: https://github.com/example/example-repo.git
targetRevision: HEAD
path: '{{path}}'
destination:
server: https://kubernetes.default.svc
namespace: '{{path.basename}}'
```
## Git Generator: Files
The Git file generator is the second subtype of the Git generator. The Git file generator generates parameters using the contents of JSON/YAML files found within a specified repository.
@@ -170,7 +207,7 @@ Suppose you have a Git repository with the following directory structure:
└── git-generator-files.yaml
```
The folders are:
The directories are:
- `guestbook` contains the Kubernetes resources for a simple guestbook application
- `cluster-config` contains JSON/YAML files describing the individual engineering clusters: one for `dev` and one for `prod`.
@@ -234,10 +271,16 @@ As with other generators, clusters *must* already be defined within Argo CD, in
In addition to the flattened key/value pairs from the configuration file, the following generator parameters are provided:
- `{{path}}`: The path to the folder containing matching configuration file within the Git repository. Example: `/clusters/clusterA`, if the config file was `/clusters/clusterA/config.json`
- `{{path}}`: The path to the directory containing matching configuration file within the Git repository. Example: `/clusters/clusterA`, if the config file was `/clusters/clusterA/config.json`
- `{{path[n]}}`: The path to the matching configuration file within the Git repository, split into array elements (`n` - array index). Example: `path[0]: clusters`, `path[1]: clusterA`
- `{{path.basename}}`: Basename of the path to the folder containing the configuration file (e.g. `clusterA`, with the above example.)
- `{{path.basename}}`: Basename of the path to the directory containing the configuration file (e.g. `clusterA`, with the above example.)
- `{{path.basenameNormalized}}`: This field is the same as `path.basename` with unsupported characters replaced with `-` (e.g. a `path` of `/directory/directory_2`, and `path.basename` of `directory_2` would produce `directory-2` here).
- `{{path.filename}}`: The matched filename. e.g., `config.json` in the above example.
- `{{path.filenameNormalized}}`: The matched filename with unsupported characters replaced with `-`.
**Note**: The right-most *directory* name always becomes `{{path.basename}}`. For example, from `- path: /one/two/three/four/config.json`, `{{path.basename}}` will be `four`.
The filename can always be accessed using `{{path.filename}}`.
## Webhook Configuration

View File

@@ -39,6 +39,8 @@ data:
# The URLs to download additional ArgoCD binaries (besides the Linux amd64 binary included by default)
# for different OS architectures. If provided, additional download buttons will be displayed on the help page.
help.download.linux-arm64: "path-or-url-to-download"
help.download.linux-ppc64le: "path-or-url-to-download"
help.download.linux-s390x: "path-or-url-to-download"
help.download.darwin-amd64: "path-or-url-to-download"
help.download.darwin-arm64: "path-or-url-to-download"
help.download.windows-amd64: "path-or-url-to-download"
@@ -231,6 +233,14 @@ data:
# If omitted, Argo CD injects the app name into the label: 'app.kubernetes.io/instance'
application.instanceLabelKey: mycompany.com/appname
# You can change the resource tracking method Argo CD uses by changing the
# setting application.resourceTrackingMethod to the desired method.
# The following methods are available:
# - label : Uses the application.instanceLabelKey label for tracking
# - annotation : Uses an annotation with additional metadata for tracking instead of the label
# - annotation+label : Also uses an annotation for tracking, but additionally labels the resource with the application name
application.resourceTrackingMethod: annotation
# disables admin user. Admin is enabled by default
admin.enabled: "false"
# add an additional local user with apiKey and login capabilities
@@ -288,3 +298,12 @@ data:
# exec.enabled indicates whether the UI exec feature is enabled. It is disabled by default.
exec.enabled: "false"
# exec.shells restricts which shells are allowed for `exec`, and in which order they are attempted
exec.shells: "bash,sh,powershell,cmd"
# oidc.tls.insecure.skip.verify determines whether certificate verification is skipped when verifying tokens with the
# configured OIDC provider (either external or the bundled Dex instance). Setting this to "true" will cause JWT
# token verification to pass despite the OIDC provider having an invalid certificate. Only set to "true" if you
# understand the risks.
oidc.tls.insecure.skip.verify: "false"

View File

@@ -106,4 +106,10 @@ data:
reposerver.repo.cache.expiration: "24h0m0s"
# Cache expiration default (default 24h0m0s)
reposerver.default.cache.expiration: "24h0m0s"
# Max combined manifest file size for a single directory-type Application. In-memory manifest representation may be as
# much as 300x the manifest file size. Limit this to stay within the memory limits of the repo-server while allowing
# for 300x memory expansion and N Applications running at the same time.
# (example 10M max * 300 expansion * 10 Apps = 30G max theoretical memory usage).
reposerver.max.combined.directory.manifests.size: '10M'
# Paths to be excluded from the tarball streamed to plugins. Separate with ;
reposerver.plugin.tar.exclusions: ""

View File

@@ -59,22 +59,7 @@ also use glob patterns in the action path: `action/*` (or regex patterns if you
`exec` is a special resource. When enabled with the `create` action, this privilege allows a user to `exec` into Pods via
the Argo CD UI. The functionality is similar to `kubectl exec`.
`exec` is a powerful privilege. It allows the user to run arbitrary code on any Pod managed by an Application for which
they have `create` privileges. If the Pod mounts a ServiceAccount token (which is the default behavior of Kubernetes),
then the user effectively has the same privileges as that ServiceAccount.
The exec feature is disabled entirely by default. To enable it, set the `exec.enabled` key to "true" on the argocd-cm
ConfigMap. You will also need to add the following to the argocd-api-server Role (if you're using Argo CD in namespaced
mode) or ClusterRole (if you're using Argo CD in cluster mode).
```yaml
- apiGroups:
- ""
resources:
- pods/exec
verbs:
- create
```
See [Web-based Terminal](web_based_terminal.md) for more info.
## Tying It All Together

View File

@@ -223,3 +223,44 @@ to configure IP addresses logging in the proxy server that sits in front of the
Argo CD's ApplicationSets feature has its own [security considerations](./applicationset/Security.md). Be aware of those
issues before using ApplicationSets.
## Limiting Directory App Memory Usage
> >2.2.10, 2.1.16, >2.3.5
Directory-type Applications (those whose source is raw JSON or YAML files) can consume significant
[repo-server](architecture.md#repository-server) memory, depending on the size and structure of the YAML files.
To avoid over-using memory in the repo-server (potentially causing a crash and denial of service), set the
`reposerver.max.combined.directory.manifests.size` config option in [argocd-cmd-params-cm](argocd-cmd-params-cm.yaml).
This option limits the combined size of all JSON or YAML files in an individual app. Note that the in-memory
representation of a manifest may be as much as 300x the size of the manifest on disk. Also note that the limit is per
Application. If manifests are generated for multiple applications at once, memory usage will be higher.
**Example:**
Suppose your repo-server has a 10G memory limit, and you have ten Applications which use raw JSON or YAML files. To
calculate the max safe combined file size per Application, divide 10G by 300 * 10 Apps (300 being the worst-case memory
growth factor for the manifests).
```
10G / 300 * 10 = 3M
```
So a reasonably safe configuration for this setup would be a 3M limit per app.
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cmd-params-cm
data:
reposerver.max.combined.directory.manifests.size: '3M'
```
The 300x ratio assumes a maliciously-crafted manifest file. If you only want to protect against accidental excessive
memory use, it is probably safe to use a smaller ratio.
Keep in mind that if a malicious user can create additional Applications, they can increase the total memory usage.
Grant [App creation privileges](rbac.md) carefully.

View File

@@ -13,28 +13,30 @@ argocd-repo-server [flags]
### Options
```
--default-cache-expiration duration Cache expiration default (default 24h0m0s)
--disable-tls Disable TLS on the gRPC endpoint
-h, --help help for argocd-repo-server
--logformat string Set the logging format. One of: text|json (default "text")
--loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
--metrics-port int Start metrics server on given port (default 8084)
--otlp-address string OpenTelemetry collector address to send traces to
--parallelismlimit int Limit on number of concurrent manifests generate requests. Any value less the 1 means no limit.
--port int Listen on given port for incoming connections (default 8081)
--redis string Redis server hostname and port (e.g. argocd-redis:6379).
--redis-ca-certificate string Path to Redis server CA certificate (e.g. /etc/certs/redis/ca.crt). If not specified, system trusted CAs will be used for server certificate validation.
--redis-client-certificate string Path to Redis client certificate (e.g. /etc/certs/redis/client.crt).
--redis-client-key string Path to Redis client key (e.g. /etc/certs/redis/client.crt).
--redis-insecure-skip-tls-verify Skip Redis server certificate validation.
--redis-use-tls Use TLS when connecting to Redis.
--redisdb int Redis database.
--repo-cache-expiration duration Cache expiration for repo state, incl. app lists, app details, manifest generation, revision meta-data (default 24h0m0s)
--revision-cache-expiration duration Cache expiration for cached revision (default 3m0s)
--sentinel stringArray Redis sentinel hostname and port (e.g. argocd-redis-ha-announce-0:6379).
--sentinelmaster string Redis sentinel master group name. (default "master")
--tlsciphers string The list of acceptable ciphers to be used when establishing TLS connections. Use 'list' to list available ciphers. (default "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:TLS_RSA_WITH_AES_256_GCM_SHA384")
--tlsmaxversion string The maximum SSL/TLS version that is acceptable (one of: 1.0|1.1|1.2|1.3) (default "1.3")
--tlsminversion string The minimum SSL/TLS version that is acceptable (one of: 1.0|1.1|1.2|1.3) (default "1.2")
--default-cache-expiration duration Cache expiration default (default 24h0m0s)
--disable-tls Disable TLS on the gRPC endpoint
-h, --help help for argocd-repo-server
--logformat string Set the logging format. One of: text|json (default "text")
--loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
--max-combined-directory-manifests-size string Max combined size of manifest files in a directory-type Application (default "10M")
--metrics-port int Start metrics server on given port (default 8084)
--otlp-address string OpenTelemetry collector address to send traces to
--parallelismlimit int Limit on number of concurrent manifests generate requests. Any value less the 1 means no limit.
--plugin-tar-exclude stringArray Globs to filter when sending tarballs to plugins.
--port int Listen on given port for incoming connections (default 8081)
--redis string Redis server hostname and port (e.g. argocd-redis:6379).
--redis-ca-certificate string Path to Redis server CA certificate (e.g. /etc/certs/redis/ca.crt). If not specified, system trusted CAs will be used for server certificate validation.
--redis-client-certificate string Path to Redis client certificate (e.g. /etc/certs/redis/client.crt).
--redis-client-key string Path to Redis client key (e.g. /etc/certs/redis/client.crt).
--redis-insecure-skip-tls-verify Skip Redis server certificate validation.
--redis-use-tls Use TLS when connecting to Redis.
--redisdb int Redis database.
--repo-cache-expiration duration Cache expiration for repo state, incl. app lists, app details, manifest generation, revision meta-data (default 24h0m0s)
--revision-cache-expiration duration Cache expiration for cached revision (default 3m0s)
--sentinel stringArray Redis sentinel hostname and port (e.g. argocd-redis-ha-announce-0:6379).
--sentinelmaster string Redis sentinel master group name. (default "master")
--tlsciphers string The list of acceptable ciphers to be used when establishing TLS connections. Use 'list' to list available ciphers. (default "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:TLS_RSA_WITH_AES_256_GCM_SHA384")
--tlsmaxversion string The maximum SSL/TLS version that is acceptable (one of: 1.0|1.1|1.2|1.3) (default "1.3")
--tlsminversion string The minimum SSL/TLS version that is acceptable (one of: 1.0|1.1|1.2|1.3) (default "1.2")
```

View File

@@ -197,6 +197,7 @@ NOTES:
* There is no need to set `redirectURI` in the `connectors.config` as shown in the dex documentation.
Argo CD will automatically use the correct `redirectURI` for any OAuth2 connectors, to match the
correct external callback URL (e.g. `https://argocd.example.com/api/dex/callback`)
* When using a custom secret (e.g., `some_K8S_secret` above,) it *must* have the label `app.kubernetes.io/part-of: argocd`.
## OIDC Configuration with DEX
@@ -495,3 +496,20 @@ data:
clientSecret: $another-secret:oidc.auth0.clientSecret # Mind the ':'
...
```
### Skipping certificate verification on OIDC provider connections
By default, all connections made by the API server to OIDC providers (either external providers or the bundled Dex
instance) must pass certificate validation. These connections occur when getting the OIDC provider's well-known
configuration, when getting the OIDC provider's keys, and when exchanging an authorization code or verifying an ID
token as part of an OIDC login flow.
Disabling certificate verification might make sense if:
* You are using the bundled Dex instance **and** your Argo CD instance has TLS configured with a self-signed certificate
**and** you understand and accept the risks of skipping OIDC provider cert verification.
* You are using an external OIDC provider **and** that provider uses an invalid certificate **and** you cannot solve
the problem by setting `oidcConfig.rootCA` **and** you understand and accept the risks of skipping OIDC provider cert
verification.
If either of those two applies, then you can disable OIDC provider certificate verification by setting
`oidc.tls.insecure.skip.verify` to `"true"` in the `argocd-cm` ConfigMap.

View File

@@ -0,0 +1,45 @@
# Web-based Terminal
![Argo CD Terminal](../assets/terminal.png)
Since v2.4, Argo CD has a web-based terminal that allows you to get a shell inside a running pod just like you would with
`kubectl exec`. It's basically SSH from your browser, full ANSI color support and all! However, for security this feature
is disabled by default.
This is a powerful privilege. It allows the user to run arbitrary code on any Pod managed by an Application for which
they have the `exec/create` privilege. If the Pod mounts a ServiceAccount token (which is the default behavior of
Kubernetes), then the user effectively has the same privileges as that ServiceAccount.
## Enabling the terminal
1. Set the `exec.enabled` key to `true` on the `argocd-cm` ConfigMap.
2. Patch the `argocd-server` Role (if using namespaced Argo) or ClusterRole (if using clustered Argo) to allow `argocd-server`
to exec into pods
```yaml
- apiGroups:
- ""
resources:
- pods/exec
verbs:
- create
```
3. Add RBAC rules to allow your users to `create` the `exec` resource, i.e.
```
p, role:myrole, exec, create, */*, allow
```
See [RBAC Configuration](rbac.md#exec-resource) for more info.
## Changing allowed shells
By default, Argo CD attempts to execute shells in this order:
1. bash
2. sh
3. powershell
4. cmd
If none of the shells are found, the terminal session will fail. To add to or change the allowed shells, change the
`exec.shells` key in the `argocd-cm` ConfigMap, separating them with commas.

View File

@@ -234,3 +234,18 @@ If you don't need to set any environment variables, you can set an empty plugin
Each CMP command will also independently timeout on the `ARGOCD_EXEC_TIMEOUT` set for the CMP sidecar. The default
is 90s. So if you increase the repo server timeout greater than 90s, be sure to set `ARGOCD_EXEC_TIMEOUT` on the
sidecar.
## Plugin tar stream exclusions
In order to increase the speed of manifest generation, certain files and folders can be excluded from being sent to your
plugin. We recommend excluding your `.git` folder if it isn't necessary. Use Go's
[filepatch.Match](https://pkg.go.dev/path/filepath#Match) syntax.
You can set it one of three ways:
1. The `--plugin-tar-exclude` argument on the repo server.
2. The `reposerver.plugin.tar.exclusions` key if you are using `argocd-cmd-params-cm`
3. Directly setting `ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS` environment variable on the repo server.
For option 1, the flag can be repeated multiple times. For option 2 and 3, you can specify multiple globs by separating
them with semicolons.

View File

@@ -0,0 +1,9 @@
# Environment Variables
The following environment variables can be used with `argocd` CLI:
| Environment Variable | Description |
| --- | --- |
| `ARGOCD_SERVER` | the address of the ArgoCD server without `https://` prefix <br> (instead of specifying `--server` for every command) <br> eg. `ARGOCD_SERVER=argocd.mycompany.com` if served through an ingress with DNS |
| `ARGOCD_AUTH_TOKEN` | the ArgoCD `apiKey` for your ArgoCD user to be able to authenticate |
| `ARGOCD_OPTS` | command-line options to pass to `argocd` CLI <br> eg. `ARGOCD_OPTS="--grpc-web"` |

View File

@@ -85,3 +85,22 @@ argocd app set <appName> --kustomize-version v3.5.4
## Build Environment
Kustomize does not support parameters and therefore cannot support the standard [build environment](build-environment.md).
## Kustomizing Helm charts
It's possible to [render Helm charts with Kustomize](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/chart.md).
Doing so requires that you pass the `--enable-helm` flag to the `kustomize build` command.
This flag is not part of the Kustomize options within Argo CD.
If you would like to render Helm charts through Kustomize in an Argo CD application, you have two options:
You can either create a [custom plugin](https://argo-cd.readthedocs.io/en/stable/user-guide/config-management-plugins/), or modify the `argocd-cm` ConfigMap to include the `--enable-helm` flag globally for all Kustomize applications:
```
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
namespace: argocd
data:
kustomize.buildOptions: --enable-helm
```

View File

@@ -2,7 +2,7 @@
ArgoCD allows users to customize some aspects of how it syncs the desired state in the target cluster. Some Sync Options can defined as annotations in a specific resource. Most of the Sync Options are configured in the Application resource `spec.syncPolicy.syncOptions` attribute.
Bellow you can find details about each available Sync Option:
Below you can find details about each available Sync Option:
## No Prune Resources

14
go.mod
View File

@@ -17,7 +17,7 @@ require (
github.com/bradleyfalzon/ghinstallation/v2 v2.0.4
github.com/casbin/casbin/v2 v2.39.1
github.com/chai2010/gettext-go v0.0.0-20170215093142-bf70f2a70fb1 // indirect
github.com/coreos/go-oidc v2.1.0+incompatible
github.com/coreos/go-oidc v2.2.1+incompatible
github.com/dustin/go-humanize v1.0.0
github.com/evanphx/json-patch v4.12.0+incompatible
github.com/fsnotify/fsnotify v1.5.1
@@ -60,7 +60,7 @@ require (
github.com/olekukonko/tablewriter v0.0.5
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pkg/errors v0.9.1
github.com/pquerna/cachecontrol v0.0.0-20180306154005-525d0eb5f91d // indirect
github.com/pquerna/cachecontrol v0.1.0 // indirect
github.com/prometheus/client_golang v1.11.0
github.com/r3labs/diff v1.1.0
github.com/robfig/cron v1.2.0
@@ -76,10 +76,10 @@ require (
github.com/xanzy/go-gitlab v0.60.0
github.com/yuin/gopher-lua v0.0.0-20200816102855-ee81675732da
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e
golang.org/x/net v0.0.0-20211209124913-491a49abca63
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8
golang.org/x/net v0.0.0-20220621193019-9d032be2e588
golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa
google.golang.org/grpc v1.45.0
google.golang.org/protobuf v1.28.0
@@ -202,7 +202,7 @@ require (
go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 // indirect
golang.org/x/exp v0.0.0-20210901193431-a062eea981d2 // indirect
golang.org/x/mod v0.5.1-0.20210830214625-1b1db11ec8f4 // indirect
golang.org/x/sys v0.0.0-20211205182925-97ca703d548d // indirect
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
golang.org/x/tools v0.1.6-0.20210820212750-d4cc65f0b2ff // indirect
@@ -214,7 +214,7 @@ require (
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/square/go-jose.v2 v2.2.2 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
k8s.io/apiserver v0.23.1

24
go.sum
View File

@@ -254,8 +254,9 @@ github.com/coredns/corefile-migration v1.0.14/go.mod h1:XnhgULOEouimnzgn0t4WPuFD
github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-oidc v2.1.0+incompatible h1:sdJrfw8akMnCuUlaZU3tE/uYXFgfqom8DBE9so9EBsM=
github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc=
github.com/coreos/go-oidc v2.2.1+incompatible h1:mh48q/BqXqgjVHpy2ZY7WnWAbenxRjsz9N1i1YxjHAk=
github.com/coreos/go-oidc v2.2.1+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
@@ -944,8 +945,8 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/posener/complete v1.2.3/go.mod h1:WZIdtGGp+qx0sLrYKtIRAruyNpv6hFCicSgv7Sy7s/s=
github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/pquerna/cachecontrol v0.0.0-20180306154005-525d0eb5f91d h1:7gXyC293Lsm2YWgQ+0uaAFFFDO82ruiQSwc3ua+Vtlc=
github.com/pquerna/cachecontrol v0.0.0-20180306154005-525d0eb5f91d/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/pquerna/cachecontrol v0.1.0 h1:yJMy84ti9h/+OEWa752kBTKv4XC30OtVVHYv/8cTqKc=
github.com/pquerna/cachecontrol v0.1.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
@@ -1343,8 +1344,10 @@ golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211209124913-491a49abca63 h1:iocB37TsdFuN6IBRZ+ry36wrkoV51/tl5vOWqkcPGvY=
golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220621193019-9d032be2e588 h1:9ubFuySsnAJYGyJrZ3koiEv8FyqofCBdz3G9Mbf2YFc=
golang.org/x/net v0.0.0-20220621193019-9d032be2e588/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/oauth2 v0.0.0-20180227000427-d7d64896b5ff/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1364,8 +1367,9 @@ golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ
golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20211005180243-6b3c2da341f1/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 h1:RerP+noqYHUQ8CMRcPlC2nvTa4dcBIjegkuWdcUDuqg=
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb h1:8tDJ3aechhddbdPAxpycgXHJRMLpk/Ab+aa4OgdN5/g=
golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb/go.mod h1:jaDAt6Dkxork7LmZnYtzbRWj0W47D86a3TGe0YHBvmE=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -1485,11 +1489,14 @@ golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211205182925-97ca703d548d h1:FjkYO/PPp4Wi0EAUOVLxePm7qVW4r4ctbWpURyuOD0E=
golang.org/x/sys v0.0.0-20211205182925-97ca703d548d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a h1:dGzPydgVsqGcTRVwiLJ1jVbufYwmzD3LfVPLKsKg+0k=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1789,8 +1796,9 @@ gopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.66.2/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce/go.mod h1:yeKp02qBN3iKW1OzL3MGk2IdtZzaj7SFntXj72NppTA=
gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
gopkg.in/square/go-jose.v2 v2.2.2 h1:orlkJ3myw8CN1nVQHBFfloD+L3egixIa4FvUP6RosSA=
gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
gopkg.in/square/go-jose.v2 v2.6.0 h1:NGk74WTnPKBNUhNzQX7PYcTLUjoq7mzKk2OKbvwk2iI=
gopkg.in/square/go-jose.v2 v2.6.0/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/warnings.v0 v0.1.1/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=

View File

@@ -5,7 +5,7 @@ kind: Kustomization
images:
- name: quay.io/argoproj/argocd
newName: quay.io/argoproj/argocd
newTag: v2.4.0
newTag: v2.4.6
resources:
- ./application-controller
- ./dex

View File

@@ -101,6 +101,18 @@ spec:
name: argocd-cmd-params-cm
key: otlp.address
optional: true
- name: ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE
valueFrom:
configMapKeyRef:
name: argocd-cmd-params-cm
key: reposerver.max.combined.directory.manifests.size
optional: true
- name: ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS
valueFrom:
configMapKeyRef:
name: argocd-cmd-params-cm
key: reposerver.plugin.tar.exclusions
optional: true
- name: HELM_CACHE_HOME
value: /helm-working-dir
- name: HELM_CONFIG_HOME

View File

@@ -9385,7 +9385,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: argocd-applicationset-controller
ports:
@@ -9597,13 +9597,25 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE
valueFrom:
configMapKeyRef:
key: reposerver.max.combined.directory.manifests.size
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS
valueFrom:
configMapKeyRef:
key: reposerver.plugin.tar.exclusions
name: argocd-cmd-params-cm
optional: true
- name: HELM_CACHE_HOME
value: /helm-working-dir
- name: HELM_CONFIG_HOME
value: /helm-working-dir
- name: HELM_DATA_HOME
value: /helm-working-dir
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
@@ -9652,7 +9664,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /var/run/argocd/argocd-cmp-server
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
name: copyutil
securityContext:
allowPrivilegeEscalation: false
@@ -9839,7 +9851,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:

View File

@@ -12,4 +12,4 @@ resources:
images:
- name: quay.io/argoproj/argocd
newName: quay.io/argoproj/argocd
newTag: v2.4.0
newTag: v2.4.6

View File

@@ -11,7 +11,7 @@ patchesStrategicMerge:
images:
- name: quay.io/argoproj/argocd
newName: quay.io/argoproj/argocd
newTag: v2.4.0
newTag: v2.4.6
resources:
- ../../base/application-controller
- ../../base/applicationset-controller

View File

@@ -10320,7 +10320,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: argocd-applicationset-controller
ports:
@@ -10417,7 +10417,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /shared/argocd-dex
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: copyutil
securityContext:
@@ -10457,7 +10457,7 @@ spec:
containers:
- command:
- argocd-notifications
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
tcpSocket:
@@ -10696,13 +10696,25 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE
valueFrom:
configMapKeyRef:
key: reposerver.max.combined.directory.manifests.size
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS
valueFrom:
configMapKeyRef:
key: reposerver.plugin.tar.exclusions
name: argocd-cmd-params-cm
optional: true
- name: HELM_CACHE_HOME
value: /helm-working-dir
- name: HELM_CONFIG_HOME
value: /helm-working-dir
- name: HELM_DATA_HOME
value: /helm-working-dir
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
@@ -10751,7 +10763,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /var/run/argocd/argocd-cmp-server
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
name: copyutil
securityContext:
allowPrivilegeEscalation: false
@@ -10998,7 +11010,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:
@@ -11206,7 +11218,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:

View File

@@ -1244,7 +1244,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: argocd-applicationset-controller
ports:
@@ -1341,7 +1341,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /shared/argocd-dex
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: copyutil
securityContext:
@@ -1381,7 +1381,7 @@ spec:
containers:
- command:
- argocd-notifications
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
tcpSocket:
@@ -1620,13 +1620,25 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE
valueFrom:
configMapKeyRef:
key: reposerver.max.combined.directory.manifests.size
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS
valueFrom:
configMapKeyRef:
key: reposerver.plugin.tar.exclusions
name: argocd-cmd-params-cm
optional: true
- name: HELM_CACHE_HOME
value: /helm-working-dir
- name: HELM_CONFIG_HOME
value: /helm-working-dir
- name: HELM_DATA_HOME
value: /helm-working-dir
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
@@ -1675,7 +1687,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /var/run/argocd/argocd-cmp-server
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
name: copyutil
securityContext:
allowPrivilegeEscalation: false
@@ -1922,7 +1934,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:
@@ -2130,7 +2142,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:

View File

@@ -9692,7 +9692,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: argocd-applicationset-controller
ports:
@@ -9789,7 +9789,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /shared/argocd-dex
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: copyutil
securityContext:
@@ -9829,7 +9829,7 @@ spec:
containers:
- command:
- argocd-notifications
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
tcpSocket:
@@ -10036,13 +10036,25 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE
valueFrom:
configMapKeyRef:
key: reposerver.max.combined.directory.manifests.size
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS
valueFrom:
configMapKeyRef:
key: reposerver.plugin.tar.exclusions
name: argocd-cmd-params-cm
optional: true
- name: HELM_CACHE_HOME
value: /helm-working-dir
- name: HELM_CONFIG_HOME
value: /helm-working-dir
- name: HELM_DATA_HOME
value: /helm-working-dir
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
@@ -10091,7 +10103,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /var/run/argocd/argocd-cmp-server
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
name: copyutil
securityContext:
allowPrivilegeEscalation: false
@@ -10334,7 +10346,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:
@@ -10536,7 +10548,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:

View File

@@ -616,7 +616,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: argocd-applicationset-controller
ports:
@@ -713,7 +713,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /shared/argocd-dex
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
name: copyutil
securityContext:
@@ -753,7 +753,7 @@ spec:
containers:
- command:
- argocd-notifications
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
tcpSocket:
@@ -960,13 +960,25 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_MAX_COMBINED_DIRECTORY_MANIFESTS_SIZE
valueFrom:
configMapKeyRef:
key: reposerver.max.combined.directory.manifests.size
name: argocd-cmd-params-cm
optional: true
- name: ARGOCD_REPO_SERVER_PLUGIN_TAR_EXCLUSIONS
valueFrom:
configMapKeyRef:
key: reposerver.plugin.tar.exclusions
name: argocd-cmd-params-cm
optional: true
- name: HELM_CACHE_HOME
value: /helm-working-dir
- name: HELM_CONFIG_HOME
value: /helm-working-dir
- name: HELM_DATA_HOME
value: /helm-working-dir
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
@@ -1015,7 +1027,7 @@ spec:
- -n
- /usr/local/bin/argocd
- /var/run/argocd/argocd-cmp-server
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
name: copyutil
securityContext:
allowPrivilegeEscalation: false
@@ -1258,7 +1270,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:
@@ -1460,7 +1472,7 @@ spec:
key: otlp.address
name: argocd-cmd-params-cm
optional: true
image: quay.io/argoproj/argocd:v2.4.0
image: quay.io/argoproj/argocd:v2.4.6
imagePullPolicy: Always
livenessProbe:
httpGet:

View File

@@ -44,6 +44,7 @@ nav:
- operator-manual/custom_tools.md
- operator-manual/custom-styles.md
- operator-manual/metrics.md
- operator-manual/web_based_terminal.md
- Notification:
- Overview: operator-manual/notifications/index.md
- operator-manual/notifications/triggers.md
@@ -129,6 +130,7 @@ nav:
- user-guide/compare-options.md
- user-guide/sync-options.md
- user-guide/parameters.md
- user-guide/environment-variables.md
- user-guide/build-environment.md
- user-guide/tracking_strategies.md
- user-guide/resource_tracking.md

View File

@@ -100,7 +100,11 @@ func (c *client) executeRequest(fullMethodName string, msg []byte, md metadata.M
}
func (c *client) startGRPCProxy() (*grpc.Server, net.Listener, error) {
serverAddr := fmt.Sprintf("%s/argocd-%s.sock", os.TempDir(), rand.RandString(16))
randSuffix, err := rand.String(16)
if err != nil {
return nil, nil, fmt.Errorf("failed to generate random socket filename: %w", err)
}
serverAddr := fmt.Sprintf("%s/argocd-%s.sock", os.TempDir(), randSuffix)
ln, err := net.Listen("unix", serverAddr)
if err != nil {

View File

@@ -1542,6 +1542,19 @@ func isValidAction(action string) bool {
return false
}
// TODO: same as validActions, refacotor to use rbacpolicy.ResourceApplications etc.
var validResources = map[string]bool{
"applications": true,
"repositories": true,
"clusters": true,
"exec": true,
"logs": true,
}
func isValidResource(resource string) bool {
return validResources[resource]
}
func validatePolicy(proj string, role string, policy string) error {
policyComponents := strings.Split(policy, ",")
if len(policyComponents) != 6 || strings.Trim(policyComponents[0], " ") != "p" {
@@ -1555,7 +1568,7 @@ func validatePolicy(proj string, role string, policy string) error {
}
// resource
resource := strings.Trim(policyComponents[2], " ")
if resource != "applications" && resource != "repositories" && resource != "clusters" {
if !isValidResource(resource) {
return status.Errorf(codes.InvalidArgument, "invalid policy rule '%s': project resource must be: 'applications', 'repositories' or 'clusters', not '%s'", policy, resource)
}
// action

View File

@@ -2546,10 +2546,19 @@ func Test_validatePolicy_projIsNotRegex(t *testing.T) {
}
func Test_validatePolicy_ValidResource(t *testing.T) {
err := validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, repositories, *, some-project/*, allow")
err := validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, applications, *, some-project/*, allow")
assert.NoError(t, err)
err = validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, repositories, *, some-project/*, allow")
assert.NoError(t, err)
err = validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, clusters, *, some-project/*, allow")
assert.NoError(t, err)
err = validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, exec, *, some-project/*, allow")
assert.NoError(t, err)
err = validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, logs, *, some-project/*, allow")
assert.NoError(t, err)
err = validatePolicy("some-project", "org-admin", "p, proj:some-project:org-admin, unknown, *, some-project/*, allow")
assert.Error(t, err)
}
func TestEnvsubst(t *testing.T) {

View File

@@ -18,6 +18,10 @@ import (
"strings"
"time"
kubeyaml "k8s.io/apimachinery/pkg/util/yaml"
"k8s.io/apimachinery/pkg/api/resource"
"github.com/argoproj/argo-cd/v2/util/io/files"
"github.com/Masterminds/semver/v3"
@@ -71,6 +75,8 @@ const (
ociPrefix = "oci://"
)
var ErrExceededMaxCombinedManifestFileSize = errors.New("exceeded max combined manifest file size")
// Service implements ManifestService interface
type Service struct {
gitCredsStore git.CredsStore
@@ -96,6 +102,8 @@ type RepoServerInitConstants struct {
PauseGenerationOnFailureForMinutes int
PauseGenerationOnFailureForRequests int
SubmoduleEnabled bool
MaxCombinedDirectoryManifestsSize resource.Quantity
CMPTarExcludedGlobs []string
}
// NewService returns a new instance of the Manifest service
@@ -206,7 +214,7 @@ func (s *Service) ListApps(ctx context.Context, q *apiclient.ListAppsRequest) (*
}
defer io.Close(closer)
apps, err := discovery.Discover(ctx, gitClient.Root(), q.EnabledSourceTypes)
apps, err := discovery.Discover(ctx, gitClient.Root(), q.EnabledSourceTypes, s.initConstants.CMPTarExcludedGlobs)
if err != nil {
return nil, err
}
@@ -450,10 +458,15 @@ func (s *Service) runManifestGenAsync(ctx context.Context, repoRoot, commitSHA,
close(ch.errCh)
close(ch.responseCh)
}()
// GenerateManifests mutates the source (applies overrides). Those overrides shouldn't be reflected in the cache
// key. Overrides will break the cache anyway, because changes to overrides will change the revision.
appSourceCopy := q.ApplicationSource.DeepCopy()
var manifestGenResult *apiclient.ManifestResponse
opContext, err := opContextSrc()
if err == nil {
manifestGenResult, err = GenerateManifests(ctx, opContext.appPath, repoRoot, commitSHA, q, false, s.gitCredsStore, WithCMPTarDoneChannel(ch.tarDoneCh))
manifestGenResult, err = GenerateManifests(ctx, opContext.appPath, repoRoot, commitSHA, q, false, s.gitCredsStore, s.initConstants.MaxCombinedDirectoryManifestsSize, WithCMPTarDoneChannel(ch.tarDoneCh), WithCMPTarExcludedGlobs(s.initConstants.CMPTarExcludedGlobs))
}
if err != nil {
// If manifest generation error caching is enabled
@@ -462,9 +475,9 @@ func (s *Service) runManifestGenAsync(ctx context.Context, repoRoot, commitSHA,
// Retrieve a new copy (if available) of the cached response: this ensures we are updating the latest copy of the cache,
// rather than a copy of the cache that occurred before (a potentially lengthy) manifest generation.
innerRes := &cache.CachedManifestResponse{}
cacheErr := s.cache.GetManifests(cacheKey, q.ApplicationSource, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, innerRes)
cacheErr := s.cache.GetManifests(cacheKey, appSourceCopy, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, innerRes)
if cacheErr != nil && cacheErr != reposervercache.ErrCacheMiss {
log.Warnf("manifest cache set error %s: %v", q.ApplicationSource.String(), cacheErr)
log.Warnf("manifest cache set error %s: %v", appSourceCopy.String(), cacheErr)
ch.errCh <- cacheErr
return
}
@@ -478,9 +491,9 @@ func (s *Service) runManifestGenAsync(ctx context.Context, repoRoot, commitSHA,
// Update the cache to include failure information
innerRes.NumberOfConsecutiveFailures++
innerRes.MostRecentError = err.Error()
cacheErr = s.cache.SetManifests(cacheKey, q.ApplicationSource, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, innerRes)
cacheErr = s.cache.SetManifests(cacheKey, appSourceCopy, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, innerRes)
if cacheErr != nil {
log.Warnf("manifest cache set error %s: %v", q.ApplicationSource.String(), cacheErr)
log.Warnf("manifest cache set error %s: %v", appSourceCopy.String(), cacheErr)
ch.errCh <- cacheErr
return
}
@@ -499,9 +512,9 @@ func (s *Service) runManifestGenAsync(ctx context.Context, repoRoot, commitSHA,
}
manifestGenResult.Revision = commitSHA
manifestGenResult.VerifyResult = opContext.verificationResult
err = s.cache.SetManifests(cacheKey, q.ApplicationSource, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, &manifestGenCacheEntry)
err = s.cache.SetManifests(cacheKey, appSourceCopy, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, &manifestGenCacheEntry)
if err != nil {
log.Warnf("manifest cache set error %s/%s: %v", q.ApplicationSource.String(), cacheKey, err)
log.Warnf("manifest cache set error %s/%s: %v", appSourceCopy.String(), cacheKey, err)
}
ch.responseCh <- manifestGenCacheEntry.ManifestResponse
}
@@ -846,7 +859,8 @@ func getRepoCredential(repoCredentials []*v1alpha1.RepoCreds, repoURL string) *v
type GenerateManifestOpt func(*generateManifestOpt)
type generateManifestOpt struct {
cmpTarDoneCh chan<- bool
cmpTarDoneCh chan<- bool
cmpTarExcludedGlobs []string
}
func newGenerateManifestOpt(opts ...GenerateManifestOpt) *generateManifestOpt {
@@ -866,14 +880,23 @@ func WithCMPTarDoneChannel(ch chan<- bool) GenerateManifestOpt {
}
}
// GenerateManifests generates manifests from a path
func GenerateManifests(ctx context.Context, appPath, repoRoot, revision string, q *apiclient.ManifestRequest, isLocal bool, gitCredsStore git.CredsStore, opts ...GenerateManifestOpt) (*apiclient.ManifestResponse, error) {
// WithCMPTarExcludedGlobs defines globs for files to filter out when streaming the tarball
// to a CMP sidecar.
func WithCMPTarExcludedGlobs(excludedGlobs []string) GenerateManifestOpt {
return func(o *generateManifestOpt) {
o.cmpTarExcludedGlobs = excludedGlobs
}
}
// GenerateManifests generates manifests from a path. Overrides are applied as a side effect on the given ApplicationSource.
func GenerateManifests(ctx context.Context, appPath, repoRoot, revision string, q *apiclient.ManifestRequest, isLocal bool, gitCredsStore git.CredsStore, maxCombinedManifestQuantity resource.Quantity, opts ...GenerateManifestOpt) (*apiclient.ManifestResponse, error) {
opt := newGenerateManifestOpt(opts...)
var targetObjs []*unstructured.Unstructured
var dest *v1alpha1.ApplicationDestination
resourceTracking := argo.NewResourceTracking()
appSourceType, err := GetAppSourceType(ctx, q.ApplicationSource, appPath, q.AppName, q.EnabledSourceTypes)
appSourceType, err := GetAppSourceType(ctx, q.ApplicationSource, appPath, q.AppName, q.EnabledSourceTypes, opt.cmpTarExcludedGlobs)
if err != nil {
return nil, err
}
@@ -897,7 +920,7 @@ func GenerateManifests(ctx context.Context, appPath, repoRoot, revision string,
if q.ApplicationSource.Plugin != nil && q.ApplicationSource.Plugin.Name != "" {
targetObjs, err = runConfigManagementPlugin(appPath, repoRoot, env, q, q.Repo.GetGitCreds(gitCredsStore))
} else {
targetObjs, err = runConfigManagementPluginSidecars(ctx, appPath, repoRoot, env, q, q.Repo.GetGitCreds(gitCredsStore), opt.cmpTarDoneCh)
targetObjs, err = runConfigManagementPluginSidecars(ctx, appPath, repoRoot, env, q, q.Repo.GetGitCreds(gitCredsStore), opt.cmpTarDoneCh, opt.cmpTarExcludedGlobs)
if err != nil {
err = fmt.Errorf("plugin sidecar failed. %s", err.Error())
}
@@ -908,7 +931,7 @@ func GenerateManifests(ctx context.Context, appPath, repoRoot, revision string,
directory = &v1alpha1.ApplicationSourceDirectory{}
}
logCtx := log.WithField("application", q.AppName)
targetObjs, err = findManifests(logCtx, appPath, repoRoot, env, *directory, q.EnabledSourceTypes)
targetObjs, err = findManifests(logCtx, appPath, repoRoot, env, *directory, q.EnabledSourceTypes, maxCombinedManifestQuantity)
}
if err != nil {
return nil, err
@@ -985,7 +1008,7 @@ func mergeSourceParameters(source *v1alpha1.ApplicationSource, path, appName str
overrides = append(overrides, filepath.Join(path, fmt.Sprintf(appSourceFile, appName)))
}
var merged v1alpha1.ApplicationSource = *source.DeepCopy()
var merged = *source.DeepCopy()
for _, filename := range overrides {
info, err := os.Stat(filename)
@@ -1031,7 +1054,7 @@ func mergeSourceParameters(source *v1alpha1.ApplicationSource, path, appName str
}
// GetAppSourceType returns explicit application source type or examines a directory and determines its application source type
func GetAppSourceType(ctx context.Context, source *v1alpha1.ApplicationSource, path, appName string, enableGenerateManifests map[string]bool) (v1alpha1.ApplicationSourceType, error) {
func GetAppSourceType(ctx context.Context, source *v1alpha1.ApplicationSource, path, appName string, enableGenerateManifests map[string]bool, tarExcludedGlobs []string) (v1alpha1.ApplicationSourceType, error) {
err := mergeSourceParameters(source, path, appName)
if err != nil {
return "", fmt.Errorf("error while parsing source parameters: %v", err)
@@ -1048,7 +1071,7 @@ func GetAppSourceType(ctx context.Context, source *v1alpha1.ApplicationSource, p
}
return *appSourceType, nil
}
appType, err := discovery.AppType(ctx, path, enableGenerateManifests)
appType, err := discovery.AppType(ctx, path, enableGenerateManifests, tarExcludedGlobs)
if err != nil {
return "", err
}
@@ -1078,63 +1101,30 @@ func isNullList(obj *unstructured.Unstructured) bool {
var manifestFile = regexp.MustCompile(`^.*\.(yaml|yml|json|jsonnet)$`)
// findManifests looks at all yaml files in a directory and unmarshals them into a list of unstructured objects
func findManifests(logCtx *log.Entry, appPath string, repoRoot string, env *v1alpha1.Env, directory v1alpha1.ApplicationSourceDirectory, enabledManifestGeneration map[string]bool) ([]*unstructured.Unstructured, error) {
func findManifests(logCtx *log.Entry, appPath string, repoRoot string, env *v1alpha1.Env, directory v1alpha1.ApplicationSourceDirectory, enabledManifestGeneration map[string]bool, maxCombinedManifestQuantity resource.Quantity) ([]*unstructured.Unstructured, error) {
// Validate the directory before loading any manifests to save memory.
potentiallyValidManifests, err := getPotentiallyValidManifests(logCtx, appPath, repoRoot, directory.Recurse, directory.Include, directory.Exclude, maxCombinedManifestQuantity)
if err != nil {
logCtx.Errorf("failed to get potentially valid manifests: %s", err)
return nil, fmt.Errorf("failed to get potentially valid manifests: %w", err)
}
var objs []*unstructured.Unstructured
err := filepath.Walk(appPath, func(path string, f os.FileInfo, err error) error {
if err != nil {
return err
}
relPath, err := filepath.Rel(appPath, path)
if err != nil {
return fmt.Errorf("failed to get relative path of symlink: %w", err)
}
if files.IsSymlink(f) {
realPath, err := filepath.EvalSymlinks(path)
if err != nil {
logCtx.Debugf("error checking symlink realpath: %s", err)
if os.IsNotExist(err) {
log.Warnf("ignoring out-of-bounds symlink at %q: %s", relPath, err)
return nil
} else {
return fmt.Errorf("failed to evaluate symlink at %q: %w", relPath, err)
}
}
if !files.Inbound(realPath, appPath) {
logCtx.Warnf("illegal filepath in symlink: %s", realPath)
return fmt.Errorf("illegal filepath in symlink at %q", relPath)
}
}
if f.IsDir() {
if path != appPath && !directory.Recurse {
return filepath.SkipDir
} else {
return nil
}
}
for _, potentiallyValidManifest := range potentiallyValidManifests {
manifestPath := potentiallyValidManifest.path
manifestFileInfo := potentiallyValidManifest.fileInfo
if !manifestFile.MatchString(f.Name()) {
return nil
}
if directory.Exclude != "" && glob.Match(directory.Exclude, relPath) {
return nil
}
if directory.Include != "" && !glob.Match(directory.Include, relPath) {
return nil
}
if strings.HasSuffix(f.Name(), ".jsonnet") {
if strings.HasSuffix(manifestFileInfo.Name(), ".jsonnet") {
if !discovery.IsManifestGenerationEnabled(v1alpha1.ApplicationSourceTypeDirectory, enabledManifestGeneration) {
return nil
continue
}
vm, err := makeJsonnetVm(appPath, repoRoot, directory.Jsonnet, env)
if err != nil {
return err
return nil, err
}
jsonStr, err := vm.EvaluateFile(path)
jsonStr, err := vm.EvaluateFile(manifestPath)
if err != nil {
return status.Errorf(codes.FailedPrecondition, "Failed to evaluate jsonnet %q: %v", f.Name(), err)
return nil, status.Errorf(codes.FailedPrecondition, "Failed to evaluate jsonnet %q: %v", manifestFileInfo.Name(), err)
}
// attempt to unmarshal either array or single object
@@ -1146,49 +1136,207 @@ func findManifests(logCtx *log.Entry, appPath string, repoRoot string, env *v1al
var jsonObj unstructured.Unstructured
err = json.Unmarshal([]byte(jsonStr), &jsonObj)
if err != nil {
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal generated json %q: %v", f.Name(), err)
return nil, status.Errorf(codes.FailedPrecondition, "Failed to unmarshal generated json %q: %v", manifestFileInfo.Name(), err)
}
objs = append(objs, &jsonObj)
}
} else {
out, err := utfutil.ReadFile(path, utfutil.UTF8)
err := getObjsFromYAMLOrJson(logCtx, manifestPath, manifestFileInfo.Name(), &objs)
if err != nil {
return err
}
if strings.HasSuffix(f.Name(), ".json") {
var obj unstructured.Unstructured
err = json.Unmarshal(out, &obj)
if err != nil {
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal %q: %v", f.Name(), err)
}
objs = append(objs, &obj)
} else {
yamlObjs, err := kube.SplitYAML(out)
if err != nil {
if len(yamlObjs) > 0 {
// If we get here, we had a multiple objects in a single YAML file which had some
// valid k8s objects, but errors parsing others (within the same file). It's very
// likely the user messed up a portion of the YAML, so report on that.
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal %q: %v", f.Name(), err)
}
// Otherwise, let's see if it looks like a resource, if yes, we return error
if bytes.Contains(out, []byte("apiVersion:")) &&
bytes.Contains(out, []byte("kind:")) &&
bytes.Contains(out, []byte("metadata:")) {
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal %q: %v", f.Name(), err)
}
// Otherwise, it might be a unrelated YAML file which we will ignore
return nil
}
objs = append(objs, yamlObjs...)
return nil, err
}
}
}
return objs, nil
}
// getObjsFromYAMLOrJson unmarshals the given yaml or json file and appends it to the given list of objects.
func getObjsFromYAMLOrJson(logCtx *log.Entry, manifestPath string, filename string, objs *[]*unstructured.Unstructured) error {
reader, err := utfutil.OpenFile(manifestPath, utfutil.UTF8)
if err != nil {
return status.Errorf(codes.FailedPrecondition, "Failed to open %q", manifestPath)
}
defer func() {
err := reader.Close()
if err != nil {
logCtx.Errorf("failed to close %q - potential memory leak", manifestPath)
}
}()
if strings.HasSuffix(filename, ".json") {
var obj unstructured.Unstructured
decoder := json.NewDecoder(reader)
err = decoder.Decode(&obj)
if err != nil {
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal %q: %v", filename, err)
}
if decoder.More() {
return status.Errorf(codes.FailedPrecondition, "Found multiple objects in %q. Only single objects are allowed in JSON files.", filename)
}
*objs = append(*objs, &obj)
} else {
yamlObjs, err := splitYAMLOrJSON(reader)
if err != nil {
if len(yamlObjs) > 0 {
// If we get here, we had a multiple objects in a single YAML file which had some
// valid k8s objects, but errors parsing others (within the same file). It's very
// likely the user messed up a portion of the YAML, so report on that.
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal %q: %v", filename, err)
}
// Read the whole file to check whether it looks like a manifest.
out, err := utfutil.ReadFile(manifestPath, utfutil.UTF8)
// Otherwise, let's see if it looks like a resource, if yes, we return error
if bytes.Contains(out, []byte("apiVersion:")) &&
bytes.Contains(out, []byte("kind:")) &&
bytes.Contains(out, []byte("metadata:")) {
return status.Errorf(codes.FailedPrecondition, "Failed to unmarshal %q: %v", filename, err)
}
// Otherwise, it might be an unrelated YAML file which we will ignore
}
*objs = append(*objs, yamlObjs...)
}
return nil
}
// splitYAMLOrJSON reads a YAML or JSON file and gets each document as an unstructured object. If the unmarshaller
// encounters an error, objects read up until the error are returned.
func splitYAMLOrJSON(reader goio.Reader) ([]*unstructured.Unstructured, error) {
d := kubeyaml.NewYAMLOrJSONDecoder(reader, 4096)
var objs []*unstructured.Unstructured
for {
u := &unstructured.Unstructured{}
if err := d.Decode(&u); err != nil {
if err == goio.EOF {
break
}
return objs, fmt.Errorf("failed to unmarshal manifest: %v", err)
}
if u == nil {
continue
}
objs = append(objs, u)
}
return objs, nil
}
// getPotentiallyValidManifestFile checks whether the given path/FileInfo may be a valid manifest file. Returns a non-nil error if
// there was an error that should not be handled by ignoring the file. Returns non-nil realFileInfo if the file is a
// potential manifest. Returns a non-empty ignoreMessage if there's a message that should be logged about why the file
// was skipped. If realFileInfo is nil and the ignoreMessage is empty, there's no need to log the ignoreMessage; the
// file was skipped for a mundane reason.
//
// The file is still only a "potentially" valid manifest file because it could be invalid JSON or YAML, or it might not
// be a valid Kubernetes resource. This function tests everything possible without actually reading the file.
//
// repoPath must be absolute.
func getPotentiallyValidManifestFile(path string, f os.FileInfo, appPath, repoRoot, include, exclude string) (realFileInfo os.FileInfo, warning string, err error) {
relPath, err := filepath.Rel(appPath, path)
if err != nil {
return nil, "", fmt.Errorf("failed to get relative path of %q: %w", path, err)
}
if !manifestFile.MatchString(f.Name()) {
return nil, "", nil
}
// If the file is a symlink, these will be overridden with the destination file's info.
var relRealPath = relPath
realFileInfo = f
if files.IsSymlink(f) {
realPath, err := filepath.EvalSymlinks(path)
if err != nil {
if os.IsNotExist(err) {
return nil, fmt.Sprintf("destination of symlink %q is missing", relPath), nil
}
return nil, "", fmt.Errorf("failed to evaluate symlink at %q: %w", relPath, err)
}
if !files.Inbound(realPath, repoRoot) {
return nil, "", fmt.Errorf("illegal filepath in symlink at %q", relPath)
}
realFileInfo, err = os.Stat(realPath)
if err != nil {
if os.IsNotExist(err) {
// This should have been caught by filepath.EvalSymlinks, but check again since that function's docs
// don't promise to return this error.
return nil, fmt.Sprintf("destination of symlink %q is missing at %q", relPath, realPath), nil
}
return nil, "", fmt.Errorf("failed to get file info for symlink at %q to %q: %w", relPath, realPath, err)
}
relRealPath, err = filepath.Rel(repoRoot, realPath)
if err != nil {
return nil, "", fmt.Errorf("failed to get relative path of %q: %w", realPath, err)
}
}
// FileInfo.Size() behavior is platform-specific for non-regular files. Allow only regular files, so we guarantee
// accurate file sizes.
if !realFileInfo.Mode().IsRegular() {
return nil, fmt.Sprintf("ignoring symlink at %q to non-regular file %q", relPath, relRealPath), nil
}
if exclude != "" && glob.Match(exclude, relPath) {
return nil, "", nil
}
if include != "" && !glob.Match(include, relPath) {
return nil, "", nil
}
return realFileInfo, "", nil
}
type potentiallyValidManifest struct {
path string
fileInfo os.FileInfo
}
// getPotentiallyValidManifests ensures that 1) there are no errors while checking for potential manifest files in the given dir
// and 2) the combined file size of the potentially-valid manifest files does not exceed the limit.
func getPotentiallyValidManifests(logCtx *log.Entry, appPath string, repoRoot string, recurse bool, include string, exclude string, maxCombinedManifestQuantity resource.Quantity) ([]potentiallyValidManifest, error) {
maxCombinedManifestFileSize := maxCombinedManifestQuantity.Value()
var currentCombinedManifestFileSize = int64(0)
var potentiallyValidManifests []potentiallyValidManifest
err := filepath.Walk(appPath, func(path string, f os.FileInfo, err error) error {
if err != nil {
return err
}
if f.IsDir() {
if path != appPath && !recurse {
return filepath.SkipDir
}
return nil
}
realFileInfo, warning, err := getPotentiallyValidManifestFile(path, f, appPath, repoRoot, include, exclude)
if err != nil {
return fmt.Errorf("invalid manifest file %q: %w", path, err)
}
if realFileInfo == nil {
if warning != "" {
logCtx.Warnf("skipping manifest file %q: %s", path, warning)
}
return nil
}
// Don't count jsonnet file size against max. It's jsonnet's responsibility to manage memory usage.
if !strings.HasSuffix(f.Name(), ".jsonnet") {
// We use the realFileInfo size (which is guaranteed to be a regular file instead of a symlink or other
// non-regular file) because .Size() behavior is platform-specific for non-regular files.
currentCombinedManifestFileSize += realFileInfo.Size()
if maxCombinedManifestFileSize != 0 && currentCombinedManifestFileSize > maxCombinedManifestFileSize {
return ErrExceededMaxCombinedManifestFileSize
}
}
potentiallyValidManifests = append(potentiallyValidManifests, potentiallyValidManifest{path: path, fileInfo: f})
return nil
})
if err != nil {
// Not wrapping, because this error should be wrapped by the caller.
return nil, err
}
return objs, nil
return potentiallyValidManifests, nil
}
func makeJsonnetVm(appPath string, repoRoot string, sourceJsonnet v1alpha1.ApplicationSourceJsonnet, env *v1alpha1.Env) (*jsonnet.VM, error) {
@@ -1327,7 +1475,7 @@ func getPluginEnvs(envVars *v1alpha1.Env, q *apiclient.ManifestRequest, creds gi
return env, nil
}
func runConfigManagementPluginSidecars(ctx context.Context, appPath, repoPath string, envVars *v1alpha1.Env, q *apiclient.ManifestRequest, creds git.Creds, tarDoneCh chan<- bool) ([]*unstructured.Unstructured, error) {
func runConfigManagementPluginSidecars(ctx context.Context, appPath, repoPath string, envVars *v1alpha1.Env, q *apiclient.ManifestRequest, creds git.Creds, tarDoneCh chan<- bool, tarExcludedGlobs []string) ([]*unstructured.Unstructured, error) {
// compute variables.
env, err := getPluginEnvs(envVars, q, creds, true)
if err != nil {
@@ -1335,14 +1483,14 @@ func runConfigManagementPluginSidecars(ctx context.Context, appPath, repoPath st
}
// detect config management plugin server (sidecar)
conn, cmpClient, err := discovery.DetectConfigManagementPlugin(ctx, appPath, env)
conn, cmpClient, err := discovery.DetectConfigManagementPlugin(ctx, appPath, env, tarExcludedGlobs)
if err != nil {
return nil, err
}
defer io.Close(conn)
// generate manifests using commands provided in plugin config file in detected cmp-server sidecar
cmpManifests, err := generateManifestsCMP(ctx, appPath, repoPath, env, cmpClient, tarDoneCh)
cmpManifests, err := generateManifestsCMP(ctx, appPath, repoPath, env, cmpClient, tarDoneCh, tarExcludedGlobs)
if err != nil {
return nil, fmt.Errorf("error generating manifests in cmp: %s", err)
}
@@ -1360,7 +1508,7 @@ func runConfigManagementPluginSidecars(ctx context.Context, appPath, repoPath st
// generateManifestsCMP will send the appPath files to the cmp-server over a gRPC stream.
// The cmp-server will generate the manifests. Returns a response object with the generated
// manifests.
func generateManifestsCMP(ctx context.Context, appPath, repoPath string, env []string, cmpClient pluginclient.ConfigManagementPluginServiceClient, tarDoneCh chan<- bool) (*pluginclient.ManifestResponse, error) {
func generateManifestsCMP(ctx context.Context, appPath, repoPath string, env []string, cmpClient pluginclient.ConfigManagementPluginServiceClient, tarDoneCh chan<- bool, tarExcludedGlobs []string) (*pluginclient.ManifestResponse, error) {
generateManifestStream, err := cmpClient.GenerateManifest(ctx, grpc_retry.Disable())
if err != nil {
return nil, fmt.Errorf("error getting generateManifestStream: %s", err)
@@ -1368,7 +1516,7 @@ func generateManifestsCMP(ctx context.Context, appPath, repoPath string, env []s
opts := []cmp.SenderOption{
cmp.WithTarDoneChan(tarDoneCh),
}
err = cmp.SendRepoStream(generateManifestStream.Context(), appPath, repoPath, generateManifestStream, env, opts...)
err = cmp.SendRepoStream(generateManifestStream.Context(), appPath, repoPath, generateManifestStream, env, tarExcludedGlobs, opts...)
if err != nil {
return nil, fmt.Errorf("error sending file to cmp-server: %s", err)
}
@@ -1386,7 +1534,7 @@ func (s *Service) GetAppDetails(ctx context.Context, q *apiclient.RepoServerAppD
return err
}
appSourceType, err := GetAppSourceType(ctx, q.Source, opContext.appPath, q.AppName, q.EnabledSourceTypes)
appSourceType, err := GetAppSourceType(ctx, q.Source, opContext.appPath, q.AppName, q.EnabledSourceTypes, s.initConstants.CMPTarExcludedGlobs)
if err != nil {
return err
}
@@ -1461,8 +1609,12 @@ func populateHelmAppDetails(res *apiclient.RepoAppDetailsResponse, appPath strin
return err
}
if err := loadFileIntoIfExists(filepath.Join(appPath, "values.yaml"), &res.Helm.Values); err != nil {
return err
if resolvedValuesPath, _, err := pathutil.ResolveFilePath(appPath, repoRoot, "values.yaml", []string{}); err == nil {
if err := loadFileIntoIfExists(resolvedValuesPath, &res.Helm.Values); err != nil {
return err
}
} else {
log.Warnf("Values file %s is not allowed: %v", filepath.Join(appPath, "values.yaml"), err)
}
var resolvedSelectedValueFiles []pathutil.ResolvedFilePath
// drop not allowed values files
@@ -1470,10 +1622,10 @@ func populateHelmAppDetails(res *apiclient.RepoAppDetailsResponse, appPath strin
if resolvedFile, _, err := pathutil.ResolveFilePath(appPath, repoRoot, file, q.GetValuesFileSchemes()); err == nil {
resolvedSelectedValueFiles = append(resolvedSelectedValueFiles, resolvedFile)
} else {
log.Debugf("Values file %s is not allowed: %v", file, err)
log.Warnf("Values file %s is not allowed: %v", file, err)
}
}
params, err := h.GetParameters(resolvedSelectedValueFiles)
params, err := h.GetParameters(resolvedSelectedValueFiles, appPath, repoRoot)
if err != nil {
return err
}
@@ -1492,15 +1644,16 @@ func populateHelmAppDetails(res *apiclient.RepoAppDetailsResponse, appPath strin
return nil
}
func loadFileIntoIfExists(path string, destination *string) error {
info, err := os.Stat(path)
func loadFileIntoIfExists(path pathutil.ResolvedFilePath, destination *string) error {
stringPath := string(path)
info, err := os.Stat(stringPath)
if err == nil && !info.IsDir() {
if bytes, err := ioutil.ReadFile(path); err != nil {
*destination = string(bytes)
} else {
bytes, err := ioutil.ReadFile(stringPath)
if err != nil {
return err
}
*destination = string(bytes)
}
return nil

View File

@@ -5,8 +5,8 @@ import (
"encoding/json"
"errors"
"fmt"
log "github.com/sirupsen/logrus"
goio "io"
"io/fs"
"io/ioutil"
"os"
"os/exec"
@@ -17,6 +17,9 @@ import (
"testing"
"time"
log "github.com/sirupsen/logrus"
"k8s.io/apimachinery/pkg/api/resource"
"github.com/ghodss/yaml"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
@@ -130,6 +133,31 @@ func newServiceWithCommitSHA(root, revision string) *Service {
return service
}
// createSymlink creates a symlink with name linkName to file destName in
// workingDir
func createSymlink(t *testing.T, workingDir, destName, linkName string) error {
oldWorkingDir, err := os.Getwd()
if err != nil {
return err
}
if workingDir != "" {
err = os.Chdir(workingDir)
if err != nil {
return err
}
defer func() {
if err := os.Chdir(oldWorkingDir); err != nil {
t.Fatal(err.Error())
}
}()
}
err = os.Symlink(destName, linkName)
if err != nil {
return err
}
return nil
}
func TestGenerateYamlManifestInDir(t *testing.T) {
service := newService("../..")
@@ -145,39 +173,39 @@ func TestGenerateYamlManifestInDir(t *testing.T) {
assert.Equal(t, countOfManifests, len(res1.Manifests))
// this will test concatenated manifests to verify we split YAMLs correctly
res2, err := GenerateManifests(context.Background(), "./testdata/concatenated", "/", "", &q, false, &git.NoopCredsStore{})
res2, err := GenerateManifests(context.Background(), "./testdata/concatenated", "/", "", &q, false, &git.NoopCredsStore{}, resource.MustParse("0"))
assert.NoError(t, err)
assert.Equal(t, 3, len(res2.Manifests))
}
func Test_GenerateManifests_NoOutOfBoundsAccess(t *testing.T) {
testCases := []struct{
name string
outOfBoundsFilename string
testCases := []struct {
name string
outOfBoundsFilename string
outOfBoundsFileContents string
mustNotContain string // Optional string that must not appear in error or manifest output. If empty, use outOfBoundsFileContents.
mustNotContain string // Optional string that must not appear in error or manifest output. If empty, use outOfBoundsFileContents.
}{
{
name: "out of bounds JSON file should not appear in error output",
outOfBoundsFilename: "test.json",
name: "out of bounds JSON file should not appear in error output",
outOfBoundsFilename: "test.json",
outOfBoundsFileContents: `{"some": "json"}`,
},
{
name: "malformed JSON file contents should not appear in error output",
outOfBoundsFilename: "test.json",
name: "malformed JSON file contents should not appear in error output",
outOfBoundsFilename: "test.json",
outOfBoundsFileContents: "$",
},
{
name: "out of bounds JSON manifest should not appear in manifest output",
name: "out of bounds JSON manifest should not appear in manifest output",
outOfBoundsFilename: "test.json",
// JSON marshalling is deterministic. So if there's a leak, exactly this should appear in the manifests.
outOfBoundsFileContents: `{"apiVersion":"v1","kind":"Secret","metadata":{"name":"test","namespace":"default"},"type":"Opaque"}`,
},
{
name: "out of bounds YAML manifest should not appear in manifest output",
outOfBoundsFilename: "test.yaml",
name: "out of bounds YAML manifest should not appear in manifest output",
outOfBoundsFilename: "test.yaml",
outOfBoundsFileContents: "apiVersion: v1\nkind: Secret\nmetadata:\n name: test\n namespace: default\ntype: Opaque",
mustNotContain: `{"apiVersion":"v1","kind":"Secret","metadata":{"name":"test","namespace":"default"},"type":"Opaque"}`,
mustNotContain: `{"apiVersion":"v1","kind":"Secret","metadata":{"name":"test","namespace":"default"},"type":"Opaque"}`,
},
}
@@ -201,7 +229,7 @@ func Test_GenerateManifests_NoOutOfBoundsAccess(t *testing.T) {
}
q := apiclient.ManifestRequest{Repo: &argoappv1.Repository{}, ApplicationSource: &argoappv1.ApplicationSource{}}
res, err := GenerateManifests(context.Background(), repoDir, "", "", &q, false, &git.NoopCredsStore{})
res, err := GenerateManifests(context.Background(), repoDir, "", "", &q, false, &git.NoopCredsStore{}, resource.MustParse("0"))
require.Error(t, err)
assert.NotContains(t, err.Error(), mustNotContain)
assert.Contains(t, err.Error(), "illegal filepath")
@@ -216,7 +244,7 @@ func TestGenerateManifests_MissingSymlinkDestination(t *testing.T) {
require.NoError(t, err)
q := apiclient.ManifestRequest{Repo: &argoappv1.Repository{}, ApplicationSource: &argoappv1.ApplicationSource{}}
_, err = GenerateManifests(context.Background(), repoDir, "", "", &q, false, &git.NoopCredsStore{})
_, err = GenerateManifests(context.Background(), repoDir, "", "", &q, false, &git.NoopCredsStore{}, resource.MustParse("0"))
require.NoError(t, err)
}
@@ -1046,15 +1074,15 @@ func TestGenerateNullList(t *testing.T) {
}
func TestIdentifyAppSourceTypeByAppDirWithKustomizations(t *testing.T) {
sourceType, err := GetAppSourceType(context.Background(), &argoappv1.ApplicationSource{}, "./testdata/kustomization_yaml", "testapp", map[string]bool{})
sourceType, err := GetAppSourceType(context.Background(), &argoappv1.ApplicationSource{}, "./testdata/kustomization_yaml", "testapp", map[string]bool{}, []string{})
assert.Nil(t, err)
assert.Equal(t, argoappv1.ApplicationSourceTypeKustomize, sourceType)
sourceType, err = GetAppSourceType(context.Background(), &argoappv1.ApplicationSource{}, "./testdata/kustomization_yml", "testapp", map[string]bool{})
sourceType, err = GetAppSourceType(context.Background(), &argoappv1.ApplicationSource{}, "./testdata/kustomization_yml", "testapp", map[string]bool{}, []string{})
assert.Nil(t, err)
assert.Equal(t, argoappv1.ApplicationSourceTypeKustomize, sourceType)
sourceType, err = GetAppSourceType(context.Background(), &argoappv1.ApplicationSource{}, "./testdata/Kustomization", "testapp", map[string]bool{})
sourceType, err = GetAppSourceType(context.Background(), &argoappv1.ApplicationSource{}, "./testdata/Kustomization", "testapp", map[string]bool{}, []string{})
assert.Nil(t, err)
assert.Equal(t, argoappv1.ApplicationSourceTypeKustomize, sourceType)
}
@@ -1107,7 +1135,7 @@ func TestGenerateFromUTF16(t *testing.T) {
Repo: &argoappv1.Repository{},
ApplicationSource: &argoappv1.ApplicationSource{},
}
res1, err := GenerateManifests(context.Background(), "./testdata/utf-16", "/", "", &q, false, &git.NoopCredsStore{})
res1, err := GenerateManifests(context.Background(), "./testdata/utf-16", "/", "", &q, false, &git.NoopCredsStore{}, resource.MustParse("0"))
assert.Nil(t, err)
assert.Equal(t, 2, len(res1.Manifests))
}
@@ -1119,17 +1147,20 @@ func TestListApps(t *testing.T) {
assert.NoError(t, err)
expectedApps := map[string]string{
"Kustomization": "Kustomize",
"app-parameters/multi": "Kustomize",
"app-parameters/single-app-only": "Kustomize",
"app-parameters/single-global": "Kustomize",
"invalid-helm": "Helm",
"invalid-kustomize": "Kustomize",
"kustomization_yaml": "Kustomize",
"kustomization_yml": "Kustomize",
"my-chart": "Helm",
"my-chart-2": "Helm",
"values-files": "Helm",
"Kustomization": "Kustomize",
"app-parameters/multi": "Kustomize",
"app-parameters/single-app-only": "Kustomize",
"app-parameters/single-global": "Kustomize",
"app-parameters/single-global-helm": "Helm",
"invalid-helm": "Helm",
"in-bounds-values-file-link": "Helm",
"invalid-kustomize": "Kustomize",
"kustomization_yaml": "Kustomize",
"kustomization_yml": "Kustomize",
"my-chart": "Helm",
"my-chart-2": "Helm",
"out-of-bounds-values-file-link": "Helm",
"values-files": "Helm",
}
assert.Equal(t, expectedApps, res.Apps)
}
@@ -1448,6 +1479,7 @@ func mkTempParameters(source string) string {
func runWithTempTestdata(t *testing.T, path string, runner func(t *testing.T, path string)) {
tempDir := mkTempParameters("./testdata/app-parameters")
runner(t, filepath.Join(tempDir, "app-parameters", path))
os.RemoveAll(tempDir)
}
func TestGenerateManifestsWithAppParameterFile(t *testing.T) {
@@ -1480,6 +1512,35 @@ func TestGenerateManifestsWithAppParameterFile(t *testing.T) {
})
})
t.Run("Single global override Helm", func(t *testing.T) {
runWithTempTestdata(t, "single-global-helm", func(t *testing.T, path string) {
service := newService(".")
manifests, err := service.GenerateManifest(context.Background(), &apiclient.ManifestRequest{
Repo: &argoappv1.Repository{},
ApplicationSource: &argoappv1.ApplicationSource{
Path: path,
},
})
require.NoError(t, err)
resourceByKindName := make(map[string]*unstructured.Unstructured)
for _, manifest := range manifests.Manifests {
var un unstructured.Unstructured
err := yaml.Unmarshal([]byte(manifest), &un)
if !assert.NoError(t, err) {
return
}
resourceByKindName[fmt.Sprintf("%s/%s", un.GetKind(), un.GetName())] = &un
}
deployment, ok := resourceByKindName["Deployment/guestbook-ui"]
require.True(t, ok)
containers, ok, _ := unstructured.NestedSlice(deployment.Object, "spec", "template", "spec", "containers")
require.True(t, ok)
image, ok, _ := unstructured.NestedString(containers[0].(map[string]interface{}), "image")
require.True(t, ok)
assert.Equal(t, "gcr.io/heptio-images/ks-guestbook-demo:0.2", image)
})
})
t.Run("Application specific override", func(t *testing.T) {
service := newService(".")
runWithTempTestdata(t, "single-app-only", func(t *testing.T, path string) {
@@ -1539,6 +1600,28 @@ func TestGenerateManifestsWithAppParameterFile(t *testing.T) {
assert.Equal(t, "gcr.io/heptio-images/ks-guestbook-demo:0.1", image)
})
})
t.Run("Override info does not appear in cache key", func(t *testing.T) {
service := newService(".")
runWithTempTestdata(t, "single-global", func(t *testing.T, path string) {
source := &argoappv1.ApplicationSource{
Path: path,
}
sourceCopy := source.DeepCopy() // make a copy in case GenerateManifest mutates it.
_, err := service.GenerateManifest(context.Background(), &apiclient.ManifestRequest{
Repo: &argoappv1.Repository{},
ApplicationSource: sourceCopy,
AppName: "test",
})
assert.NoError(t, err)
res := &cache.CachedManifestResponse{}
// Try to pull from the cache with a `source` that does not include any overrides. Overrides should not be
// part of the cache key, because you can't get the overrides without a repo operation. And avoiding repo
// operations is the point of the cache.
err = service.cache.GetManifests(mock.Anything, source, &argoappv1.ClusterInfo{}, "", "", "", "test", res)
assert.NoError(t, err)
})
})
}
func TestGenerateManifestWithAnnotatedAndRegularGitTagHashes(t *testing.T) {
@@ -1653,7 +1736,7 @@ func TestFindResources(t *testing.T) {
Recurse: true,
Include: tc.include,
Exclude: tc.exclude,
}, map[string]bool{})
}, map[string]bool{}, resource.MustParse("0"))
if !assert.NoError(t, err) {
return
}
@@ -1670,7 +1753,7 @@ func TestFindManifests_Exclude(t *testing.T) {
objs, err := findManifests(&log.Entry{}, "testdata/app-include-exclude", ".", nil, argoappv1.ApplicationSourceDirectory{
Recurse: true,
Exclude: "subdir/deploymentSub.yaml",
}, map[string]bool{})
}, map[string]bool{}, resource.MustParse("0"))
if !assert.NoError(t, err) || !assert.Len(t, objs, 1) {
return
@@ -1683,7 +1766,7 @@ func TestFindManifests_Exclude_NothingMatches(t *testing.T) {
objs, err := findManifests(&log.Entry{}, "testdata/app-include-exclude", ".", nil, argoappv1.ApplicationSourceDirectory{
Recurse: true,
Exclude: "nothing.yaml",
}, map[string]bool{})
}, map[string]bool{}, resource.MustParse("0"))
if !assert.NoError(t, err) || !assert.Len(t, objs, 2) {
return
@@ -1693,6 +1776,477 @@ func TestFindManifests_Exclude_NothingMatches(t *testing.T) {
[]string{"nginx-deployment", "nginx-deployment-sub"}, []string{objs[0].GetName(), objs[1].GetName()})
}
func tempDir(t *testing.T) string {
dir, err := ioutil.TempDir(".", "")
require.NoError(t, err)
t.Cleanup(func() {
err = os.RemoveAll(dir)
if err != nil {
panic(err)
}
})
absDir, err := filepath.Abs(dir)
require.NoError(t, err)
return absDir
}
func walkFor(t *testing.T, root string, testPath string, run func(info fs.FileInfo)) {
var hitExpectedPath = false
err := filepath.Walk(root, func(path string, info fs.FileInfo, err error) error {
if path == testPath {
require.NoError(t, err)
hitExpectedPath = true
run(info)
}
return nil
})
require.NoError(t, err)
assert.True(t, hitExpectedPath, "did not hit expected path when walking directory")
}
func Test_getPotentiallyValidManifestFile(t *testing.T) {
// These tests use filepath.Walk instead of os.Stat to get file info, because FileInfo from os.Stat does not return
// true for IsSymlink like os.Walk does.
// These tests do not use t.TempDir() because those directories can contain symlinks which cause test to fail
// InBound checks.
t.Run("non-JSON/YAML is skipped with an empty ignore message", func(t *testing.T) {
appDir := tempDir(t)
filePath := filepath.Join(appDir, "not-json-or-yaml")
file, err := os.OpenFile(filePath, os.O_RDONLY|os.O_CREATE, 0644)
require.NoError(t, err)
err = file.Close()
require.NoError(t, err)
walkFor(t, appDir, filePath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(filePath, info, appDir, appDir, "", "")
assert.Nil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.NoError(t, err)
})
})
t.Run("circular link should throw an error", func(t *testing.T) {
appDir := tempDir(t)
aPath := filepath.Join(appDir, "a.json")
bPath := filepath.Join(appDir, "b.json")
err := os.Symlink(bPath, aPath)
require.NoError(t, err)
err = os.Symlink(aPath, bPath)
require.NoError(t, err)
walkFor(t, appDir, aPath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(aPath, info, appDir, appDir, "", "")
assert.Nil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.ErrorContains(t, err, "too many links")
})
})
t.Run("symlink with missing destination should throw an error", func(t *testing.T) {
appDir := tempDir(t)
aPath := filepath.Join(appDir, "a.json")
bPath := filepath.Join(appDir, "b.json")
err := os.Symlink(bPath, aPath)
require.NoError(t, err)
walkFor(t, appDir, aPath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(aPath, info, appDir, appDir, "", "")
assert.Nil(t, realFileInfo)
assert.NotEmpty(t, ignoreMessage)
assert.NoError(t, err)
})
})
t.Run("out-of-bounds symlink should throw an error", func(t *testing.T) {
appDir := tempDir(t)
linkPath := filepath.Join(appDir, "a.json")
err := os.Symlink("..", linkPath)
require.NoError(t, err)
walkFor(t, appDir, linkPath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(linkPath, info, appDir, appDir, "", "")
assert.Nil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.ErrorContains(t, err, "illegal filepath in symlink")
})
})
t.Run("symlink to a non-regular file should be skipped with warning", func(t *testing.T) {
appDir := tempDir(t)
dirPath := filepath.Join(appDir, "test.dir")
err := os.MkdirAll(dirPath, 0644)
require.NoError(t, err)
linkPath := filepath.Join(appDir, "test.json")
err = os.Symlink(dirPath, linkPath)
require.NoError(t, err)
walkFor(t, appDir, linkPath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(linkPath, info, appDir, appDir, "", "")
assert.Nil(t, realFileInfo)
assert.Contains(t, ignoreMessage, "non-regular file")
assert.NoError(t, err)
})
})
t.Run("non-included file should be skipped with no message", func(t *testing.T) {
appDir := tempDir(t)
filePath := filepath.Join(appDir, "not-included.yaml")
file, err := os.OpenFile(filePath, os.O_RDONLY|os.O_CREATE, 0644)
require.NoError(t, err)
err = file.Close()
require.NoError(t, err)
walkFor(t, appDir, filePath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(filePath, info, appDir, appDir, "*.json", "")
assert.Nil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.NoError(t, err)
})
})
t.Run("excluded file should be skipped with no message", func(t *testing.T) {
appDir := tempDir(t)
filePath := filepath.Join(appDir, "excluded.json")
file, err := os.OpenFile(filePath, os.O_RDONLY|os.O_CREATE, 0644)
require.NoError(t, err)
err = file.Close()
require.NoError(t, err)
walkFor(t, appDir, filePath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(filePath, info, appDir, appDir, "", "excluded.*")
assert.Nil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.NoError(t, err)
})
})
t.Run("symlink to a regular file is potentially valid", func(t *testing.T) {
appDir := tempDir(t)
filePath := filepath.Join(appDir, "regular-file")
file, err := os.OpenFile(filePath, os.O_RDONLY|os.O_CREATE, 0644)
require.NoError(t, err)
err = file.Close()
require.NoError(t, err)
linkPath := filepath.Join(appDir, "link.json")
err = os.Symlink(filePath, linkPath)
require.NoError(t, err)
walkFor(t, appDir, linkPath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(linkPath, info, appDir, appDir, "", "")
assert.NotNil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.NoError(t, err)
})
})
t.Run("a regular file is potentially valid", func(t *testing.T) {
appDir := tempDir(t)
filePath := filepath.Join(appDir, "regular-file.json")
file, err := os.OpenFile(filePath, os.O_RDONLY|os.O_CREATE, 0644)
require.NoError(t, err)
err = file.Close()
require.NoError(t, err)
walkFor(t, appDir, filePath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(filePath, info, appDir, appDir, "", "")
assert.NotNil(t, realFileInfo)
assert.Empty(t, ignoreMessage)
assert.NoError(t, err)
})
})
t.Run("realFileInfo is for the destination rather than the symlink", func(t *testing.T) {
appDir := tempDir(t)
filePath := filepath.Join(appDir, "regular-file")
file, err := os.OpenFile(filePath, os.O_RDONLY|os.O_CREATE, 0644)
require.NoError(t, err)
err = file.Close()
require.NoError(t, err)
linkPath := filepath.Join(appDir, "link.json")
err = os.Symlink(filePath, linkPath)
require.NoError(t, err)
walkFor(t, appDir, linkPath, func(info fs.FileInfo) {
realFileInfo, ignoreMessage, err := getPotentiallyValidManifestFile(linkPath, info, appDir, appDir, "", "")
assert.NotNil(t, realFileInfo)
assert.Equal(t, filepath.Base(filePath), realFileInfo.Name())
assert.Empty(t, ignoreMessage)
assert.NoError(t, err)
})
})
}
func Test_getPotentiallyValidManifests(t *testing.T) {
// Tests which return no manifests and an error check to make sure the directory exists before running. A missing
// directory would produce those same results.
logCtx := log.WithField("test", "test")
t.Run("unreadable file throws error", func(t *testing.T) {
appDir := t.TempDir()
unreadablePath := filepath.Join(appDir, "unreadable.json")
err := os.WriteFile(unreadablePath, []byte{}, 0666)
require.NoError(t, err)
err = os.Chmod(appDir, 0000)
require.NoError(t, err)
manifests, err := getPotentiallyValidManifests(logCtx, appDir, appDir, false, "", "", resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
// allow cleanup
err = os.Chmod(appDir, 0777)
if err != nil {
panic(err)
}
})
t.Run("no recursion when recursion is disabled", func(t *testing.T) {
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/recurse", "./testdata/recurse", false, "", "", resource.MustParse("0"))
assert.Len(t, manifests, 1)
assert.NoError(t, err)
})
t.Run("recursion when recursion is enabled", func(t *testing.T) {
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/recurse", "./testdata/recurse", true, "", "", resource.MustParse("0"))
assert.Len(t, manifests, 2)
assert.NoError(t, err)
})
t.Run("non-JSON/YAML is skipped", func(t *testing.T) {
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/non-manifest-file", "./testdata/non-manifest-file", false, "", "", resource.MustParse("0"))
assert.Empty(t, manifests)
assert.NoError(t, err)
})
t.Run("circular link should throw an error", func(t *testing.T) {
const testDir = "./testdata/circular-link"
require.DirExists(t, testDir)
require.NoError(t, createSymlink(t, testDir, "a.json", "b.json"))
defer os.Remove(path.Join(testDir, "a.json"))
require.NoError(t, createSymlink(t, testDir, "b.json", "a.json"))
defer os.Remove(path.Join(testDir, "b.json"))
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/circular-link", "./testdata/circular-link", false, "", "", resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("out-of-bounds symlink should throw an error", func(t *testing.T) {
require.DirExists(t, "./testdata/out-of-bounds-link")
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/out-of-bounds-link", "./testdata/out-of-bounds-link", false, "", "", resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("symlink to a regular file works", func(t *testing.T) {
repoRoot, err := filepath.Abs("./testdata/in-bounds-link")
require.NoError(t, err)
appPath, err := filepath.Abs("./testdata/in-bounds-link/app")
require.NoError(t, err)
manifests, err := getPotentiallyValidManifests(logCtx, appPath, repoRoot, false, "", "", resource.MustParse("0"))
assert.Len(t, manifests, 1)
assert.NoError(t, err)
})
t.Run("symlink to nowhere should be ignored", func(t *testing.T) {
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/link-to-nowhere", "./testdata/link-to-nowhere", false, "", "", resource.MustParse("0"))
assert.Empty(t, manifests)
assert.NoError(t, err)
})
t.Run("link to over-sized manifest fails", func(t *testing.T) {
repoRoot, err := filepath.Abs("./testdata/in-bounds-link")
require.NoError(t, err)
appPath, err := filepath.Abs("./testdata/in-bounds-link/app")
require.NoError(t, err)
// The file is 35 bytes.
manifests, err := getPotentiallyValidManifests(logCtx, appPath, repoRoot, false, "", "", resource.MustParse("34"))
assert.Empty(t, manifests)
assert.ErrorIs(t, err, ErrExceededMaxCombinedManifestFileSize)
})
t.Run("group of files should be limited at precisely the sum of their size", func(t *testing.T) {
// There is a total of 10 files, ech file being 10 bytes.
manifests, err := getPotentiallyValidManifests(logCtx, "./testdata/several-files", "./testdata/several-files", false, "", "", resource.MustParse("365"))
assert.Len(t, manifests, 10)
assert.NoError(t, err)
manifests, err = getPotentiallyValidManifests(logCtx, "./testdata/several-files", "./testdata/several-files", false, "", "", resource.MustParse("100"))
assert.Empty(t, manifests)
assert.ErrorIs(t, err, ErrExceededMaxCombinedManifestFileSize)
})
}
func Test_findManifests(t *testing.T) {
logCtx := log.WithField("test", "test")
noRecurse := argoappv1.ApplicationSourceDirectory{Recurse: false}
t.Run("unreadable file throws error", func(t *testing.T) {
appDir := t.TempDir()
unreadablePath := filepath.Join(appDir, "unreadable.json")
err := os.WriteFile(unreadablePath, []byte{}, 0666)
require.NoError(t, err)
err = os.Chmod(appDir, 0000)
require.NoError(t, err)
manifests, err := findManifests(logCtx, appDir, appDir, nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
// allow cleanup
err = os.Chmod(appDir, 0777)
if err != nil {
panic(err)
}
})
t.Run("no recursion when recursion is disabled", func(t *testing.T) {
manifests, err := findManifests(logCtx, "./testdata/recurse", "./testdata/recurse", nil, noRecurse, nil, resource.MustParse("0"))
assert.Len(t, manifests, 2)
assert.NoError(t, err)
})
t.Run("recursion when recursion is enabled", func(t *testing.T) {
recurse := argoappv1.ApplicationSourceDirectory{Recurse: true}
manifests, err := findManifests(logCtx, "./testdata/recurse", "./testdata/recurse", nil, recurse, nil, resource.MustParse("0"))
assert.Len(t, manifests, 4)
assert.NoError(t, err)
})
t.Run("non-JSON/YAML is skipped", func(t *testing.T) {
manifests, err := findManifests(logCtx, "./testdata/non-manifest-file", "./testdata/non-manifest-file", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.NoError(t, err)
})
t.Run("circular link should throw an error", func(t *testing.T) {
const testDir = "./testdata/circular-link"
require.DirExists(t, testDir)
require.NoError(t, createSymlink(t, testDir, "a.json", "b.json"))
defer os.Remove(path.Join(testDir, "a.json"))
require.NoError(t, createSymlink(t, testDir, "b.json", "a.json"))
defer os.Remove(path.Join(testDir, "b.json"))
manifests, err := findManifests(logCtx, "./testdata/circular-link", "./testdata/circular-link", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("out-of-bounds symlink should throw an error", func(t *testing.T) {
require.DirExists(t, "./testdata/out-of-bounds-link")
manifests, err := findManifests(logCtx, "./testdata/out-of-bounds-link", "./testdata/out-of-bounds-link", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("symlink to a regular file works", func(t *testing.T) {
repoRoot, err := filepath.Abs("./testdata/in-bounds-link")
require.NoError(t, err)
appPath, err := filepath.Abs("./testdata/in-bounds-link/app")
require.NoError(t, err)
manifests, err := findManifests(logCtx, appPath, repoRoot, nil, noRecurse, nil, resource.MustParse("0"))
assert.Len(t, manifests, 1)
assert.NoError(t, err)
})
t.Run("symlink to nowhere should be ignored", func(t *testing.T) {
manifests, err := findManifests(logCtx, "./testdata/link-to-nowhere", "./testdata/link-to-nowhere", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.NoError(t, err)
})
t.Run("link to over-sized manifest fails", func(t *testing.T) {
repoRoot, err := filepath.Abs("./testdata/in-bounds-link")
require.NoError(t, err)
appPath, err := filepath.Abs("./testdata/in-bounds-link/app")
require.NoError(t, err)
// The file is 35 bytes.
manifests, err := findManifests(logCtx, appPath, repoRoot, nil, noRecurse, nil, resource.MustParse("34"))
assert.Empty(t, manifests)
assert.ErrorIs(t, err, ErrExceededMaxCombinedManifestFileSize)
})
t.Run("group of files should be limited at precisely the sum of their size", func(t *testing.T) {
// There is a total of 10 files, each file being 10 bytes.
manifests, err := findManifests(logCtx, "./testdata/several-files", "./testdata/several-files", nil, noRecurse, nil, resource.MustParse("365"))
assert.Len(t, manifests, 10)
assert.NoError(t, err)
manifests, err = findManifests(logCtx, "./testdata/several-files", "./testdata/several-files", nil, noRecurse, nil, resource.MustParse("364"))
assert.Empty(t, manifests)
assert.ErrorIs(t, err, ErrExceededMaxCombinedManifestFileSize)
})
t.Run("jsonnet isn't counted against size limit", func(t *testing.T) {
// Each file is 36 bytes. Only the 36-byte json file should be counted against the limit.
manifests, err := findManifests(logCtx, "./testdata/jsonnet-and-json", "./testdata/jsonnet-and-json", nil, noRecurse, nil, resource.MustParse("36"))
assert.Len(t, manifests, 2)
assert.NoError(t, err)
manifests, err = findManifests(logCtx, "./testdata/jsonnet-and-json", "./testdata/jsonnet-and-json", nil, noRecurse, nil, resource.MustParse("35"))
assert.Empty(t, manifests)
assert.ErrorIs(t, err, ErrExceededMaxCombinedManifestFileSize)
})
t.Run("partially valid YAML file throws an error", func(t *testing.T) {
require.DirExists(t, "./testdata/partially-valid-yaml")
manifests, err := findManifests(logCtx, "./testdata/partially-valid-yaml", "./testdata/partially-valid-yaml", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("invalid manifest throws an error", func(t *testing.T) {
require.DirExists(t, "./testdata/invalid-manifests")
manifests, err := findManifests(logCtx, "./testdata/invalid-manifests", "./testdata/invalid-manifests", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("irrelevant YAML gets skipped, relevant YAML gets parsed", func(t *testing.T) {
manifests, err := findManifests(logCtx, "./testdata/irrelevant-yaml", "./testdata/irrelevant-yaml", nil, noRecurse, nil, resource.MustParse("0"))
assert.Len(t, manifests, 1)
assert.NoError(t, err)
})
t.Run("multiple JSON objects in one file throws an error", func(t *testing.T) {
require.DirExists(t, "./testdata/json-list")
manifests, err := findManifests(logCtx, "./testdata/json-list", "./testdata/json-list", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("invalid JSON throws an error", func(t *testing.T) {
require.DirExists(t, "./testdata/invalid-json")
manifests, err := findManifests(logCtx, "./testdata/invalid-json", "./testdata/invalid-json", nil, noRecurse, nil, resource.MustParse("0"))
assert.Empty(t, manifests)
assert.Error(t, err)
})
t.Run("valid JSON returns manifest and no error", func(t *testing.T) {
manifests, err := findManifests(logCtx, "./testdata/valid-json", "./testdata/valid-json", nil, noRecurse, nil, resource.MustParse("0"))
assert.Len(t, manifests, 1)
assert.NoError(t, err)
})
t.Run("YAML with an empty document doesn't throw an error", func(t *testing.T) {
manifests, err := findManifests(logCtx, "./testdata/yaml-with-empty-document", "./testdata/yaml-with-empty-document", nil, noRecurse, nil, resource.MustParse("0"))
assert.Len(t, manifests, 1)
assert.NoError(t, err)
})
}
func TestTestRepoOCI(t *testing.T) {
service := newService(".")
_, err := service.TestRepository(context.Background(), &apiclient.TestRepositoryRequest{
@@ -1904,3 +2458,23 @@ func Test_populateHelmAppDetails(t *testing.T) {
assert.Len(t, res.Helm.Parameters, 3)
assert.Len(t, res.Helm.ValueFiles, 4)
}
func Test_populateHelmAppDetails_values_symlinks(t *testing.T) {
t.Run("inbound", func(t *testing.T) {
res := apiclient.RepoAppDetailsResponse{}
q := apiclient.RepoServerAppDetailsQuery{Repo: &argoappv1.Repository{}, Source: &argoappv1.ApplicationSource{}}
err := populateHelmAppDetails(&res, "./testdata/in-bounds-values-file-link/", "./testdata/in-bounds-values-file-link/", &q)
require.NoError(t, err)
assert.NotEmpty(t, res.Helm.Values)
assert.NotEmpty(t, res.Helm.Parameters)
})
t.Run("out of bounds", func(t *testing.T) {
res := apiclient.RepoAppDetailsResponse{}
q := apiclient.RepoServerAppDetailsQuery{Repo: &argoappv1.Repository{}, Source: &argoappv1.ApplicationSource{}}
err := populateHelmAppDetails(&res, "./testdata/out-of-bounds-values-file-link/", "./testdata/out-of-bounds-values-file-link/", &q)
require.NoError(t, err)
assert.Empty(t, res.Helm.Values)
assert.Empty(t, res.Helm.Parameters)
})
}

View File

@@ -0,0 +1,4 @@
helm:
parameters:
- name: image.tag
value: '0.2'

View File

@@ -0,0 +1,2 @@
name: my-chart
version: 1.1.0

View File

@@ -0,0 +1,18 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: guestbook-ui
spec:
selector:
matchLabels:
app: guestbook-ui
template:
metadata:
labels:
app: guestbook-ui
spec:
containers:
- image: gcr.io/heptio-images/ks-guestbook-demo:{{.Values.image.tag}}
name: guestbook-ui
ports:
- containerPort: 81

View File

@@ -0,0 +1,2 @@
image:
tag: 0.1

View File

View File

@@ -0,0 +1 @@
../cm.yaml

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ServiceAccount

View File

@@ -0,0 +1,2 @@
name: my-chart
version: 1.1.0

View File

@@ -0,0 +1 @@
some: yaml

View File

@@ -0,0 +1 @@
values-2.yaml

View File

@@ -0,0 +1 @@
[

View File

@@ -0,0 +1 @@
some: [irrelevant, yaml]

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ConfigMap

View File

@@ -0,0 +1,2 @@
{"apiVersion": "v1", "kind": "ConfigMap"}
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "Pod"}

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "Pod"}

View File

@@ -0,0 +1 @@
nowhere

View File

@@ -0,0 +1 @@
../out-of-bounds.json

View File

@@ -0,0 +1,2 @@
name: my-chart
version: 1.1.0

View File

@@ -0,0 +1 @@
../out-of-bounds.yaml

View File

View File

@@ -0,0 +1 @@
some: yaml

View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: ConfigMap
---
invalid:

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ConfigMap

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ConfigMap

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ConfigMap

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ConfigMap

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1,2 @@
apiVersion: v1
kind: ConfigMap

View File

@@ -0,0 +1 @@
This file shouldn't be counted in the manifest file size limit, because it isn't JSON or YAML.

View File

@@ -0,0 +1 @@
{"apiVersion": "v1", "kind": "ConfigMap"}

View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: ConfigMap
---
---

View File

@@ -3,10 +3,11 @@ package reposerver
import (
"crypto/tls"
"fmt"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"os"
"path/filepath"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
grpc_middleware "github.com/grpc-ecosystem/go-grpc-middleware"
grpc_logrus "github.com/grpc-ecosystem/go-grpc-middleware/logging/logrus"
grpc_prometheus "github.com/grpc-ecosystem/go-grpc-prometheus"
@@ -15,6 +16,7 @@ import (
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/keepalive"
"google.golang.org/grpc/reflection"
"github.com/argoproj/argo-cd/v2/common"
@@ -86,6 +88,11 @@ func NewServer(metricsServer *metrics.MetricsServer, cache *reposervercache.Cach
grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(streamInterceptors...)),
grpc.MaxRecvMsgSize(apiclient.MaxGRPCMessageSize),
grpc.MaxSendMsgSize(apiclient.MaxGRPCMessageSize),
grpc.KeepaliveEnforcementPolicy(
keepalive.EnforcementPolicy{
MinTime: common.GRPCKeepAliveEnforcementMinimum,
},
),
}
// We do allow for non-TLS servers to be created, in case of mTLS will be

View File

@@ -33,17 +33,19 @@ type terminalHandler struct {
enf *rbac.Enforcer
cache *servercache.Cache
appResourceTreeFn func(ctx context.Context, app *appv1.Application) (*appv1.ApplicationTree, error)
allowedShells []string
}
// NewHandler returns a new terminal handler.
func NewHandler(appLister applisters.ApplicationNamespaceLister, db db.ArgoDB, enf *rbac.Enforcer, cache *servercache.Cache,
appResourceTree AppResourceTreeFn) *terminalHandler {
appResourceTree AppResourceTreeFn, allowedShells []string) *terminalHandler {
return &terminalHandler{
appLister: appLister,
db: db,
enf: enf,
cache: cache,
appResourceTreeFn: appResourceTree,
allowedShells: allowedShells,
}
}
@@ -123,7 +125,7 @@ func (s *terminalHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Namespace name is not valid", http.StatusBadRequest)
return
}
shell := q.Get("shell") // No need to validate. Will only buse used if it's in the allow-list.
shell := q.Get("shell") // No need to validate. Will only be used if it's in the allow-list.
ctx := r.Context()
@@ -216,14 +218,12 @@ func (s *terminalHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
defer session.Done()
validShells := []string{"bash", "sh", "powershell", "cmd"}
if isValidShell(validShells, shell) {
if isValidShell(s.allowedShells, shell) {
cmd := []string{shell}
err = startProcess(kubeClientset, config, namespace, podName, container, cmd, session)
} else {
// No shell given or it was not valid: try some shells until one succeeds or all fail
// FIXME: if the first shell fails then the first keyboard event is lost
for _, testShell := range validShells {
// No shell given or the given shell was not allowed: try the configured shells until one succeeds or all fail.
for _, testShell := range s.allowedShells {
cmd := []string{testShell}
if err = startProcess(kubeClientset, config, namespace, podName, container, cmd, session); err == nil {
break

View File

@@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"net/http"
"sync"
"time"
"github.com/gorilla/websocket"
@@ -26,6 +27,7 @@ type terminalSession struct {
sizeChan chan remotecommand.TerminalSize
doneChan chan struct{}
tty bool
readLock sync.Mutex
}
// newTerminalSession create terminalSession
@@ -60,7 +62,9 @@ func (t *terminalSession) Next() *remotecommand.TerminalSize {
// Read called in a loop from remotecommand as long as the process is running
func (t *terminalSession) Read(p []byte) (int, error) {
t.readLock.Lock()
_, message, err := t.wsConn.ReadMessage()
t.readLock.Unlock()
if err != nil {
log.Errorf("read message err: %v", err)
return copy(p, EndOfTransmission), err

View File

@@ -22,6 +22,8 @@ import (
// nolint:staticcheck
golang_proto "github.com/golang/protobuf/proto"
netCtx "context"
"github.com/argoproj/pkg/sync"
"github.com/go-redis/redis/v8"
"github.com/golang-jwt/jwt/v4"
@@ -35,11 +37,11 @@ import (
log "github.com/sirupsen/logrus"
"github.com/soheilhy/cmux"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
netCtx "golang.org/x/net/context"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/keepalive"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/reflection"
"google.golang.org/grpc/status"
@@ -132,7 +134,7 @@ var backoff = wait.Backoff{
var (
clientConstraint = fmt.Sprintf(">= %s", common.MinClientVersion)
baseHRefRegex = regexp.MustCompile(`<base href="(.*)">`)
baseHRefRegex = regexp.MustCompile(`<base href="(.*?)">`)
// limits number of concurrent login requests to prevent password brute forcing. If set to 0 then no limit is enforced.
maxConcurrentLoginRequestsCount = 50
replicasCount = 1
@@ -282,11 +284,97 @@ func (a *ArgoCDServer) healthCheck(r *http.Request) error {
return nil
}
type Listeners struct {
Main net.Listener
Metrics net.Listener
GatewayConn *grpc.ClientConn
}
func (l *Listeners) Close() error {
if l.Main != nil {
if err := l.Main.Close(); err != nil {
return err
}
l.Main = nil
}
if l.Metrics != nil {
if err := l.Metrics.Close(); err != nil {
return err
}
l.Metrics = nil
}
if l.GatewayConn != nil {
if err := l.GatewayConn.Close(); err != nil {
return err
}
l.GatewayConn = nil
}
return nil
}
func startListener(host string, port int) (net.Listener, error) {
var conn net.Listener
var realErr error
_ = wait.ExponentialBackoff(backoff, func() (bool, error) {
conn, realErr = net.Listen("tcp", fmt.Sprintf("%s:%d", host, port))
if realErr != nil {
return false, nil
}
return true, nil
})
return conn, realErr
}
func (a *ArgoCDServer) Listen() (*Listeners, error) {
mainLn, err := startListener(a.ListenHost, a.ListenPort)
if err != nil {
return nil, err
}
metricsLn, err := startListener(a.ListenHost, a.MetricsPort)
if err != nil {
io.Close(mainLn)
return nil, err
}
var dOpts []grpc.DialOption
dOpts = append(dOpts, grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(apiclient.MaxGRPCMessageSize)))
dOpts = append(dOpts, grpc.WithUserAgent(fmt.Sprintf("%s/%s", common.ArgoCDUserAgentName, common.GetVersion().Version)))
dOpts = append(dOpts, grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()))
dOpts = append(dOpts, grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()))
if a.useTLS() {
// The following sets up the dial Options for grpc-gateway to talk to gRPC server over TLS.
// grpc-gateway is just translating HTTP/HTTPS requests as gRPC requests over localhost,
// so we need to supply the same certificates to establish the connections that a normal,
// external gRPC client would need.
tlsConfig := a.settings.TLSConfig()
if a.TLSConfigCustomizer != nil {
a.TLSConfigCustomizer(tlsConfig)
}
tlsConfig.InsecureSkipVerify = true
dCreds := credentials.NewTLS(tlsConfig)
dOpts = append(dOpts, grpc.WithTransportCredentials(dCreds))
} else {
dOpts = append(dOpts, grpc.WithTransportCredentials(insecure.NewCredentials()))
}
conn, err := grpc.Dial(fmt.Sprintf("localhost:%d", a.ListenPort), dOpts...)
if err != nil {
io.Close(mainLn)
io.Close(metricsLn)
return nil, err
}
return &Listeners{Main: mainLn, Metrics: metricsLn, GatewayConn: conn}, nil
}
// Init starts informers used by the API server
func (a *ArgoCDServer) Init(ctx context.Context) {
go a.projInformer.Run(ctx.Done())
go a.appInformer.Run(ctx.Done())
}
// Run runs the API Server
// We use k8s.io/code-generator/cmd/go-to-protobuf to generate the .proto files from the API types.
// k8s.io/ go-to-protobuf uses protoc-gen-gogo, which comes from gogo/protobuf (a fork of
// golang/protobuf).
func (a *ArgoCDServer) Run(ctx context.Context, port int, metricsPort int) {
func (a *ArgoCDServer) Run(ctx context.Context, listeners *Listeners) {
a.userStateStorage.Init(ctx)
grpcS, appResourceTreeFn := a.newGRPCServer()
@@ -294,10 +382,10 @@ func (a *ArgoCDServer) Run(ctx context.Context, port int, metricsPort int) {
var httpS *http.Server
var httpsS *http.Server
if a.useTLS() {
httpS = newRedirectServer(port, a.RootPath)
httpsS = a.newHTTPServer(ctx, port, grpcWebS, appResourceTreeFn)
httpS = newRedirectServer(a.ListenPort, a.RootPath)
httpsS = a.newHTTPServer(ctx, a.ListenPort, grpcWebS, appResourceTreeFn, listeners.GatewayConn)
} else {
httpS = a.newHTTPServer(ctx, port, grpcWebS, appResourceTreeFn)
httpS = a.newHTTPServer(ctx, a.ListenPort, grpcWebS, appResourceTreeFn, listeners.GatewayConn)
}
if a.RootPath != "" {
httpS.Handler = withRootPath(httpS.Handler, a)
@@ -311,26 +399,13 @@ func (a *ArgoCDServer) Run(ctx context.Context, port int, metricsPort int) {
httpsS.Handler = &bug21955Workaround{handler: httpsS.Handler}
}
metricsServ := metrics.NewMetricsServer(a.ListenHost, metricsPort)
metricsServ := metrics.NewMetricsServer(a.ListenHost, a.MetricsPort)
if a.RedisClient != nil {
cacheutil.CollectMetrics(a.RedisClient, metricsServ)
}
// Start listener
var conn net.Listener
var realErr error
_ = wait.ExponentialBackoff(backoff, func() (bool, error) {
conn, realErr = net.Listen("tcp", fmt.Sprintf("%s:%d", a.ListenHost, port))
if realErr != nil {
a.log.Warnf("failed listen: %v", realErr)
return false, nil
}
return true, nil
})
errors.CheckError(realErr)
// CMux is used to support servicing gRPC and HTTP1.1+JSON on the same port
tcpm := cmux.New(conn)
tcpm := cmux.New(listeners.Main)
var tlsm cmux.CMux
var grpcL net.Listener
var httpL net.Listener
@@ -360,10 +435,7 @@ func (a *ArgoCDServer) Run(ctx context.Context, port int, metricsPort int) {
// Start the muxed listeners for our servers
log.Infof("argocd %s serving on port %d (url: %s, tls: %v, namespace: %s, sso: %v)",
common.GetVersion(), port, a.settings.URL, a.useTLS(), a.Namespace, a.settings.IsSSOConfigured())
go a.projInformer.Run(ctx.Done())
go a.appInformer.Run(ctx.Done())
common.GetVersion(), a.ListenPort, a.settings.URL, a.useTLS(), a.Namespace, a.settings.IsSSOConfigured())
go func() { a.checkServeErr("grpcS", grpcS.Serve(grpcL)) }()
go func() { a.checkServeErr("httpS", httpS.Serve(httpL)) }()
@@ -374,17 +446,13 @@ func (a *ArgoCDServer) Run(ctx context.Context, port int, metricsPort int) {
go a.watchSettings()
go a.rbacPolicyLoader(ctx)
go func() { a.checkServeErr("tcpm", tcpm.Serve()) }()
go func() { a.checkServeErr("metrics", metricsServ.ListenAndServe()) }()
go func() { a.checkServeErr("metrics", metricsServ.Serve(listeners.Metrics)) }()
if !cache.WaitForCacheSync(ctx.Done(), a.projInformer.HasSynced, a.appInformer.HasSynced) {
log.Fatal("Timed out waiting for project cache to sync")
}
a.stopCh = make(chan struct{})
<-a.stopCh
errors.CheckError(conn.Close())
if err := metricsServ.Shutdown(ctx); err != nil {
log.Fatalf("Failed to gracefully shutdown metrics server: %v", err)
}
}
func (a *ArgoCDServer) Initialized() bool {
@@ -541,6 +609,11 @@ func (a *ArgoCDServer) newGRPCServer() (*grpc.Server, application.AppResourceTre
grpc.MaxRecvMsgSize(apiclient.MaxGRPCMessageSize),
grpc.MaxSendMsgSize(apiclient.MaxGRPCMessageSize),
grpc.ConnectionTimeout(300 * time.Second),
grpc.KeepaliveEnforcementPolicy(
keepalive.EnforcementPolicy{
MinTime: common.GRPCKeepAliveEnforcementMinimum,
},
),
}
sensitiveMethods := map[string]bool{
"/cluster.ClusterService/Create": true,
@@ -702,7 +775,7 @@ func compressHandler(handler http.Handler) http.Handler {
// newHTTPServer returns the HTTP server to serve HTTP/HTTPS requests. This is implemented
// using grpc-gateway as a proxy to the gRPC server.
func (a *ArgoCDServer) newHTTPServer(ctx context.Context, port int, grpcWebHandler http.Handler, appResourceTreeFn application.AppResourceTreeFn) *http.Server {
func (a *ArgoCDServer) newHTTPServer(ctx context.Context, port int, grpcWebHandler http.Handler, appResourceTreeFn application.AppResourceTreeFn, conn *grpc.ClientConn) *http.Server {
endpoint := fmt.Sprintf("localhost:%d", port)
mux := http.NewServeMux()
httpS := http.Server{
@@ -718,26 +791,6 @@ func (a *ArgoCDServer) newHTTPServer(ctx context.Context, port int, grpcWebHandl
},
},
}
var dOpts []grpc.DialOption
dOpts = append(dOpts, grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(apiclient.MaxGRPCMessageSize)))
dOpts = append(dOpts, grpc.WithUserAgent(fmt.Sprintf("%s/%s", common.ArgoCDUserAgentName, common.GetVersion().Version)))
dOpts = append(dOpts, grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()))
dOpts = append(dOpts, grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()))
if a.useTLS() {
// The following sets up the dial Options for grpc-gateway to talk to gRPC server over TLS.
// grpc-gateway is just translating HTTP/HTTPS requests as gRPC requests over localhost,
// so we need to supply the same certificates to establish the connections that a normal,
// external gRPC client would need.
tlsConfig := a.settings.TLSConfig()
if a.TLSConfigCustomizer != nil {
a.TLSConfigCustomizer(tlsConfig)
}
tlsConfig.InsecureSkipVerify = true
dCreds := credentials.NewTLS(tlsConfig)
dOpts = append(dOpts, grpc.WithTransportCredentials(dCreds))
} else {
dOpts = append(dOpts, grpc.WithTransportCredentials(insecure.NewCredentials()))
}
// HTTP 1.1+JSON Server
// grpc-ecosystem/grpc-gateway is used to proxy HTTP requests to the corresponding gRPC call
@@ -755,7 +808,7 @@ func (a *ArgoCDServer) newHTTPServer(ctx context.Context, port int, grpcWebHandl
}
mux.Handle("/api/", handler)
terminalHandler := application.NewHandler(a.appLister, a.db, a.enf, a.Cache, appResourceTreeFn)
terminalHandler := application.NewHandler(a.appLister, a.db, a.enf, a.Cache, appResourceTreeFn, a.settings.ExecShells)
mux.HandleFunc("/terminal", func(writer http.ResponseWriter, request *http.Request) {
argocdSettings, err := a.settingsMgr.GetSettings()
if err != nil {
@@ -790,17 +843,17 @@ func (a *ArgoCDServer) newHTTPServer(ctx context.Context, port int, grpcWebHandl
terminalHandler.ServeHTTP(writer, request)
})
mustRegisterGWHandler(versionpkg.RegisterVersionServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(clusterpkg.RegisterClusterServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(applicationpkg.RegisterApplicationServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(repositorypkg.RegisterRepositoryServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(repocredspkg.RegisterRepoCredsServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(sessionpkg.RegisterSessionServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(settingspkg.RegisterSettingsServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(projectpkg.RegisterProjectServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(accountpkg.RegisterAccountServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(certificatepkg.RegisterCertificateServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(gpgkeypkg.RegisterGPGKeyServiceHandlerFromEndpoint, ctx, gwmux, endpoint, dOpts)
mustRegisterGWHandler(versionpkg.RegisterVersionServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(clusterpkg.RegisterClusterServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(applicationpkg.RegisterApplicationServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(repositorypkg.RegisterRepositoryServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(repocredspkg.RegisterRepoCredsServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(sessionpkg.RegisterSessionServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(settingspkg.RegisterSettingsServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(projectpkg.RegisterProjectServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(accountpkg.RegisterAccountServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(certificatepkg.RegisterCertificateServiceHandler, ctx, gwmux, conn)
mustRegisterGWHandler(gpgkeypkg.RegisterGPGKeyServiceHandler, ctx, gwmux, conn)
// Swagger UI
swagger.ServeSwaggerUI(mux, assets.SwaggerJSON, "/swagger-ui", a.RootPath)
@@ -893,7 +946,7 @@ func (s *ArgoCDServer) getIndexData() ([]byte, error) {
if s.BaseHRef == "/" || s.BaseHRef == "" {
s.indexData = data
} else {
s.indexData = []byte(baseHRefRegex.ReplaceAllString(string(data), fmt.Sprintf(`<base href="/%s/">`, strings.Trim(s.BaseHRef, "/"))))
s.indexData = []byte(replaceBaseHRef(string(data), fmt.Sprintf(`<base href="/%s/">`, strings.Trim(s.BaseHRef, "/"))))
}
})
@@ -968,16 +1021,20 @@ func isMainJsBundle(url *url.URL) bool {
return mainJsBundleRegex.Match([]byte(filename))
}
type registerFunc func(ctx context.Context, mux *runtime.ServeMux, endpoint string, opts []grpc.DialOption) error
type registerFunc func(ctx context.Context, mux *runtime.ServeMux, conn *grpc.ClientConn) error
// mustRegisterGWHandler is a convenience function to register a gateway handler
func mustRegisterGWHandler(register registerFunc, ctx context.Context, mux *runtime.ServeMux, endpoint string, opts []grpc.DialOption) {
err := register(ctx, mux, endpoint, opts)
func mustRegisterGWHandler(register registerFunc, ctx context.Context, mux *runtime.ServeMux, conn *grpc.ClientConn) {
err := register(ctx, mux, conn)
if err != nil {
panic(err)
}
}
func replaceBaseHRef(data string, replaceWith string) string {
return baseHRefRegex.ReplaceAllString(data, replaceWith)
}
// Authenticate checks for the presence of a valid token when accessing server-side resources.
func (a *ArgoCDServer) Authenticate(ctx context.Context) (context.Context, error) {
if a.DisableAuth {
@@ -1010,6 +1067,7 @@ func (a *ArgoCDServer) Authenticate(ctx context.Context) (context.Context, error
if !argoCDSettings.AnonymousUserEnabled {
return ctx, claimsErr
} else {
// nolint:staticcheck
ctx = context.WithValue(ctx, "claims", "")
}
}

View File

@@ -29,20 +29,17 @@ func TestUserAgent(t *testing.T) {
s, closer := fakeServer()
defer closer()
lns, err := s.Listen()
assert.NoError(t, err)
cancelInformer := test.StartInformer(s.projInformer)
defer cancelInformer()
port, err := test.GetFreePort()
assert.NoError(t, err)
metricsPort, err := test.GetFreePort()
assert.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go s.Run(ctx, port, metricsPort)
s.Init(ctx)
go s.Run(ctx, lns)
defer func() { time.Sleep(3 * time.Second) }()
err = test.WaitForPortListen(fmt.Sprintf("127.0.0.1:%d", port), 10*time.Second)
assert.NoError(t, err)
type testData struct {
userAgent string
errorMsg string
@@ -72,7 +69,7 @@ func TestUserAgent(t *testing.T) {
for _, test := range tests {
opts := apiclient.ClientOptions{
ServerAddr: fmt.Sprintf("localhost:%d", port),
ServerAddr: fmt.Sprintf("localhost:%d", s.ListenPort),
PlainText: true,
UserAgent: test.userAgent,
}
@@ -99,25 +96,21 @@ func Test_StaticHeaders(t *testing.T) {
{
s, closer := fakeServer()
defer closer()
lns, err := s.Listen()
assert.NoError(t, err)
cancelInformer := test.StartInformer(s.projInformer)
defer cancelInformer()
port, err := test.GetFreePort()
assert.NoError(t, err)
metricsPort, err := test.GetFreePort()
assert.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go s.Run(ctx, port, metricsPort)
s.Init(ctx)
go s.Run(ctx, lns)
defer func() { time.Sleep(3 * time.Second) }()
err = test.WaitForPortListen(fmt.Sprintf("127.0.0.1:%d", port), 10*time.Second)
assert.NoError(t, err)
// Allow server startup
time.Sleep(1 * time.Second)
client := http.Client{}
url := fmt.Sprintf("http://127.0.0.1:%d/test.html", port)
url := fmt.Sprintf("http://127.0.0.1:%d/test.html", s.ListenPort)
req, err := http.NewRequest("GET", url, nil)
assert.NoError(t, err)
resp, err := client.Do(req)
@@ -134,23 +127,19 @@ func Test_StaticHeaders(t *testing.T) {
s.ContentSecurityPolicy = "frame-ancestors 'none';"
cancelInformer := test.StartInformer(s.projInformer)
defer cancelInformer()
port, err := test.GetFreePort()
assert.NoError(t, err)
metricsPort, err := test.GetFreePort()
lns, err := s.Listen()
assert.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go s.Run(ctx, port, metricsPort)
s.Init(ctx)
go s.Run(ctx, lns)
defer func() { time.Sleep(3 * time.Second) }()
err = test.WaitForPortListen(fmt.Sprintf("127.0.0.1:%d", port), 10*time.Second)
assert.NoError(t, err)
// Allow server startup
time.Sleep(1 * time.Second)
client := http.Client{}
url := fmt.Sprintf("http://127.0.0.1:%d/test.html", port)
url := fmt.Sprintf("http://127.0.0.1:%d/test.html", s.ListenPort)
req, err := http.NewRequest("GET", url, nil)
assert.NoError(t, err)
resp, err := client.Do(req)
@@ -167,23 +156,22 @@ func Test_StaticHeaders(t *testing.T) {
s.ContentSecurityPolicy = ""
cancelInformer := test.StartInformer(s.projInformer)
defer cancelInformer()
port, err := test.GetFreePort()
assert.NoError(t, err)
metricsPort, err := test.GetFreePort()
lns, err := s.Listen()
assert.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go s.Run(ctx, port, metricsPort)
s.Init(ctx)
go s.Run(ctx, lns)
defer func() { time.Sleep(3 * time.Second) }()
err = test.WaitForPortListen(fmt.Sprintf("127.0.0.1:%d", port), 10*time.Second)
err = test.WaitForPortListen(fmt.Sprintf("127.0.0.1:%d", s.ListenPort), 10*time.Second)
assert.NoError(t, err)
// Allow server startup
time.Sleep(1 * time.Second)
client := http.Client{}
url := fmt.Sprintf("http://127.0.0.1:%d/test.html", port)
url := fmt.Sprintf("http://127.0.0.1:%d/test.html", s.ListenPort)
req, err := http.NewRequest("GET", url, nil)
assert.NoError(t, err)
resp, err := client.Do(req)

View File

@@ -41,8 +41,13 @@ func fakeServer() (*ArgoCDServer, func()) {
kubeclientset := fake.NewSimpleClientset(cm, secret)
appClientSet := apps.NewSimpleClientset()
redis, closer := test.NewInMemoryRedis()
port, err := test.GetFreePort()
if err != nil {
panic(err)
}
argoCDOpts := ArgoCDServerOpts{
ListenPort: port,
Namespace: test.FakeArgoCDNamespace,
KubeClientset: kubeclientset,
AppClientset: appClientSet,
@@ -61,7 +66,8 @@ func fakeServer() (*ArgoCDServer, func()) {
),
RedisClient: redis,
}
return NewServer(context.Background(), argoCDOpts), closer
srv := NewServer(context.Background(), argoCDOpts)
return srv, closer
}
func TestEnforceProjectToken(t *testing.T) {
@@ -508,7 +514,7 @@ func getTestServer(t *testing.T, anonymousEnabled bool, withFakeSSO bool) (argoc
cm.Data["users.anonymous.enabled"] = "true"
}
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
return // Start with a placeholder. We need the server URL before setting up the real handler.
// Start with a placeholder. We need the server URL before setting up the real handler.
}))
ts.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
dexMockHandler(t, ts.URL)(w, r)
@@ -583,7 +589,7 @@ func TestAuthenticate_3rd_party_JWTs(t *testing.T) {
anonymousEnabled: false,
claims: jwt.RegisteredClaims{Audience: jwt.ClaimStrings{"test-client"}, Subject: "admin", ExpiresAt: jwt.NewNumericDate(time.Now())},
expectedErrorContains: "token is expired",
expectedClaims: jwt.RegisteredClaims{Issuer:"sso"},
expectedClaims: jwt.RegisteredClaims{Issuer: "sso"},
},
{
test: "anonymous enabled, expired token, admin claim",
@@ -601,7 +607,7 @@ func TestAuthenticate_3rd_party_JWTs(t *testing.T) {
t.Parallel()
// Must be declared here to avoid race.
ctx := context.Background() //nolint:ineffassign,staticcheck
ctx := context.Background() //nolint:ineffassign,staticcheck
argocd, dexURL := getTestServer(t, testDataCopy.anonymousEnabled, true)
testDataCopy.claims.Issuer = fmt.Sprintf("%s/api/dex", dexURL)
@@ -698,7 +704,7 @@ func TestAuthenticate_no_SSO(t *testing.T) {
t.Parallel()
// Must be declared here to avoid race.
ctx := context.Background() //nolint:ineffassign,staticcheck
ctx := context.Background() //nolint:ineffassign,staticcheck
argocd, dexURL := getTestServer(t, testDataCopy.anonymousEnabled, false)
token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.RegisteredClaims{Issuer: fmt.Sprintf("%s/api/dex", dexURL)})
@@ -806,7 +812,7 @@ func TestAuthenticate_bad_request_metadata(t *testing.T) {
t.Parallel()
// Must be declared here to avoid race.
ctx := context.Background() //nolint:ineffassign,staticcheck
ctx := context.Background() //nolint:ineffassign,staticcheck
argocd, _ := getTestServer(t, testDataCopy.anonymousEnabled, true)
ctx = metadata.NewIncomingContext(context.Background(), testDataCopy.metadata)
@@ -1065,39 +1071,39 @@ func TestOIDCConfigChangeDetection_NoChange(t *testing.T) {
}
func TestIsMainJsBundle(t *testing.T) {
testCases := []struct{
testCases := []struct {
name string
url string
isMainJsBundle bool
}{
{
name: "localhost with valid main bundle",
url: "https://localhost:8080/main.e4188e5adc97bbfc00c3.js",
name: "localhost with valid main bundle",
url: "https://localhost:8080/main.e4188e5adc97bbfc00c3.js",
isMainJsBundle: true,
},
{
name: "localhost and deep path with valid main bundle",
url: "https://localhost:8080/some/argo-cd-instance/main.e4188e5adc97bbfc00c3.js",
name: "localhost and deep path with valid main bundle",
url: "https://localhost:8080/some/argo-cd-instance/main.e4188e5adc97bbfc00c3.js",
isMainJsBundle: true,
},
{
name: "font file",
url: "https://localhost:8080/assets/fonts/google-fonts/Heebo-Bols.woff2",
name: "font file",
url: "https://localhost:8080/assets/fonts/google-fonts/Heebo-Bols.woff2",
isMainJsBundle: false,
},
{
name: "no dot after main",
url: "https://localhost:8080/main/e4188e5adc97bbfc00c3.js",
name: "no dot after main",
url: "https://localhost:8080/main/e4188e5adc97bbfc00c3.js",
isMainJsBundle: false,
},
{
name: "wrong extension character",
url: "https://localhost:8080/main.e4188e5adc97bbfc00c3/js",
name: "wrong extension character",
url: "https://localhost:8080/main.e4188e5adc97bbfc00c3/js",
isMainJsBundle: false,
},
{
name: "wrong hash length",
url: "https://localhost:8080/main.e4188e5adc97bbfc00c3abcdefg.js",
name: "wrong hash length",
url: "https://localhost:8080/main.e4188e5adc97bbfc00c3abcdefg.js",
isMainJsBundle: false,
},
}
@@ -1111,3 +1117,123 @@ func TestIsMainJsBundle(t *testing.T) {
})
}
}
func TestReplaceBaseHRef(t *testing.T) {
testCases := []struct {
name string
data string
expected string
replaceWith string
}{
{
name: "non-root basepath",
data: `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Argo CD</title>
<base href="/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel='icon' type='image/png' href='assets/favicon/favicon-32x32.png' sizes='32x32'/>
<link rel='icon' type='image/png' href='assets/favicon/favicon-16x16.png' sizes='16x16'/>
<link href="assets/fonts.css" rel="stylesheet">
</head>
<body>
<noscript>
<p>
Your browser does not support JavaScript. Please enable JavaScript to view the site.
Alternatively, Argo CD can be used with the <a href="https://argoproj.github.io/argo-cd/cli_installation/">Argo CD CLI</a>.
</p>
</noscript>
<div id="app"></div>
</body>
</html>`,
expected: `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Argo CD</title>
<base href="/path1/path2/path3/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel='icon' type='image/png' href='assets/favicon/favicon-32x32.png' sizes='32x32'/>
<link rel='icon' type='image/png' href='assets/favicon/favicon-16x16.png' sizes='16x16'/>
<link href="assets/fonts.css" rel="stylesheet">
</head>
<body>
<noscript>
<p>
Your browser does not support JavaScript. Please enable JavaScript to view the site.
Alternatively, Argo CD can be used with the <a href="https://argoproj.github.io/argo-cd/cli_installation/">Argo CD CLI</a>.
</p>
</noscript>
<div id="app"></div>
</body>
</html>`,
replaceWith: `<base href="/path1/path2/path3/">`,
},
{
name: "root basepath",
data: `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Argo CD</title>
<base href="/any/path/test/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel='icon' type='image/png' href='assets/favicon/favicon-32x32.png' sizes='32x32'/>
<link rel='icon' type='image/png' href='assets/favicon/favicon-16x16.png' sizes='16x16'/>
<link href="assets/fonts.css" rel="stylesheet">
</head>
<body>
<noscript>
<p>
Your browser does not support JavaScript. Please enable JavaScript to view the site.
Alternatively, Argo CD can be used with the <a href="https://argoproj.github.io/argo-cd/cli_installation/">Argo CD CLI</a>.
</p>
</noscript>
<div id="app"></div>
</body>
</html>`,
expected: `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Argo CD</title>
<base href="/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel='icon' type='image/png' href='assets/favicon/favicon-32x32.png' sizes='32x32'/>
<link rel='icon' type='image/png' href='assets/favicon/favicon-16x16.png' sizes='16x16'/>
<link href="assets/fonts.css" rel="stylesheet">
</head>
<body>
<noscript>
<p>
Your browser does not support JavaScript. Please enable JavaScript to view the site.
Alternatively, Argo CD can be used with the <a href="https://argoproj.github.io/argo-cd/cli_installation/">Argo CD CLI</a>.
</p>
</noscript>
<div id="app"></div>
</body>
</html>`,
replaceWith: `<base href="/">`,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
result := replaceBaseHRef(testCase.data, testCase.replaceWith)
assert.Equal(t, testCase.expected, result)
})
}
}

View File

@@ -38,6 +38,7 @@ import (
projectFixture "github.com/argoproj/argo-cd/v2/test/e2e/fixture/project"
repoFixture "github.com/argoproj/argo-cd/v2/test/e2e/fixture/repos"
"github.com/argoproj/argo-cd/v2/test/e2e/testdata"
"github.com/argoproj/argo-cd/v2/util/argo"
. "github.com/argoproj/argo-cd/v2/util/argo"
. "github.com/argoproj/argo-cd/v2/util/errors"
"github.com/argoproj/argo-cd/v2/util/io"
@@ -947,7 +948,7 @@ func TestLocalManifestSync(t *testing.T) {
And(func(app *Application) {
res, _ := RunCli("app", "manifests", app.Name)
assert.Contains(t, res, "containerPort: 80")
assert.Contains(t, res, "image: gcr.io/heptio-images/ks-guestbook-demo:0.2")
assert.Contains(t, res, "image: quay.io/argoprojlabs/argocd-e2e-container:0.2")
}).
Given().
LocalPath(guestbookPathLocal).
@@ -958,7 +959,7 @@ func TestLocalManifestSync(t *testing.T) {
And(func(app *Application) {
res, _ := RunCli("app", "manifests", app.Name)
assert.Contains(t, res, "containerPort: 81")
assert.Contains(t, res, "image: gcr.io/heptio-images/ks-guestbook-demo:0.3")
assert.Contains(t, res, "image: quay.io/argoprojlabs/argocd-e2e-container:0.3")
}).
Given().
LocalPath("").
@@ -969,7 +970,7 @@ func TestLocalManifestSync(t *testing.T) {
And(func(app *Application) {
res, _ := RunCli("app", "manifests", app.Name)
assert.Contains(t, res, "containerPort: 80")
assert.Contains(t, res, "image: gcr.io/heptio-images/ks-guestbook-demo:0.2")
assert.Contains(t, res, "image: quay.io/argoprojlabs/argocd-e2e-container:0.2")
})
}
@@ -1190,7 +1191,8 @@ func TestPermissionWithScopedRepo(t *testing.T) {
Name(projName).
Destination("*,*").
When().
Create()
Create().
AddSource("*")
repoFixture.Given(t, true).
When().
@@ -2063,3 +2065,241 @@ func TestDisableManifestGeneration(t *testing.T) {
assert.Equal(t, app.Status.SourceType, ApplicationSourceTypeDirectory)
})
}
func TestSwitchTrackingMethod(t *testing.T) {
ctx := Given(t)
ctx.
SetTrackingMethod(string(argo.TrackingMethodAnnotation)).
Path("deployment").
When().
CreateApp().
Sync().
Refresh(RefreshTypeNormal).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add resource with tracking annotation. This should put the
// application OutOfSync.
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "other-configmap",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: fmt.Sprintf("%s:/ConfigMap:%s/other-configmap", Name(), DeploymentNamespace()),
},
},
}, metav1.CreateOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeOutOfSync)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Delete resource to bring application back in sync
FailOnErr(nil, KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Delete(context.Background(), "other-configmap", metav1.DeleteOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
SetTrackingMethod(string(argo.TrackingMethodLabel)).
Sync().
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add a resource with a tracking annotation. This should not
// affect the application, because we now use the tracking method
// "label".
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "other-configmap",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: fmt.Sprintf("%s:/ConfigMap:%s/other-configmap", Name(), DeploymentNamespace()),
},
},
}, metav1.CreateOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add a resource with the tracking label. The app should become
// OutOfSync.
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "extra-configmap",
Labels: map[string]string{
common.LabelKeyAppInstance: Name(),
},
},
}, metav1.CreateOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeOutOfSync)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Delete resource to bring application back in sync
FailOnErr(nil, KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Delete(context.Background(), "extra-configmap", metav1.DeleteOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy))
}
func TestSwitchTrackingLabel(t *testing.T) {
ctx := Given(t)
ctx.
Path("deployment").
When().
CreateApp().
Sync().
Refresh(RefreshTypeNormal).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add extra resource that carries the default tracking label
// We expect the app to go out of sync.
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "other-configmap",
Labels: map[string]string{
common.LabelKeyAppInstance: Name(),
},
},
}, metav1.CreateOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeOutOfSync)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Delete resource to bring application back in sync
FailOnErr(nil, KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Delete(context.Background(), "other-configmap", metav1.DeleteOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
// Change tracking label
SetTrackingLabel("argocd.tracking").
Sync().
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Create resource with the new tracking label, the application
// is expected to go out of sync
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "other-configmap",
Labels: map[string]string{
"argocd.tracking": Name(),
},
},
}, metav1.CreateOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeOutOfSync)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Delete resource to bring application back in sync
FailOnErr(nil, KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Delete(context.Background(), "other-configmap", metav1.DeleteOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add extra resource that carries the default tracking label
// We expect the app to stay in sync, because the configured
// label is different.
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "other-configmap",
Labels: map[string]string{
common.LabelKeyAppInstance: Name(),
},
},
}, metav1.CreateOptions{}))
}).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy))
}
func TestAnnotationTrackingExtraResources(t *testing.T) {
ctx := Given(t)
SetTrackingMethod(string(argo.TrackingMethodAnnotation))
ctx.
Path("deployment").
When().
CreateApp().
Sync().
Refresh(RefreshTypeNormal).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add a resource with an annotation that is not referencing the
// resource.
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "extra-configmap",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: fmt.Sprintf("%s:apps/Deployment:%s/guestbook-cm", Name(), DeploymentNamespace()),
},
},
}, metav1.CreateOptions{}))
}).
Refresh(RefreshTypeNormal).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
Expect(HealthIs(health.HealthStatusHealthy)).
When().
And(func() {
// Add a resource with an annotation that is self-referencing the
// resource.
FailOnErr(KubeClientset.CoreV1().ConfigMaps(DeploymentNamespace()).Create(context.Background(), &v1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "other-configmap",
Annotations: map[string]string{
common.AnnotationKeyAppInstance: fmt.Sprintf("%s:/ConfigMap:%s/other-configmap", Name(), DeploymentNamespace()),
},
},
}, metav1.CreateOptions{}))
}).
Refresh(RefreshTypeNormal).
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(SyncStatusIs(SyncStatusCodeOutOfSync)).
Expect(HealthIs(health.HealthStatusHealthy))
}

View File

@@ -22,7 +22,7 @@ metadata:
spec:
containers:
- name: main
image: alpine:3.10.2
image: quay.io/argoprojlabs/argocd-e2e-container:0.1
imagePullPolicy: IfNotPresent
command:
- "true"
@@ -43,7 +43,7 @@ metadata:
spec:
containers:
- name: main
image: alpine:3.10.2
image: quay.io/argoprojlabs/argocd-e2e-container:0.1
imagePullPolicy: IfNotPresent
command:
- "true"

Some files were not shown because too many files have changed in this diff Show More