Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add changes to ensure azlinux and mcrypt images are working fine #2682

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Add changes to ensure azlinux and mcrypt images are working fine #2682

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3fc49af
Add changes to ensure azlinux and mcrypt images are working fine
nss10 Dec 10, 2024
9743055
Making changes to include poetry with a fallback to python
nss10 Dec 11, 2024
332c6fb
Change `|` to `||`
nss10 Dec 13, 2024
920e6ad
Edit the deploy yamls for fence
nss10 Jan 8, 2025
1701006
Merge branch 'master' into chore/fence_deploy_azlinux
nss10 Jan 9, 2025
da48642
Adding `poetry run python` for python for all jobs that use fence
nss10 Jan 13, 2025
625502e
Merge branch 'master' into chore/fence_deploy_azlinux
nss10 Jan 14, 2025
e87e92b
Merge branch 'master' into chore/fence_deploy_azlinux
nss10 Jan 16, 2025
83a44a1
Merge branch 'master' into chore/fence_deploy_azlinux
nss10 Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ repos:
hooks:
- id: detect-secrets
args: ['--baseline', '.secrets.baseline']
additional_dependencies: ['gibberish-detector']
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
Expand Down
53 changes: 1 addition & 52 deletions .secrets.baseline
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2412,57 +2412,6 @@
"line_number": 44
}
],
"kube/services/jobs/usersync-job.yaml": [
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "dbd5f43594a152b52261c8e21520a3989823fe55",
"is_verified": false,
"line_number": 64
},
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "1c062eaac9e6fa0766377d3cfc3e4a88982fecdb",
"is_verified": false,
"line_number": 67
},
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "694cfd0a009a42055e975de9111b2f3c6e8a3634",
"is_verified": false,
"line_number": 70
},
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "4b09a441cef18c75560f6c3caeafc96f2163c3fd",
"is_verified": false,
"line_number": 77
},
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "7e7478a28dcc3695a083b66b47243b050c813e2d",
"is_verified": false,
"line_number": 80
},
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "2f57bb00fcb93481c2be444e3e9f322b6cb5fadb",
"is_verified": false,
"line_number": 83
},
{
"type": "Secret Keyword",
"filename": "kube/services/jobs/usersync-job.yaml",
"hashed_secret": "ea73fcfdaa415890d5fde24d3b2245671be32f73",
"is_verified": false,
"line_number": 86
}
],
"kube/services/jobs/useryaml-job.yaml": [
{
"type": "Secret Keyword",
Expand Down Expand Up @@ -3253,5 +3202,5 @@
}
]
},
"generated_at": "2024-11-25T11:39:59Z"
"generated_at": "2024-12-10T19:30:47Z"
}
5 changes: 3 additions & 2 deletions kube/services/fence/fence-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,8 +225,9 @@ spec:
- "-c"
- |
echo "${FENCE_PUBLIC_CONFIG:-""}" > "/var/www/fence/fence-config-public.yaml"
python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-public.yaml /var/www/fence/fence-config-secret.yaml > /var/www/fence/fence-config.yaml
poetry run python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-public.yaml /var/www/fence/fence-config-secret.yaml > /var/www/fence/fence-config.yaml
bash /fence/dockerrun.bash && if [[ -f /dockerrun.sh ]]; then bash /dockerrun.sh; fi
poetry run gunicorn -c deployment/wsgi/gunicorn.conf.py
initContainers:
- name: fence-init
GEN3_FENCE_IMAGE
Expand Down Expand Up @@ -262,7 +263,7 @@ spec:
- "-c"
- |
echo "${FENCE_PUBLIC_CONFIG:-""}" > "/var/www/fence/fence-config-public.yaml"
python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-public.yaml /var/www/fence/fence-config-secret.yaml > /var/www/fence/fence-config.yaml
poetry run python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-public.yaml /var/www/fence/fence-config-secret.yaml > /var/www/fence/fence-config.yaml
if fence-create migrate --help > /dev/null 2>&1; then
if ! grep -E 'ENABLE_DB_MIGRATION"?: *false' /var/www/fence/fence-config.yaml; then
echo "Running db migration: fence-create migrate"
Expand Down
2 changes: 1 addition & 1 deletion kube/services/jobs/usersync-job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ spec:
name: "projects"
containers:
- name: usersync
GEN3_FENCE_IMAGE
GEN3_FENCE_MCRYPT_IMAGE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding this, enforces all commons to have fence_mcrypt in their manifest and doesn't seem like a great idea

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... It will force every job to use the new image for the usersync job regardless of if they're using the latest Fence, which is going to change what every Prod env runs without them explicitly updating a manifest or anything. We shouldn't do this. It doesn't allow someone to test the update in a staging/preprod before deploying in prod. We want to only use this new image if they're deploying the new Fence as configured in the manifest. Maybe we need a new manifest entry

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested adding a new fence_mcrypt image line in the manifest's versions, specifically for jobs where usersync requires this branch. The team would then need to manually update the usersync job in the AdminVM.

Need opinions.

imagePullPolicy: Always
env:
- name: prometheus_multiproc_dir
Expand Down
5 changes: 3 additions & 2 deletions kube/services/presigned-url-fence/presigned-url-fence-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,11 +220,12 @@ spec:
- |
echo "${FENCE_PUBLIC_CONFIG:-""}" > "/var/www/fence/fence-config-public.yaml"
echo -e "ENABLE_DB_MIGRATION: false" > "/var/www/fence/fence-config-bonus1.yaml"
python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-public.yaml /var/www/fence/fence-config-secret.yaml > /var/www/fence/fence-config-step1.yaml
python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-bonus1.yaml /var/www/fence/fence-config-step1.yaml > /var/www/fence/fence-config.yaml
poetry run python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-public.yaml /var/www/fence/fence-config-secret.yaml > /var/www/fence/fence-config-step1.yaml
poetry run python /var/www/fence/yaml_merge.py /var/www/fence/fence-config-bonus1.yaml /var/www/fence/fence-config-step1.yaml > /var/www/fence/fence-config.yaml
nginx_limit=$(cat /var/www/fence/fence-config.yaml | sed -n -e 's/^.*\"OVERRIDE_NGINX_RATE_LIMIT\": //p' | sed 's/.$//')
if [ -z "$nginx_limit" ]; then
nginx_limit=$(cat /fence/fence/config-default.yaml | sed -n -e 's/^.*OVERRIDE_NGINX_RATE_LIMIT: //p')
fi
export OVERRIDE_NGINX_RATE_LIMIT="$nginx_limit"
([[ ! -f /entrypoint.sh ]] || bash /entrypoint.sh) && /fence/dockerrun.bash && if [[ -f /dockerrun.sh ]]; then bash /dockerrun.sh; fi
poetry run gunicorn -c deployment/wsgi/gunicorn.conf.py
Loading