PSS baseline / restricted also for Notebooks, Katib, Kserve, Dashboard and istio-ingressgateway

[juliusvonkohout]

Validation Checklist

• I confirm that this is a Kubeflow-related issue.
• I am reporting this in the appropriate repository.
• I have followed the Kubeflow installation guidelines.
• The issue report is detailed and includes version numbers where applicable.
• This issue pertains to Kubeflow development.
• I am available to work on this issue.
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

master

Detailed Description

It should be directly in the respecitve integration tests, not in the separate https://github.com/kubeflow/manifests/blob/b0fd125a26c3d64aaa17df7a41070a9767698886/.github/workflows/pss_test.yaml

Steps to Reproduce

https://github.com/kubeflow/manifests/blob/b0fd125a26c3d64aaa17df7a41070a9767698886/.github/workflows/pss_test.yaml

Screenshots or Videos (Optional)

No response

[akagami-harsh]

/assign

[ViciousEagle03]

Hello @akagami-harsh , I noticed you’re already working on a couple of issues, and since I’m still learning the codebase, I’d really appreciate the chance to work on this one. Would you mind unassigning it so I can give it a try, if you dont mind?
Thanks a ton!😊

[juliusvonkohout]

@ViciousEagle03 since there is already a good PR now, can you take a look at https://github.com/kubeflow/manifests/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22help%20wanted%22

Maybe you can investigate #2928

[juliusvonkohout]

Thank you for #3026, in a follow up PR
please

• add the namespace knative-serving to the PSS tests for kserve
• split and merge https://github.com/kubeflow/manifests/blob/master/.github/workflows/pss_test.yaml into https://github.com/kubeflow/manifests/blob/master/.github/workflows/pipeline_test.yaml and https://github.com/kubeflow/manifests/blob/master/.github/workflows/dex_test.yaml (dex and oauth2-proxy) and rename it also to dex_oauth2-proxy_test
• Actually i made a mistake with removing the "auth" namesace labels and PSS testing, becasue Dex is installed in "auth", so please add it back
• Try to restructure https://github.com/kubeflow/manifests/blob/master/.github/workflows/profiles_test.yaml
• Add the PSS tests also for https://github.com/kubeflow/manifests/blob/master/.github/workflows/pipeline_run_from_notebook.yaml at the end to see if the spawned jupyterlab is PSS compliant
• Warning: existing pods in namespace "istio-system" violate the new PodSecurity enforce level "restricted:latest" Warning: istio-ingressgateway-65f4848f75-kc7k8: seccompProfile

We also need to add securitycontexts here in kubeflow/manifests and the upstream repositories (kubeflow/katib, kubeflow/notebooks, kserve) to fix the warnings from the tests.

# notebooks
 Patching the PSS-restricted labels for namespace istio-system...
Warning: existing pods in namespace "istio-system" violate the new PodSecurity enforce level "restricted:latest"
Warning: istio-ingressgateway-65f4848f75-ssj9x: seccompProfile
namespace/istio-system patched
Patching the PSS-restricted labels for namespace kubeflow...
Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
namespace/kubeflow patched
Warning: notebook-controller-deployment-68b6c4855c-nn5pv: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile

# kserve
Patching the PSS-restricted labels for namespace istio-system...
Warning: existing pods in namespace "istio-system" violate the new PodSecurity enforce level "restricted:latest"
Warning: cluster-local-gateway-6b77b659d5-swrtm (and 1 other pod): seccompProfile
namespace/istio-system patched
Patching the PSS-restricted labels for namespace cert-manager...
namespace/cert-manager patched
Patching the PSS-restricted labels for namespace kubeflow...
Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
namespace/kubeflow patched
Warning: kserve-controller-manager-7cfcfd6d6b-7rhcq: seccompProfile
Warning: kserve-localmodel-controller-manager-655ccdf64-xn89g: unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile
Warning: kserve-models-web-app-5f5dfb549c-b8929: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile

# katib
Patching the PSS-restricted labels for namespace istio-system...
Warning: existing pods in namespace "istio-system" violate the new PodSecurity enforce level "restricted:latest"
namespace/istio-system patched
Warning: istio-ingressgateway-65f4848f75-j7wcn: seccompProfile
Patching the PSS-restricted labels for namespace cert-manager...
namespace/cert-manager patched
Patching the PSS-restricted labels for namespace kubeflow...
Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
Warning: katib-controller-7868c788bc-lw9fz (and 3 other pods): allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/kubeflow patched

The seccomp profile for istio is related to #2958

Update

manifests/.github/workflows/pss_test.yaml

Lines 39 to 47 in 8d69c8d

	run: |
	kubectl get cm istio-sidecar-injector -n istio-system -o yaml > temporary_patch.yaml
	sed -i '0,/runAsNonRoot: true/{s//&\n seccompProfile:\n type: RuntimeDefault/}' temporary_patch.yaml
	sed -i '/runAsNonRoot: true/{N; /runAsUser: {{ .ProxyUID | default "1337" }}/a\
	seccompProfile:\n type: RuntimeDefault
	}' temporary_patch.yaml
	kubectl apply -f temporary_patch.yaml
	rm temporary_patch.yaml

so the sidecar seccomp profile still needs to be integrated directly into istio and https://github.com/kubeflow/manifests/blob/master/.github/workflows/pss_test.yaml must be merged into https://github.com/kubeflow/manifests/blob/master/.github/workflows/pipeline_test.yaml. We should use the new "Values.gateways.seccompProfile" and securitycontext to get rid of the manual patch at

manifests/.github/workflows/pss_test.yaml

Lines 35 to 45 in 26d970d

	- name: Configure istio init container with seccompProfile attribute
	run: |
	kubectl get cm istio-sidecar-injector -n istio-system -o yaml > temporary_patch.yaml
	sed -i '0,/runAsNonRoot: true/{s//&\n seccompProfile:\n type: RuntimeDefault/}' temporary_patch.yaml
	sed -i '/runAsNonRoot: true/{N; /runAsUser: {{ .ProxyUID | default "1337" }}/a\
	seccompProfile:\n type: RuntimeDefault
	}' temporary_patch.yaml
	kubectl apply -f temporary_patch.yaml
	rm temporary_patch.yaml

.

[juliusvonkohout]

After merging #3042

I see

Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
namespace/kubeflow patched
Warning: centraldashboard-5796446d58-z4dzv: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile
Warning: profiles-deployment-779878fd4d-qgbpk: seccompProfile

Patching the PSS-restricted labels for namespace kubeflow...
Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
Warning: kserve-controller-manager-7cfcfd6d6b-8lswm: seccompProfile
Warning: kserve-localmodel-controller-manager-655ccdf64-5tzqb: unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile
Warning: kserve-models-web-app-5f5dfb549c-x6jpz: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile
namespace/kubeflow patched

 Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
namespace/kubeflow patched
Warning: notebook-controller-deployment-68b6c4855c-mw69k: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile

Patching the PSS-restricted labels for namespace kubeflow...
Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
Warning: cache-server-74475fd484-vrknf (and 12 other pods): unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile
namespace/kubeflow patched


Patching the PSS-restricted labels for namespace kubeflow...
Warning: existing pods in namespace "kubeflow" violate the new PodSecurity enforce level "restricted:latest"
Warning: admission-webhook-deployment-799878b4bb-4z5g9: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/kubeflow patched
Warning: cache-server-74475fd484-7mnfc (and 12 other pods): unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile
Warning: jupyter-web-app-deployment-6d5d4885f5-7cpcg (and 1 other pod): allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile

for a follow up PR in kubeflow/manifests and multiple PRs in the upstream manifests repositories.

Somehow the already fixed cahce-server from kubeflow/pipelines is having trouble again.