Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(pipeline): minio filesystem permission access error #3045

Merged
merged 3 commits into from
Mar 10, 2025
Merged

fix(pipeline): minio filesystem permission access error #3045

merged 3 commits into from
Mar 10, 2025
+3 −0

Conversation

milinddethe15
Copy link
Contributor

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

Describe the changes you have made, including any refactoring or feature additions.

This change will will make Kubernetes automatically set the correct ownership of the mounted volumes to be accessible by user 1000

📦 Dependencies

List any dependencies or related PRs (e.g., "Depends on #123").

🐛 Related Issues

Link any issues that are resolved or affected by this PR.

fixes #3040

✅ Contributor Checklist

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

@google-oss-prow google-oss-prow bot requested review from gkcalat and zijianjoy March 8, 2025 21:28
@ericgtkb
Copy link

ericgtkb commented Mar 9, 2025 •
edited
Loading

Tried with this fgGroup: 1000 change on minikube, but getting the following error:

Error from server: failed to create typed patch object (kubeflow/minio; apps/v1, Kind=Deployment): .spec.template.spec.containers[name="minio"].securityContext.fsGroup: field not declared in schema

@ericgtkb
Copy link

ericgtkb commented Mar 9, 2025

Looks like it needs to be one level above .spec.template.spec.securityContext.fsGroup instead of .spec.template.spec.containers.securityContext.fsGroup. After this change minio starts without issues.

@juliusvonkohout
Copy link
Member

juliusvonkohout commented Mar 9, 2025 •
edited
Loading

Please check https://github.com/minio/minio/blob/11507d46da0c98af6066e76a58eab4e81bfb6d58/helm/minio/values.yaml#L276

So

Container level
securityContext:
  enabled: true
  runAsUser: 1000
  runAsGroup: 1000 # here I am not sure whether 1000 or 0

Pod level
securityContext:
  fsGroup: 1000
  fsGroupChangePolicy: "OnRootMismatch"

@juliusvonkohout
Update minio-deployment.yaml
322494b 
Signed-off-by: Julius von Kohout <[email protected]>
@juliusvonkohout
Copy link
Member

/ok-to-test

@juliusvonkohout
Update minio-deployment.yaml
6dd0a75 
Signed-off-by: Julius von Kohout <[email protected]>
@juliusvonkohout
Copy link
Member

We also must open a PR against kubeflow/pipelines

This was referenced Mar 9, 2025
Open
Closed
@juliusvonkohout
Copy link
Member

juliusvonkohout commented Mar 9, 2025 •
edited
Loading

@ericgtkb @fraenkel please test this branch on your clusters and report back on which exact clusters it worked. CC @tarekabouzeid for testing as well. AKS, EKS, GKE, RANCHER would be good.

@ericgtkb
Copy link

Just confirming that it works on minikube - minio pod starts without issues.

@juliusvonkohout
Copy link
Member

/lgtm
/approve

@google-oss-prow google-oss-prow bot added the lgtm label Mar 10, 2025
@google-oss-prow Google OSS Prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@juliusvonkohout
Copy link
Member

kubeflow/pipelines#11734 is the upstream PR.

@google-oss-prow google-oss-prow bot merged commit d345ef6 into kubeflow:master Mar 10, 2025
9 checks passed
@juliusvonkohout juliusvonkohout added this to the 1.10 milestone Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gkcalat gkcalat Awaiting requested review from gkcalat

@zijianjoy zijianjoy Awaiting requested review from zijianjoy

Assignees

@juliusvonkohout juliusvonkohout

Labels
approved lgtm ok-to-test size/XS
Projects
None yet
Milestone
1.10
Development

Successfully merging this pull request may close these issues.

Minio for pipeline no longer starts on some clusters
3 participants
@milinddethe15 @ericgtkb @juliusvonkohout