Add default container user, remove hard-coded UID/GID injection#917
Add default container user, remove hard-coded UID/GID injection#917wnevis-cmyk wants to merge 4 commits into
Conversation
Signed-off-by: Will Nevis <wnevis@coreweave.com>
cee24c7 to
af98303
Compare
Stop defaulting PodSecurityContext.FSGroup to constants.ContainerGID for the StatefulSet, backup Job, and restore Job. Restrictive admission controllers (e.g. OpenShift restricted-v2 SCC) assign a project-scoped fsGroup, and a hard-coded default can prevent admission. FSGroupChangePolicy=OnRootMismatch is retained. Signed-off-by: Will Nevis <wnevis@coreweave.com>
OpenShift forbids setting blockOwnerDeletion on an owner reference unless the caller has permission to set finalizers on the owner kind. The MOCO controller does not have (and should not require) /finalizers permission on MySQLCluster from the PVC owner-reference path, so on OpenShift the PVC creation fails with: persistentvolumeclaims is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on The PVC retains the owner reference (with Controller=true) so garbage collection still cascades when the MySQLCluster is deleted; only the blockOwnerDeletion flag is dropped. Refs: cybozu-go#389 Signed-off-by: Will Nevis <wnevis@coreweave.com>
|
@wnevis-cmyk |
|
@wnevis-cmyk
Since some of these are breaking changes, we'd like to proceed carefully. |
Address review feedback on PR cybozu-go#917: - Gate default runAsUser/runAsGroup and fsGroup injection behind a new --disable-default-security-context flag (defaults to existing behavior) so existing installs are unaffected and OpenShift users can opt in. - Restore WithBlockOwnerDeletion(true) on the PVC owner reference; the finalizer-based fix for cybozu-go#389 will be handled separately. - Keep the fluent-bit USER 10000:10000 directive.
Address review feedback on PR cybozu-go#917: - Gate default runAsUser/runAsGroup and fsGroup injection behind a new --disable-default-security-context flag (defaults to existing behavior) so existing installs are unaffected and OpenShift users can opt in. - Restore WithBlockOwnerDeletion(true) on the PVC owner reference; the finalizer-based fix for cybozu-go#389 will be handled separately. - Keep the fluent-bit USER 10000:10000 directive. Signed-off-by: Will Nevis <wnevis@coreweave.com>
e9ffd28 to
2baf81e
Compare
@yamatcha I've gone ahead and implemented that flag-based approach for default security context and removed the blockOwnerDeletion bit. It should be good to go -- thanks for the help on this. Also, yes, that RBAC role binding for the finalizers is a decent enough workaround in the meantime. Worth mentioning that I did do another local test on my CRC cluster and it seems to be working with the workaround from #389 |
Summary
This change makes MOCO compatible with restrictive Pod Security admission, in particular OpenShift's
restricted-v2SCC, by:USERso the operator does not need to inject UIDs.runAsUser/runAsGroup(on managed containers) andfsGroup(on managed pods) via a new command-line flag, so OpenShift can assign its project-scoped UID / GID / fsGroup ranges.Restrictive SCCs assign project-scoped UID / GID / fsGroup ranges and reject pods that pin those fields to a fixed value. By default MOCO still injects its historical UID/GID/fsGroup values, so existing installations are unaffected; OpenShift users enable the new flag to let the platform assign these values instead.
Non-rootness is preserved by ensuring every container image MOCO ships already declares a non-root
USERin its Dockerfile. Themysql,mysqld_exporter, andmoco-agentimages already do this; this PR adds the missingUSER 10000:10000tofluent-bit.Users who need a specific UID/GID/fsGroup can still set it explicitly via
PodTemplate.Spec.Containers[*].SecurityContextorPodTemplate.Spec.SecurityContexton theMySQLClusterresource (cf. #777, which made per-containerSecurityContextoverridable).Related issues
blockOwnerDeletion; see "Out of scope" below.SecurityContextare how users opt back in to specific UID/GID values when needed.Changes
containers/fluent-bit/Dockerfile— addUSER 10000:10000so the fluent-bit sidecar runs as non-root without operator-injected UIDs.cmd/moco-controller/cmd/root.go— add a new--disable-default-security-contextflag (defaultfalse).cmd/moco-controller/cmd/run.go— wire the flag intoMySQLClusterReconciler.controllers/mysqlcluster_controller.goDisableDefaultSecurityContextfield toMySQLClusterReconciler.RunAsUser/RunAsGroupand managed pods (StatefulSet, backupJob, restoreJob) getFSGroupdefaulted toconstants.ContainerUID/ContainerGID.FSGroupChangePolicy=OnRootMismatchis always retained.RunAsUser/RunAsGroup/FSGroup, leaving those fields for the platform (e.g. OpenShift SCC) to assign.controllers/mysql_container.go—updateContainerWithSecurityContextis now a reconciler method that honors the flag, and adefaultPodSecurityContext()helper centralizes the pod-level defaulting.docs/moco-controller.md— document the new flag.Out of scope
The original draft of this PR also dropped
blockOwnerDeletionfrom the PVC owner reference to work around the OpenShiftOwnerReferencesPermissionEnforcementadmission check (#389). Per maintainer feedback, that behavior is preserved here and the underlying issue will be addressed separately (likely via a finalizer-based approach). For now, OpenShift/OKD users can apply the RBAC workaround described in #389 (grantingupdateonmysqlclusters/finalizersto thestatefulset-controllerservice account).Compatibility
No CRD or API changes. With the flag at its default (
false), generated pod and containerSecurityContextare identical to before this PR, so existing deployments are unaffected. On OpenShift, setting--disable-default-security-contextlets the SCC admit the pod and assign project-scoped UID and fsGroup values; non-rootness is still guaranteed by the images'USERdirectives.Helm users can enable the flag via
extraArgs: ["--disable-default-security-context"].