OpenShift Security: SCCs, RBAC, OAuth and Hardening

The first thing almost every developer hits on OpenShift is a pod that refuses to start with a message about running as root. The second thing they do is open a ticket that says "please give my app the privileged SCC". How you answer that ticket, calmly and with a better option, is most of the job of an OpenShift platform engineer at a bank. This post gives you the whole security stack in the order an interviewer will ask about it: Security Context Constraints in real depth (the single most-asked OpenShift topic), how to make images that just work under the default policy, Pod Security Admission and how OpenShift bridges it to SCCs, the OAuth server and LDAP/Active Directory integration, RBAC with OpenShift's default roles, secrets and etcd encryption, and the hardening switches you turn on a regulated cluster. By the end you can explain "why won't my container run as root" in thirty seconds, fix it in five minutes, and describe the governance around it like someone who has owned a production platform.

Why the default is strict, and why that is the point

On a laptop, on minikube and on most vanilla Kubernetes clusters, a container with no USER line in its Dockerfile runs as root, UID 0, and nobody notices. On OpenShift the same image either starts as a random user like UID 1000680000 and crashes because it cannot write to /var/cache/nginx, or never gets created at all and the ReplicaSet reports that the pod is "forbidden". Nothing is broken. OpenShift is enforcing a policy that vanilla Kubernetes leaves to you.

The reason is simple once you say it out loud: root inside a container is still UID 0 to the node's kernel, and every container escape ever published gets easier when the process is already root. A bank running hundreds of application teams on shared worker nodes cannot rely on every Dockerfile being careful, so the platform refuses root by default and makes exceptions an explicit, auditable act. Post 11 showed you the securityContext that lets a pod ask for a UID, capabilities or privileged mode; what Kubernetes lacked for years was a built-in way to limit what a pod may ask for. OpenShift has had that since version 3: the Security Context Constraint.

Security Context Constraints: the deep dive

What an SCC actually is

Security Context Constraint (SCC) = a cluster-scoped OpenShift object (securitycontextconstraints.security.openshift.io) that defines what a pod's security context is allowed to request, and fills in safe defaults when the pod does not specify one. It is enforced by an admission plugin in the API server, so it runs on every pod creation, before the scheduler ever sees the pod. SCCs are not namespaced; they are cluster policy. What is namespaced is who gets to use each one, and that is where RBAC comes in later.

The admission plugin does two jobs. First it mutates: if the pod did not set runAsUser, the SCC strategy picks one from the project's allocated range and writes it into the pod, and it fills in SELinux options and fsGroup the same way. Second it validates: if the pod asked for something the SCC forbids, such as privileged: true or a hostPath volume, the pod is rejected with an error that names the SCC and the field. That mutate-then-validate behaviour is why an image with no USER line ends up running as a strange high UID on OpenShift: the pod never said which UID it wanted, so the default SCC chose one for it.

SCC vs PodSecurityPolicy vs Pod Security Admission

Interviewers like to check that you know the history. Kubernetes tried to solve the same problem with PodSecurityPolicy (PSP), heavily inspired by SCCs, but PSP was hard to use, never left beta and was removed in Kubernetes 1.25. Its replacement is Pod Security Admission (PSA), a built-in admission controller that applies one of three fixed profiles (privileged, baseline, restricted) to a whole namespace via labels. PSA is simple and portable but coarse: it cannot say "this one service account in this namespace may use a host port", and it does not mutate anything. OpenShift kept SCCs because they are finer-grained and mutate, and from 4.11 runs PSA alongside them with a controller that keeps the two in sync. The one-sentence version for an interview: "SCCs are OpenShift's original, fine-grained, per-service-account policy for what a pod may request; PSP was the Kubernetes copy that got removed; PSA is the current upstream replacement, namespace-scoped and label-driven; OpenShift runs both, with SCCs doing the real enforcement and PSA levels synced from SCC grants."

The built-in SCCs

Every OpenShift 4 cluster ships with a fixed set of SCCs. The -v2 variants arrived in 4.11 when restricted-v2 became the default for all authenticated users; the older restricted, nonroot and hostnetwork remain for backward compatibility and should not be granted to new workloads. Learn this table; placing a workload against it without looking is what "knows SCCs" sounds like in a conversation.

The scale from top to bottom is not linear: restricted-v2 to nonroot-v2 is a tiny step (you still cannot be root), nonroot-v2 to anyuid is a big one (you can be root), and anyuid to privileged is the difference between "root inside a well-fenced container" and "root on the node". The right answer to almost every application request lives in the top three rows.

The fields an SCC controls

Open the default SCC and read it top to bottom once. Every field maps to something you already know from a pod's securityContext.

$ oc get scc
NAME                              PRIV    CAPS                   SELINUX     RUNASUSER          FSGROUP     SUPGROUP    PRIORITY     READONLYROOTFS   VOLUMES
anyuid                            false   <no value>             MustRunAs   RunAsAny           RunAsAny    RunAsAny    10           false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]
hostaccess                        false   <no value>             MustRunAs   MustRunAsRange     MustRunAs   RunAsAny    <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","hostPath","persistentVolumeClaim","projected","secret"]
hostmount-anyuid                  false   <no value>             MustRunAs   RunAsAny           RunAsAny    RunAsAny    <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","hostPath","nfs","persistentVolumeClaim","projected","secret"]
hostnetwork                       false   <no value>             MustRunAs   MustRunAsRange     MustRunAs   MustRunAs   <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]
hostnetwork-v2                    false   ["NET_BIND_SERVICE"]   MustRunAs   MustRunAsRange     MustRunAs   MustRunAs   <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]
node-exporter                     true    <no value>             RunAsAny    RunAsAny           RunAsAny    RunAsAny    <no value>   false            ["*"]
nonroot                           false   <no value>             MustRunAs   MustRunAsNonRoot   RunAsAny    RunAsAny    <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]
nonroot-v2                        false   ["NET_BIND_SERVICE"]   MustRunAs   MustRunAsNonRoot   RunAsAny    RunAsAny    <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]
privileged                        true    ["*"]                  RunAsAny    RunAsAny           RunAsAny    RunAsAny    <no value>   false            ["*"]
restricted                        false   <no value>             MustRunAs   MustRunAsRange     MustRunAs   RunAsAny    <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]
restricted-v2                     false   ["NET_BIND_SERVICE"]   MustRunAs   MustRunAsRange     MustRunAs   RunAsAny    <no value>   false            ["configMap","csi","downwardAPI","emptyDir","ephemeral","persistentVolumeClaim","projected","secret"]

$ oc describe scc restricted-v2
Name:                                  restricted-v2
Priority:                              <none>
Access:
  Users:                               <none>
  Groups:                              <none>
Settings:
  Allow Privileged:                    false
  Allow Privilege Escalation:          false
  Default Add Capabilities:            <none>
  Required Drop Capabilities:          ALL
  Allowed Capabilities:                NET_BIND_SERVICE
  Allowed Seccomp Profiles:            runtime/default
  Allowed Volume Types:                configMap,csi,downwardAPI,emptyDir,ephemeral,persistentVolumeClaim,projected,secret
  Allow Host Network:                  false
  Allow Host Ports:                    false
  Allow Host PID:                      false
  Allow Host IPC:                      false
  Read Only Root Filesystem:           false
  Run As User Strategy: MustRunAsRange
    UID:                               <none>
    UID Range Min:                     <none>
    UID Range Max:                     <none>
  SELinux Context Strategy: MustRunAs
    Level:                             <none>
  FSGroup Strategy: MustRunAs
    Ranges:                            <none>
  Supplemental Groups Strategy: RunAsAny
    Ranges:                            <none>

Walk through the important ones with the words an interviewer expects:

• runAsUser strategy. MustRunAsRange means the UID must fall in a range, and if the pod does not set one, the plugin picks the first UID of the range. The range is not in the SCC (notice "UID Range Min: none"); it comes from the project. Every namespace gets an annotation openshift.io/sa.scc.uid-range, for example 1000680000/10000, meaning "start at 1000680000, block of 10000", so pods in that project run as 1000680000 by default. MustRunAsNonRoot means any UID except 0, but the pod or image must specify it. RunAsAny means the SCC does not care, which is what lets anyuid run root. MustRunAs with a fixed UID exists too, mostly for custom SCCs.
• SELinux. MustRunAs with the level taken from the project's openshift.io/sa.scc.mcs annotation (something like s0:c26,c5). Every project gets its own MCS category pair, so even two root containers in different projects cannot read each other's files on the node. SELinux is enforcing on every RHCOS node and SCCs wire per-project labels into it automatically; this is the layer people forget when they say "OpenShift is just Kubernetes with a UI".
• fsGroup and supplementalGroups. The GIDs added to the process, which is how a pod gets write access to a mounted volume. fsGroup is MustRunAs from the project's openshift.io/sa.scc.supplemental-groups range; supplementalGroups is RunAsAny in restricted-v2. When a team asks why their arbitrary-UID pod can write to a PVC at all, the answer is fsGroup: the CSI driver or kubelet chowns the volume to that group.
• allowPrivilegedContainer. Whether securityContext.privileged: true is allowed. Only privileged and node-exporter say yes.
• allowedCapabilities / requiredDropCapabilities / defaultAddCapabilities. Linux capabilities are the ~40 slices root's power is split into. The v2 SCCs require dropping ALL and allow adding back only NET_BIND_SERVICE (bind to ports under 1024). anyuid only requires dropping MKNOD. If a vendor asks for SYS_ADMIN, that is effectively root on the node and belongs in the same conversation as privileged.
• allowHostNetwork, allowHostPorts, allowHostPID, allowHostIPC. Whether the pod can share the node's network, port space, process table or IPC namespace. All false in restricted-v2.
• volumes. The list of volume types the pod may mount. The absence of hostPath in restricted-v2 is deliberate; a hostPath mount of / is a full node compromise.
• seccompProfiles. Which seccomp (syscall filter) profiles the pod may use; v2 SCCs pin runtime/default, and set it if the pod is silent.
• allowPrivilegeEscalation. Whether a process can gain more privileges than its parent via setuid binaries. False in v2, which is the "no sudo inside the container" switch.
• readOnlyRootFilesystem. Whether the container's root filesystem must be read-only. Off in the built-ins; a good custom SCC or Kyverno/ACS policy turns it on for hardened workloads.

The few fields not listed here (allowedUnsafeSysctls, forbiddenSysctls and friends) you can read straight from the schema with oc explain scc.allowedUnsafeSysctls; an interviewer respects "I'd check oc explain for the exact field" far more than a guessed name.

How OpenShift chooses which SCC applies

This is the question that separates people who have debugged SCCs from people who have read about them. A pod does not name an SCC. The admission plugin works it out:

1. It collects every SCC that the pod's service account may use, plus, when a human is creating a bare pod with oc create or oc run, the SCCs that human may use. "May use" is decided by RBAC: a subject has access if it is listed in the SCC's users/groups (legacy) or, the modern way, if a Role/ClusterRole granting the use verb on that named SCC is bound to it. Since restricted-v2 is bound to the group system:authenticated, every service account has at least that one.
2. It sorts that set: highest priority first; among equal priorities, most restrictive first; then by name.
3. It tries each SCC in order. The first one that can both fill in defaults and validate the pod without violating any field wins. If none can, the pod is rejected and the error lists why each candidate failed.
4. The winning SCC is recorded on the pod as the annotation openshift.io/scc: <name>, and the mutated securityContext is what you see in oc get pod -o yaml.

Two consequences bite people constantly. First, anyuid has priority 10 and the rest have none, so the moment a service account is granted anyuid, every pod using that SA gets anyuid, even pods that would have been happy under restricted-v2, and they now run as whatever UID the image says, usually root. That is by design (Red Hat wanted "granted anyuid" to mean "use anyuid"), but it makes an anyuid grant a blanket, not a per-pod opt-in. Second, pods created by a Deployment are created by the ReplicaSet controller, not by you, so your SCC access is irrelevant for them; only the pod's service account counts. A cluster-admin can oc run a root pod by hand and then be baffled that the same spec fails inside a Deployment.

$ oc get ns payments -o yaml | grep 'sa.scc'
    openshift.io/sa.scc.mcs: s0:c27,c14
    openshift.io/sa.scc.supplemental-groups: 1000710000/10000
    openshift.io/sa.scc.uid-range: 1000710000/10000

$ oc get pod api-7c9b6d4f8-x2lqn -n payments -o yaml | grep -E 'openshift.io/scc|runAsUser|fsGroup|seLinux' -A1
    openshift.io/scc: restricted-v2
      runAsUser: 1000710000
    fsGroup: 1000710000
    seLinuxOptions:
      level: s0:c27,c14

Reading those two commands together is the whole story: the project's annotations supplied the numbers, restricted-v2 wrote them into the pod, and the annotation tells you which SCC did it. When a pod is running under an SCC you did not expect, this is the first place you look.

Granting an SCC the right way

Sooner or later a workload legitimately needs more than restricted-v2: a vendor's database image runs as a fixed UID 999 with files owned by 999, or a network agent needs hostNetwork. The rule that keeps a bank's cluster auditable is: grant SCCs to a dedicated service account, never to a user, and never to the default service account of a project. Users create nothing in production; controllers do, using the pod's SA. And the default SA is used by every pod that forgot to set one, so granting it anyuid quietly makes the whole project root-capable.

$ oc create sa vendor-db -n payments
serviceaccount/vendor-db created

$ oc adm policy add-scc-to-user nonroot-v2 -z vendor-db -n payments
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:nonroot-v2 added: "vendor-db"

$ oc get rolebinding -n payments | grep scc
system:openshift:scc:nonroot-v2   ClusterRole/system:openshift:scc:nonroot-v2   2m

Notice what the command did. In OpenShift 4 it no longer edits the SCC's users list; it creates (or reuses) a ClusterRole named system:openshift:scc:nonroot-v2 that grants use on that SCC, and binds it to the SA with a RoleBinding in the project. That is the modern, GitOps-friendly form, and you can write it yourself instead of using the helper:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: scc-nonroot-v2-user
rules:
- apiGroups: ["security.openshift.io"]
  resources: ["securitycontextconstraints"]
  resourceNames: ["nonroot-v2"]
  verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: vendor-db-scc
  namespace: payments
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-nonroot-v2-user
subjects:
- kind: ServiceAccount
  name: vendor-db
  namespace: payments

Then point the workload at the SA, because a grant to an SA nobody uses does nothing:

spec:
  template:
    spec:
      serviceAccountName: vendor-db
      containers:
      - name: db
        image: registry.corp.example.com/vendor/db:14.2
        securityContext:
          runAsUser: 999

Removing is symmetrical: oc adm policy remove-scc-from-user nonroot-v2 -z vendor-db -n payments, or delete the RoleBinding in Git. To audit who can use a sensitive SCC across the cluster, ask RBAC rather than reading the SCC object, because the SCC's own users field only shows legacy direct grants:

$ oc adm policy who-can use scc anyuid
Namespace: default
Verb:      use
Resource:  securitycontextconstraints.security.openshift.io

Users:  system:admin
        system:serviceaccount:openshift-monitoring:prometheus-operator
Groups: system:cluster-admins
        system:masters

$ oc get rolebindings,clusterrolebindings -A -o wide | grep 'scc:anyuid'
payments     rolebinding.rbac.authorization.k8s.io/system:openshift:scc:anyuid   ClusterRole/system:openshift:scc:anyuid   12d   payments/legacy-app
             clusterrolebinding.rbac.authorization.k8s.io/system:openshift:scc:anyuid ...

The governance stance for a bank: restricted-v2 and nonroot-v2 are self-service through the golden-path Helm chart (Post 26); anyuid, hostnetwork-v2 and anything with host access require a security exception ticket naming the image, the reason, a compensating control and an expiry date; privileged is reserved for platform operators and never granted to an application namespace. The platform team owns all SCC RoleBindings in Git, and a nightly job reports any binding to anyuid or privileged that has no ticket (compliance reporting is in Post 31).

Making images run under restricted-v2

The best SCC grant is the one you never need. Most "won't start on OpenShift" tickets are image problems, and fixing the image works in every project, on EKS and on vanilla Kubernetes, and never needs an exception review. The rules are few.

Rule 1: assume an arbitrary UID. Your process will run as some UID like 1000710000 that does not exist in /etc/passwd, so do not depend on a specific UID or on a username lookup succeeding. What you can depend on is that the process's primary group is always GID 0, the root group. So make every directory the app writes to group-owned by root with group permissions equal to the owner's: chgrp -R 0 /app && chmod -R g=u /app. Being in group 0 confers none of root's privileges; it is just a shared GID, which is exactly why Red Hat chose it.

Rule 2: do not bind below 1024. Listen on 8080 or 8443 and let the Service or Route map 80/443 to it. If a vendor binary insists on port 80, restricted-v2 does allow adding NET_BIND_SERVICE back, but you must ask for it explicitly in the container's securityContext.capabilities.add; everything is dropped by default.

Rule 3: know your writable paths. Temp files, caches, PID files, log directories: each either lives under a group-0-writable directory you prepared in the image or is an emptyDir mount at runtime. If you can, mount an emptyDir at /tmp and run with readOnlyRootFilesystem: true; it is the cheapest hardening you can add.

Rule 4: set a numeric non-root USER, and never end on USER root. A numeric USER 1001 lets the kubelet verify "non-root" without resolving a name, which matters under nonroot-v2 and under PSA. Ending a Dockerfile on USER root is the single most common cause of the "must run as non-root" family of errors.

Rule 5: prefer UBI base images. Red Hat's Universal Base Images (registry.access.redhat.com/ubi9, ubi9-minimal, ubi9-micro) are built with these rules in mind, patched on Red Hat's cadence, and what your scanner (Post 31) and Red Hat support expect to see.

Here is a Dockerfile that works everywhere except OpenShift, and the version that works everywhere.

# BEFORE: runs as root, writes to root-owned paths, binds port 80
FROM nginx:1.27
COPY site/ /usr/share/nginx/html/
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

# AFTER: arbitrary-UID clean, group-0 permissions, high port
FROM registry.access.redhat.com/ubi9/nginx-124
USER 0
COPY site/ /opt/app-root/src/
COPY nginx.conf /etc/nginx/nginx.conf
RUN sed -i 's/listen\s*80;/listen 8080;/' /etc/nginx/nginx.conf \
 && mkdir -p /var/cache/nginx /var/run/nginx \
 && chgrp -R 0 /var/cache/nginx /var/run/nginx /var/log/nginx /opt/app-root \
 && chmod -R g=u /var/cache/nginx /var/run/nginx /var/log/nginx /opt/app-root
USER 1001
EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"]

Switching to USER 0 for the build steps and back to a numeric non-root user at the end is fine; what matters is the last USER line and the permissions you leave behind. This is what the failures look like when you skip it, so you can recognise them at a glance:

$ oc get pods -n web
NAME                      READY   STATUS                       RESTARTS   AGE
site-6d8f9c7b4-k2p9x      0/1     CreateContainerConfigError   0          41s

$ oc describe pod site-6d8f9c7b4-k2p9x -n web | tail -3
  Warning  Failed     12s (x4 over 40s)  kubelet  Error: container has runAsNonRoot and image will run as root (pod: "site-6d8f9c7b4-k2p9x_web(3f1c...)", container: site)

That one comes from the kubelet, not admission: the pod (or nonroot-v2, or a PSA-driven template) said runAsNonRoot: true, no runAsUser was set, and the image's USER is root. The sibling message image has non-numeric user (nginx), cannot verify user is non-root is the Rule 4 case. Next, the admission rejection, which you find on the ReplicaSet because no pod ever exists:

$ oc get events -n web --sort-by=.lastTimestamp | tail -2
Warning  FailedCreate  replicaset/site-6d8f9c7b4  Error creating: pods "site-6d8f9c7b4-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{0}: 0 is not an allowed group, spec.containers[0].securityContext.runAsUser: Invalid value: 0: must be in the ranges: [1000710000, 1000719999], spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]

Read it like a checklist: the plugin tried every SCC, told you the SA is not allowed most of them, and for the one it is allowed, restricted-v2, listed exactly which requested fields violate it (runAsUser: 0, privileged: true). The fix is written in the error. Finally, the runtime crash when the image starts as an arbitrary UID but was never prepared for it:

$ oc logs site-6d8f9c7b4-k2p9x -n web
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
$ oc logs api-5f7d8b9c6-vq4tz -n web
nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)

Permission denied on a path is Rule 1 or 3; permission denied on a bind is Rule 2. None of them need an SCC.

Pod Security Admission in OpenShift

Pod Security Admission (PSA) = the upstream Kubernetes admission controller that enforces one of three Pod Security Standards per namespace, chosen with labels: pod-security.kubernetes.io/enforce, pod-security.kubernetes.io/audit and pod-security.kubernetes.io/warn, each set to privileged, baseline or restricted (with an optional -version label pinning the standard's version). enforce rejects violating pods; audit writes an annotation into the audit log; warn returns a warning to the client but lets the pod through.

OpenShift 4.11 and later run PSA with a global default of enforce: privileged (so PSA blocks nothing on its own) and warn and audit at restricted. Real enforcement still comes from SCCs. To keep the two consistent, a controller in the cluster-policy-controller watches the SCC RoleBindings in each namespace and syncs the PSA labels to match: a namespace whose service accounts can only use restricted-v2 gets warn/audit at restricted; one where an SA has anyuid gets baseline; one with privileged gets privileged. You can opt a namespace out of syncing with security.openshift.io/scc.podSecurityLabelSync: "false", at which point you own the labels yourself. Namespaces whose names start with openshift- are excluded from the sync and treated as platform namespaces; do not put application workloads there.

$ oc get ns payments --show-labels
NAME       STATUS   AGE   LABELS
payments   Active   30d   kubernetes.io/metadata.name=payments,pod-security.kubernetes.io/audit=restricted,pod-security.kubernetes.io/audit-version=latest,pod-security.kubernetes.io/warn=restricted,pod-security.kubernetes.io/warn-version=latest

$ oc apply -f deployment.yaml
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "api" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "api" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "api" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "api" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
deployment.apps/api created

That warning is the thing teams ask about most after SCC errors. Nothing was rejected; the Deployment was created and its pods will run fine under restricted-v2. The warning exists because PSA evaluates the pod template as you wrote it, and you left blank the fields SCC admission will fill in later. The right response is to add them to the template explicitly, which also makes the manifest portable to EKS and to any future release where OpenShift tightens global enforcement:

securityContext:
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault
containers:
- name: api
  securityContext:
    allowPrivilegeEscalation: false
    capabilities:
      drop: ["ALL"]

Put those eight lines in the golden-path Helm chart and the warning disappears from every team's pipeline at once. Watch for the PodSecurityViolation alert in the monitoring stack (Post 24), which fires when audit mode records workloads that would fail restricted enforcement; it is your early-warning list of manifests to fix before the global default ever moves.

Authentication: the OAuth server and identity providers

Vanilla Kubernetes has no users. The API server accepts client certificates, bearer tokens and OIDC tokens, but creating and managing humans is somebody else's problem (on EKS it is IAM, see Post 27). OpenShift ships an integrated OAuth server (the oauth-openshift pods in openshift-authentication) that sits in front of one or more identity providers (IdPs), verifies the person against the IdP, and mints an OpenShift access token that the API server trusts. Users and identities become real objects: oc get users, oc get identities, oc get groups.

Configuration lives in one cluster-scoped custom resource, the OAuth named cluster, edited with oc edit oauth cluster; the authentication operator redeploys the OAuth server when it changes. The IdP types you will meet at a bank, in order of likelihood:

• LDAP / Active Directory: the classic. The OAuth server binds to the directory with a service account and verifies the user's password with a second bind. Groups are not synced by the IdP itself; that is a separate job below.
• OpenID Connect (OIDC): Microsoft Entra ID (Azure AD), Keycloak / Red Hat build of Keycloak, Okta, PingFederate. The OAuth server redirects the browser to the IdP, receives an ID token, and can map a groups claim into OpenShift groups directly. This is where MFA lives, because the IdP enforces it at login and OpenShift never sees a password.
• htpasswd: a file of bcrypt hashes stored in a Secret. Not for real users; keep one or two break-glass accounts here for the day the directory or the IdP is down.
• GitHub / GitLab / Google: fine for a lab, rarely allowed in a regulated environment.

Here is a realistic LDAP-against-Active-Directory provider. The bind password and CA live in openshift-config, the namespace the authentication operator reads from.

$ oc create secret generic ad-bind-password --from-literal=bindPassword='S3cret!' -n openshift-config
$ oc create configmap ad-ca --from-file=ca.crt=corp-root-ca.pem -n openshift-config

apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
spec:
  identityProviders:
  - name: corp-ad
    mappingMethod: claim
    type: LDAP
    ldap:
      url: "ldaps://ldap.corp.example.com/OU=Users,DC=corp,DC=example,DC=com?sAMAccountName?sub?(objectClass=person)"
      bindDN: "CN=svc-openshift,OU=ServiceAccounts,DC=corp,DC=example,DC=com"
      bindPassword:
        name: ad-bind-password
      ca:
        name: ad-ca
      insecure: false
      attributes:
        id: ["dn"]
        preferredUsername: ["sAMAccountName"]
        name: ["displayName"]
        email: ["mail"]
  - name: break-glass
    mappingMethod: claim
    type: HTPasswd
    htpasswd:
      fileData:
        name: htpass-secret
  tokenConfig:
    accessTokenMaxAgeSeconds: 28800
    accessTokenInactivityTimeout: 30m

The url is an RFC 2255 LDAP URL: base DN, then the attribute used as the login name, the search scope, and a filter. mappingMethod: claim means the first IdP to claim a username owns it; if two IdPs could produce the same name, look at lookup or add in the docs before you ship it. You can watch the operator roll the change out:

$ oc get co authentication
NAME             VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication   4.17.9    True        True          False      14s
$ oc get pods -n openshift-authentication
NAME                               READY   STATUS    RESTARTS   AGE
oauth-openshift-6c9d8f7b5d-7hx2q   1/1     Running   0          22s
oauth-openshift-6c9d8f7b5d-mz8vk   1/1     Running   0          40s
oauth-openshift-6c9d8f7b5d-t5w3p   1/1     Running   0          58s

What oc login actually does

When you run oc login -u alice https://api.prod.corp.example.com:6443, oc discovers the OAuth server from the API server's well-known endpoint, sends your credentials to its authorize endpoint in a challenge flow, the OAuth server checks them against the IdP and answers with a bearer token, and oc stores that token in your kubeconfig. Every later request carries it; the API server looks it up (an OAuthAccessToken object, stored hashed) and resolves it to a User. oc whoami -t prints your current token; oc login --web (4.15 and later) does the same dance through the browser so an OIDC IdP can enforce MFA. Tokens expire per accessTokenMaxAgeSeconds (default 24 hours) and, if configured, after accessTokenInactivityTimeout (minimum five minutes); users list and revoke their own with oc get useroauthaccesstokens and oc delete useroauthaccesstoken <name>. Newer releases can also point the kube-apiserver straight at an external OIDC issuer through the Authentication CR (type: OIDC), bypassing the built-in OAuth server; treat it as version-specific and check the docs for your release.

Groups from LDAP: the unit of access

Authentication tells you who someone is; a bank grants access by group, never by individual, so AD group membership must become Group objects in OpenShift. That is LDAP group sync: oc adm groups sync reads a sync config, queries the directory, and creates or updates OpenShift groups with the same members. Run it on a schedule as a CronJob in a platform namespace, with a service account allowed to manage groups.user.openshift.io, and it becomes the joiner/mover/leaver process for the cluster: HR removes someone from the AD group and within the hour they have lost cluster access with no ticket to the platform team.

kind: LDAPSyncConfig
apiVersion: v1
url: ldaps://ldap.corp.example.com
bindDN: "CN=svc-openshift,OU=ServiceAccounts,DC=corp,DC=example,DC=com"
bindPassword:
  file: /etc/ldap-sync/bindPassword
ca: /etc/ldap-sync/ca.crt
insecure: false
augmentedActiveDirectory:
  groupsQuery:
    baseDN: "OU=Groups,DC=corp,DC=example,DC=com"
    scope: sub
    derefAliases: never
    pageSize: 0
  groupUIDAttribute: dn
  groupNameAttributes: [ cn ]
  usersQuery:
    baseDN: "OU=Users,DC=corp,DC=example,DC=com"
    scope: sub
    derefAliases: never
    filter: (objectClass=person)
    pageSize: 0
  userNameAttributes: [ sAMAccountName ]
  groupMembershipAttributes: [ memberOf ]

$ oc adm groups sync --sync-config=ldap-sync.yaml --whitelist=ocp-groups.txt
apiVersion: v1
items:
- apiVersion: user.openshift.io/v1
  kind: Group
  metadata:
    annotations:
      openshift.io/ldap.sync-time: "2026-09-08T14:02:11Z"
      openshift.io/ldap.uid: CN=ocp-platform-admins,OU=Groups,DC=corp,DC=example,DC=com
    name: ocp-platform-admins
  users:
  - asharma
  - jmoreau
...
$ oc adm groups sync --sync-config=ldap-sync.yaml --whitelist=ocp-groups.txt --confirm
group/ocp-platform-admins
group/ocp-payments-developers
group/ocp-payments-admins

Without --confirm the command is a dry run, which is how you test a new config. The whitelist file limits the sync to the groups you care about, which matters when the directory has ten thousand of them, and oc adm groups prune with the same config removes OpenShift groups whose LDAP counterpart is gone. Of the three schema flavours (rfc2307, activeDirectory, augmentedActiveDirectory), augmentedActiveDirectory is the one that gives you readable group names from AD; start from the docs' example for your directory rather than from memory.

kubeadmin, break-glass and service account tokens

The installer creates a temporary user, kubeadmin, whose password sits in a Secret in kube-system and in the installer's output. It is a shared, unauditable superuser and must go once a real IdP works. The order matters: first bind cluster-admin to a group of named humans and prove one of them can log in, then delete it.

$ oc adm policy add-cluster-role-to-group cluster-admin ocp-platform-admins
clusterrole.rbac.authorization.k8s.io/cluster-admin added: "ocp-platform-admins"
$ oc login -u asharma           # prove it works from a fresh terminal
$ oc auth can-i '*' '*'
yes
$ oc delete secret kubeadmin -n kube-system
secret "kubeadmin" deleted

Deleting that Secret is irreversible; there is no "re-enable kubeadmin". What stays is the installer's auth/kubeconfig, which authenticates with an X.509 client certificate as system:admin and never touches the OAuth server. That file is your true break-glass for the day the IdP or the OAuth server is down: vault it with the same controls as a root password, rotate the cluster's client CA if it is ever exposed, and keep one htpasswd break-glass user alongside it with a vaulted password and an alert on its use in the audit log.

Service accounts authenticate with tokens too, but since 4.11 (Kubernetes 1.24) OpenShift no longer creates a long-lived token Secret for each SA. Pods get a bound token projected into /var/run/secrets/kubernetes.io/serviceaccount/token, scoped to that pod and rotated by the kubelet. For a script or a CI runner, mint a short-lived one: oc create token ci-deployer -n payments --duration=1h. If a legacy system truly needs a non-expiring token, create a Secret of type kubernetes.io/service-account-token annotated with the SA name, and treat it as the liability it is. For the majority of pods that never call the API, set automountServiceAccountToken: false on the SA or pod so a compromised container has no credential to steal.

Cluster-admin hygiene, in the words a bank's security reviewer wants to hear: cluster-admin is bound to exactly one LDAP group with a handful of named members reviewed quarterly; MFA is enforced at the IdP; every action is attributable in the API audit log by username; and the two break-glass paths (the cert kubeconfig and the htpasswd user) are vaulted and alarmed.

Authorization: RBAC and OpenShift's default roles

Once the API server knows who you are, RBAC decides what you may do, and it is byte-for-byte the Kubernetes RBAC from Post 11: Role and ClusterRole hold rules, RoleBinding and ClusterRoleBinding attach them to users, groups or service accounts. What OpenShift adds is a set of well-designed default ClusterRoles, a project model that expects delegation, and oc adm policy helpers that save you from writing bindings by hand.

The helpers wrap the same objects you would write by hand, and each prints what it created so you can find it in Git afterwards:

$ oc adm policy add-role-to-group admin ocp-payments-admins -n payments
clusterrole.rbac.authorization.k8s.io/admin added: "ocp-payments-admins"
$ oc adm policy add-role-to-group edit ocp-payments-developers -n payments
clusterrole.rbac.authorization.k8s.io/edit added: "ocp-payments-developers"
$ oc adm policy add-role-to-user view svc-observer -n payments      # a user, for a one-off
$ oc adm policy add-cluster-role-to-group cluster-reader ocp-sre-oncall

$ oc adm policy who-can create deployments -n payments
Namespace: payments
Verb:      create
Resource:  deployments.apps

Users:  system:admin
        system:serviceaccount:openshift-gitops:openshift-gitops-argocd-application-controller
Groups: ocp-payments-admins
        ocp-payments-developers
        system:cluster-admins
        system:masters

$ oc auth can-i delete secrets -n payments --as=jmoreau
no
$ oc auth can-i --list -n payments --as-group=ocp-payments-developers | head -5

The model that scales to hundreds of teams: the platform team creates the project (with quotas, NetworkPolicies and the SCC bindings from the project template, Post 26) and binds admin to the team's admin group. From then on the team's admins grant edit and view to their own people with RoleBindings in their own project, and can create narrower Roles, without ever asking the platform team. What they cannot do is grant themselves anything cluster-scoped, use an SCC they were not given, or see another team's project. That is delegation with a hard ceiling, and it is why admin is a project role and not a cluster role.

It is also why banks remove self-service project creation. Out of the box every logged-in user can run oc new-project, which creates a namespace with no quota, no NetworkPolicy and a fresh default SA. The standard is to remove that right so projects come only from the platform's onboarding pipeline:

$ oc adm policy remove-cluster-role-from-group self-provisioner system:authenticated:oauth
clusterrole.rbac.authorization.k8s.io/self-provisioner removed: "system:authenticated:oauth"
$ oc patch clusterrolebinding.rbac self-provisioners \
    -p '{"metadata":{"annotations":{"rbac.authorization.kubernetes.io/autoupdate":"false"}}}'
clusterrolebinding.rbac.authorization.k8s.io/self-provisioners patched
$ oc edit project.config.openshift.io cluster   # set spec.projectRequestMessage
$ oc new-project scratch --as=jmoreau
Error from server (Forbidden): You may not request a new project via this API. Open a Platform Onboarding ticket at https://go/ocp-onboard.

The patch matters: without it, the next control-plane upgrade's bootstrap reconciles the default binding back. Setting projectRequestMessage turns a confusing Forbidden into a signpost.

Two more things about defaults. view and cluster-reader both exclude Secrets on purpose; a custom ClusterRole with get, list on * for a role that "just needs read access" has quietly granted read on every Secret in the cluster. And operators that add CRDs usually ship ClusterRoles labelled rbac.authorization.k8s.io/aggregate-to-admin: "true" (or -edit, -view) so that project admins automatically gain rights on the new kinds; if a team cannot see an operator's objects in their project, a missing aggregation label is the first suspect.

Secrets and sensitive data on the platform

A Kubernetes Secret is base64, not encryption (Post 6). On the platform side the question is what protects Secrets where they actually live: in etcd, on the control-plane nodes' disks, and in every etcd backup you copy to S3. By default OpenShift stores them in plain text there (encryption.type: identity). Turning on etcd encryption at rest is one edit to the APIServer CR and is on every bank's baseline:

$ oc patch apiserver cluster --type=merge -p '{"spec":{"encryption":{"type":"aesgcm"}}}'
apiserver.config.openshift.io/cluster patched
$ oc get openshiftapiserver -o jsonpath='{.items[0].status.conditions[?(@.type=="Encrypted")].message}{"\n"}'
EncryptionCompleted
All resources encrypted: routes.route.openshift.io
$ oc get kubeapiserver -o jsonpath='{.items[0].status.conditions[?(@.type=="Encrypted")].message}{"\n"}'
EncryptionCompleted
All resources encrypted: secrets, configmaps

aescbc is the older option and aesgcm the faster one available since 4.13; either satisfies auditors. The operators encrypt Secrets, ConfigMaps, Routes and OAuth tokens and manage the keys for you (they live in the openshift-config-managed namespace, so an etcd backup alone is not enough to decrypt). The rewrite takes minutes on a small cluster and longer on a big one; take a fresh etcd backup after it completes (Post 20 covers the procedure).

Encryption at rest does not change who can read a Secret through the API, so the rest of the story is keeping secrets out of the places people can see them. Secrets in Git are forbidden, even in a private repository, because Git never forgets. The pattern that works with GitOps (Post 30) is to commit a reference and let a controller fetch the value at runtime: the External Secrets Operator creates Kubernetes Secrets from HashiCorp Vault, AWS Secrets Manager or Azure Key Vault; the Vault Agent Injector adds a sidecar that writes secrets to a memory-backed volume; the Secrets Store CSI driver mounts them as files without ever creating a Secret object. Post 31 compares them; a bank picks one, wires it into the golden path, and treats a raw Secret in a Git repo as a pipeline-failing policy violation.

Three platform-owned secrets deserve their own controls. The global pull secret (pull-secret in openshift-config) holds the credentials nodes use to pull the OpenShift payload from Red Hat and, once you add it, from your corporate registry; update it with oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=merged.json and the MCO rolls it to every node. Per-project image pull secrets should be scoped robot accounts from the registry (Quay or Artifactory), linked to the SA with oc secrets link myapp regcred --for=pull, never a human's credentials. And the service account token is a secret too: turn automount off wherever the pod does not talk to the API.

Hardening the cluster

Hardening is a set of switches on a few cluster-scoped CRs plus a handful of operators. Interviewers do not expect you to recite every field, but they do expect you to know which object each control lives on and what changing it costs (a control-plane rollout, a node reboot, an outage window). Post 31 builds the full compliance programme on top of these switches; this section is the switch list.

The API server: audit, TLS, anonymous access, exposure

Audit logging turns "who deleted the namespace" from a mystery into a query. The audit profile is set on the APIServer CR: Default logs metadata for every request (who, what, when, response code) and request bodies only for OAuth token operations; WriteRequestBodies adds the full body of every create, update, patch and delete, which is what auditors ask for; AllRequestBodies adds read bodies and is too noisy for most clusters; None disables auditing and should be unreachable by policy. customRules can layer profiles by group, for example bodies for human users but metadata only for service accounts.

$ oc patch apiserver cluster --type=merge -p '{"spec":{"audit":{"profile":"WriteRequestBodies"}}}'
apiserver.config.openshift.io/cluster patched
$ oc adm node-logs --role=master --path=kube-apiserver/audit.log | tail -1 | jq -r '[.verb,.user.username,.objectRef.namespace,.objectRef.resource,.objectRef.name,.responseStatus.code] | @tsv'
delete   jmoreau   payments   deployments   api   200

Changing the profile rolls the kube-apiserver pods one by one with no outage. The logs live on the control-plane nodes and rotate, so forward them off-box with the ClusterLogForwarder's audit input to Loki or the bank's SIEM (Post 24), where retention is measured in years. Three more switches live nearby. TLS security profiles (spec.tlsSecurityProfile on the APIServer CR, on the IngressController for Routes, and via KubeletConfig for kubelets) set the ciphers and minimum TLS version: Intermediate (TLS 1.2 and up) is the default and what most banks run, Old exists for legacy clients and is a finding, Custom pins exact ciphers, and Modern (TLS 1.3 only) is not supported on every component in every release, so check before you promise it. Anonymous access: OpenShift binds system:unauthenticated to a few discovery roles so that oc can find the OAuth endpoint before login; recent releases tighten this on new installs, and on an existing cluster you list the ClusterRoleBindings with that subject and remove the ones your version's hardening guide says are safe to drop. Exposure: at a bank the API and console are not on the internet, so use publish: Internal at install time; the IngressController's endpointPublishingStrategy.loadBalancer.scope can be switched to Internal later, but the API load balancer's scope cannot on every cloud, so decide before installing.

Nodes: FIPS, SSH, kernel parameters

Nodes are RHCOS and change only through the MachineConfig Operator (Post 20), which makes node hardening declarative and drift-resistant. FIPS mode is fips: true in install-config.yaml, cannot be turned on later (Canadian federal work often needs it, so decide before installing), and is verified with oc debug node/<n> -- chroot /host fips-mode-setup --check. SSH to nodes should be unnecessary: the installer's key lands through the 99-master-ssh and 99-worker-ssh MachineConfigs, so replace it with a key held only by the platform team or empty it and rely on oc debug node/, which goes through the API and therefore through RBAC and the audit log. Kernel parameters and sysctls a hardening benchmark demands go in a MachineConfig's kernelArguments or a KubeletConfig, and each costs a rolling reboot of the pool, so batch them.

Image sources: only the corporate registry

The cleanest supply-chain control is refusing to run images from anywhere except the registry your scanner watches. It lives on the Image CR, image.config.openshift.io/cluster: allowedRegistriesForImport restricts where ImageStreams may import from, and registrySources restricts what nodes will pull, with either an allowedRegistries list (everything else blocked) or a blockedRegistries list (everything else allowed), never both:

apiVersion: config.openshift.io/v1
kind: Image
metadata:
  name: cluster
spec:
  allowedRegistriesForImport:
  - domainName: registry.corp.example.com
    insecure: false
  registrySources:
    allowedRegistries:
    - registry.corp.example.com
    - quay.io
    - registry.redhat.io
    - registry.access.redhat.com
    - image-registry.openshift-image-registry.svc:5000

The four extra entries are not optional: the cluster's own payload comes from quay.io, operators from registry.redhat.io, and builds from the internal registry, so an allowedRegistries list that omits them breaks upgrades and operator installs. The change rolls out through the MCO as a new /etc/containers/registries.conf and policy.json, so expect the pools to update. For a disconnected or partially-connected cluster the companion object is ImageDigestMirrorSet (4.13 and later; it replaced ImageContentSourcePolicy), which tells CRI-O "when asked for registry.redhat.io/x, pull from registry.corp.example.com/redhat/x instead":

apiVersion: config.openshift.io/v1
kind: ImageDigestMirrorSet
metadata:
  name: redhat-mirror
spec:
  imageDigestMirrors:
  - source: registry.redhat.io
    mirrors:
    - registry.corp.example.com/redhat
  - source: quay.io/openshift-release-dev
    mirrors:
    - registry.corp.example.com/openshift-release

Signature verification is the next layer: Red Hat signs its images, and recent releases add ClusterImagePolicy and namespaced ImagePolicy objects for sigstore-based verification enforced by CRI-O at pull time. Their API and support status changed across 4.15 to 4.19, so say "we verify signatures at the node with sigstore policies, and I'd check the exact CR status for our version" rather than quoting fields; older approaches used GPG keys in policy.json delivered by MachineConfig.

The security operators and Advanced Cluster Security

Four Red Hat-supported operators cover the "prove it" side of hardening; Post 31 covers the programme built on them. The Compliance Operator scans the cluster and its nodes against profiles derived from public benchmarks: ocp4-cis and ocp4-cis-node (CIS OpenShift benchmark, platform and node halves), ocp4-pci-dss and ocp4-pci-dss-node, ocp4-moderate (NIST 800-53), ocp4-stig, and rhcos4-* for the OS. You bind profiles to a ScanSetting (schedule, storage, node roles) with a ScanSettingBinding; the operator produces one ComplianceCheckResult per rule and, where it knows how, a ComplianceRemediation that is a ready-made MachineConfig or object patch. The default-auto-apply ScanSetting applies remediations automatically, which most banks do on non-production first.

apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-baseline
  namespace: openshift-compliance
profiles:
- name: ocp4-cis
  kind: Profile
  apiGroup: compliance.openshift.io/v1alpha1
- name: ocp4-cis-node
  kind: Profile
  apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1

$ oc get compliancesuites -n openshift-compliance
NAME           PHASE   RESULT
cis-baseline   DONE    NON-COMPLIANT
$ oc get compliancecheckresults -n openshift-compliance -l compliance.openshift.io/check-status=FAIL | head -4
NAME                                                          STATUS   SEVERITY
ocp4-cis-api-server-encryption-provider-cipher                FAIL     medium
ocp4-cis-audit-log-forwarding-enabled                         FAIL     medium
ocp4-cis-node-master-kubelet-configure-tls-cipher-suites      FAIL     medium

The failures map straight onto the switches from the previous sections; the operator is a checklist that runs itself. The File Integrity Operator runs AIDE on every node and reports a FileIntegrityNodeStatus of Failed when a file outside the expected set changes, which is the "was this node tampered with" control auditors ask for. The Security Profiles Operator manages seccomp and SELinux profiles as CRs and can record a profile from a running workload, for the rare app that must run tighter than runtime/default. And Red Hat Advanced Cluster Security (RHACS, formerly StackRox) sits above all of them: a Central server plus a Sensor and Collector per secured cluster that scans images for CVEs, enforces policies at build (roxctl in CI), at deploy (an admission webhook that can block a pod using anyuid, a latest tag, or an image with critical CVEs) and at runtime (process and network behaviour off the baseline), draws a live network graph that can generate NetworkPolicies, and reports against CIS, PCI and NIST. For the interview, place it correctly: SCCs and PSA decide what a pod may request, ACS decides whether the image and its behaviour are acceptable.

NetworkPolicy and multi-tenancy in one breath

Everything above is about identity and privilege; NetworkPolicy is about reach, and on a shared cluster it is the control that makes "tenant" mean anything. Post 21 covers the mechanics with OVN-Kubernetes; the security summary is that every project created through the onboarding template starts with a default-deny ingress policy plus allows for the ingress router and the monitoring stack, teams add allows for their own callers, and cluster-wide guardrails that teams cannot override are expressed as AdminNetworkPolicy. Multi-tenancy on OpenShift is therefore a stack: a project boundary for RBAC, a UID range and an SELinux category per project from SCCs, NetworkPolicy for traffic, ResourceQuota and LimitRange for capacity, and dedicated tainted node pools for the workloads that regulation says may not share a kernel with anything else.

A security troubleshooting method: "my pod won't start"

Interviewers grade the order in which you look, not the fix. Say it as a sequence, every time.

1. Is there a pod at all? oc get pods. If the Deployment shows 0/1 and no pod exists, it was rejected at admission. Read the FailedCreate message in oc get events --sort-by=.lastTimestamp or oc describe rs; if it contains unable to validate against any security context constraint, it is an SCC problem and the message lists exactly which fields violate which SCC.
2. Pod exists but never starts? oc describe pod. CreateContainerConfigError with runAsNonRoot and image will run as root is the kubelet enforcing non-root against an image whose USER is root. A missing Secret or ConfigMap shows up here too, and is not a security issue.
3. Pod starts and crashes? oc logs --previous. Permission denied on a path or a bind is the arbitrary-UID rules; fix the image.
4. Which SCC did it get, and with what context? oc get pod x -o yaml | grep -E 'openshift.io/scc|securityContext' -A5. Compare the assigned UID and SELinux level with what the app expects.
5. Would it be admitted under a different SA or SCC? Test without deploying:

$ oc adm policy scc-subject-review -f pod.yaml -z vendor-db -n payments
RESOURCE        ALLOWED BY
Pod/vendor-db   nonroot-v2

$ oc adm policy scc-review -f deployment.yaml -n payments
RESOURCE                   SERVICE ACCOUNT   ALLOWED BY
Deployment/legacy-app      default           <none>

scc-subject-review answers "which SCC would admit this spec for this subject"; scc-review answers "for the SA named in this workload, which SCC applies", and <none> means it would be rejected. Then fix in this order and only go as far as you must: image first (arbitrary UID, group 0, high port, writable paths), service account second (a dedicated SA, referenced in the spec, automount off), SCC last (the smallest one that admits the pod, granted to that SA, with a ticket if it is anyuid or beyond). When a team says "it works on minikube", that describes minikube's policy, not a bug in yours.

Likely interview questions

Next up: Operators and OLM, the mechanism that installs and upgrades everything from the Compliance Operator to your database, and how to troubleshoot an operator that will not install, will not upgrade, or is Degraded.