In this blog post I'll describe how you can harden your OpenShift clusters using the Compliance Operator. It is an OpenShift Operator that allows an administrator to run different compliance scans and provide remediations for the issues found. Compliance Operator leverages OpenSCAP under the hood to perform the scans. Among the others it provides CIS OpenShift benchmark compliance profiles, which provides comprehensive set of security controls for OpenShift clusters similar to CIS Kubernetes.
After the installation you can check what compliance profiles are available:
$ NAMESPACE=openshift-compliance
$ oc get -n $NAMESPACE profiles.compliance
NAME AGE
ocp4-cis 3d2h
ocp4-cis-node 3d2h
ocp4-e8 3d2h
ocp4-moderate 3d2h
ocp4-moderate-node 3d2h
ocp4-nerc-cip 3d2h
ocp4-nerc-cip-node 3d2h
ocp4-pci-dss 3d2h
ocp4-pci-dss-node 3d2h
rhcos4-e8 3d2h
rhcos4-moderate 3d2h
rhcos4-nerc-cip 3d2h
For CIS Openshift benchmark compliance scan we'll use ocp4-cis for master nodes scanning and ocp4-cis-node for worker nodes scanning. You can review each of them and check what compliance rules are included using following commands:
$ oc get -n $NAMESPACE -o yaml profiles.compliance ocp4-cis
$ oc get -n $NAMESPACE -o yaml profiles.compliance ocp4-cis-node
You can run both scans using following commands:
$ echo "---
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
name: ocp4-cis-node
profiles:
- name: ocp4-cis-node
kind: Profile
apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
name: default
kind: ScanSetting
apiGroup: compliance.openshift.io/v1alpha1
" | oc create -f - -n $NAMESPACE
$ echo "---
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
name: ocp4-cis
profiles:
- name: ocp4-cis
kind: Profile
apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
name: default
kind: ScanSetting
apiGroup: compliance.openshift.io/v1alpha1
" | oc create -f - -n $NAMESPACE
Wait until they finish executing and show PHASE value DONE as below:
$ oc get -n $NAMESPACE compliancesuites
NAME PHASE RESULT
ocp4-cis DONE NON-COMPLIANT
ocp4-cis-node DONE NON-COMPLIANT
If you'll see RESULT value COMPLANT you are done, but most probably you won't.
Extracting raw scan results is a bit complicated. The scans provide two kinds of raw results: the full report in the ARF format and just the list of scan results in the XCCDF format. The ARF reports are, due to their large size, copied into persistent volumes. The XCCDF results are much smaller and can be stored in a configmap, from which you can extract the results. For easier filtering, the configmaps are labeled with the scan name. You can find more details about scan results extracting here.
Here is just simple example of how you can extract scan results to local file system and find failed compliance rules:
$ oc get -n $NAMESPACE cm -l=compliance.openshift.io/scan-name=ocp4-cis
$ oc extract -n $NAMESPACE cm/ocp4-cis-api-checks-pod --keys=results --confirm
$ cat results | grep fail -B1
<rule-result idref="xccdf_org.ssgproject.content_rule_audit_log_forwarding_enabled" role="full" time="2022-01-20T07:30:17+00:00" severity="medium" weight="1.000000">
<result>fail</result>
--
<rule-result idref="xccdf_org.ssgproject.content_rule_configure_network_policies_namespaces" role="full" time="2022-01-20T07:30:17+00:00" severity="high" weight="1.000000">
<result>fail</result>
Great thing about the Compliance Operator is that for majority of failed compliance rules there will be also remediation created automatically:
$ oc get -n $NAMESPACE complianceremediations
NAME
ocp4-cis-api-server-encryption-provider-config
...
$ oc get -n $NAMESPACE complianceremediation/ocp4-cis-api-server-encryption-provider-config -o yaml
Finally these remediations can be applied manually to the cluster configuration:
$ oc patch -n $NAMESPACE complianceremediations/ocp4-cis-api-server-encryption-provider-config --patch '{"spec":{"apply":true}}' --type=merge
Applying all remediations might not be enough to achieve COMPLIANT results from the scan. There are a couple of compliance rules that will require manual intervention i.e. creation of network policies in every namespace or Kubernetes API audit log forwarding off the cluster configuration. For guidelines on how to implement these remediations please refer to Hardening Guide for OpenShift Container Platform or CIS RedHat OpenShift Container Platform v4 Benchmark.
