Understanding the Custom Resource Definitions - Compliance Operator | Security and compliance

Defining the compliance scan requirements
Configuring the compliance scan settings
Processing the compliance scan requirements with compliance scans settings
Tracking the compliance scans
Viewing the compliance results
Compliance Operator custom resource definition workflow
ProfileBundle object
Profile object
Rule object
TailoredProfile object
ScanSetting object

You can use the Custom Resource Definitions (CRDs) provided by the Compliance Operator to run compliance scans and get remediation for the issues found.

The Compliance Operator in the OKD provides you with several Custom Resource Definitions (CRDs) to run the compliance scans. The Compliance Operator converts security policies into CRDs, which you can use.

Defining the compliance scan requirements

You can define and set the rules for your compliance scan requirements in the Compliance Operator CRDs, which include ProfileBundle and Profile objects. You can also customize the default profiles by using a TailoredProfile object.

Configuring the compliance scan settings

After you have defined the requirements of the compliance scan, you can use a ScanSetting object to configure the scan by specifying the type, occurrence, and location of the scan.

Processing the compliance scan requirements with compliance scans settings

When you have defined the compliance scan requirements and configured the settings to run the scans, the Compliance Operator processes these settings by using the ScanSettingBinding object.

Tracking the compliance scans

After the creation of compliance suite, you can monitor the status of the deployed scans by using the ComplianceSuite object.

Viewing the compliance results

When the compliance suite reaches the DONE phase, you can view the scan results and possible remediation.

Compliance Operator custom resource definition workflow

You can use the Compliance Operator Custom Resource Definition (CRD) workflow to define requirements, configure settings, process scans, monitor compliance checks, and review results.

The CRD workflow includes the following steps:

Define your compliance scan requirements.
Configure the compliance scan settings.
Process compliance requirements with compliance scans settings.
Monitor the compliance scans.
Check the compliance scan results.

ProfileBundle object

When you install the Compliance Operator, it includes ready-to-run ProfileBundle objects. The Compliance Operator parses the ProfileBundle object and creates a Profile object for each profile in the bundle. It also parses Rule and Variable objects, which are used by the Profile object.

Example ProfileBundle object

apiVersion: compliance.openshift.io/v1alpha1
kind: ProfileBundle
  name: <profile bundle name>
  namespace: openshift-compliance
status:
  dataStreamStatus: VALID

where:

status.dataStreamStatus: Specifies whether the Compliance Operator was able to parse the content files. Value is VALID when parsing succeeds.

When the contentFile fails, an errorMessage attribute is displayed, which provides details of the error that occurred.

Troubleshooting

When you roll back to a known content image from an invalid image, the ProfileBundle object stops responding and displays PENDING state. As a workaround, you can move to a different image than the earlier one or you can delete and re-create the ProfileBundle object to return to the working state.

Profile object

You can use the Profile object to review parsed out details about an OpenSCAP profile, such as its XCCDF identifier and profile checks for a Node or Platform type.

The Profile object defines the rules and variables that can be evaluated for a certain compliance standard. You can either directly use the Profile object or further customize it using a TailorProfile object.

You cannot create or modify the Profile object manually because it is derived from a single ProfileBundle object. Typically, a single ProfileBundle object can include several Profile objects.

Example Profile object

apiVersion: compliance.openshift.io/v1alpha1
description: <description of the profile>
id: xccdf_org.ssgproject.content_profile_moderate
kind: Profile
metadata:
  annotations:
    compliance.openshift.io/product: <product name>
    compliance.openshift.io/product-type: Node
  creationTimestamp: "YYYY-MM-DDTMM:HH:SSZ"
  generation: 1
  labels:
    compliance.openshift.io/profile-bundle: <profile bundle name>
  name: rhcos4-moderate
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ProfileBundle
    name: <profile bundle name>
    uid: <uid string>
  resourceVersion: "<version number>"
  selfLink: /apis/compliance.openshift.io/v1alpha1/namespaces/openshift-compliance/profiles/rhcos4-moderate
  uid: <uid string>
rules:
- rhcos4-account-disable-post-pw-expiration
- rhcos4-accounts-no-uid-except-zero
- rhcos4-audit-rules-dac-modification-chmod
- rhcos4-audit-rules-dac-modification-chown
title: <title of the profile>

where:

id: Specifies the XCCDF name of the profile. Use this identifier when you define a ComplianceScan object as the value of the profile attribute of the scan.
metadata.annotations.compliance.openshift.io/product-type: Specifies either a Node or Platform. Node profiles scan the cluster nodes and platform profiles scan the Kubernetes platform.
rules: Specifies the list of rules for the profile. Each rule corresponds to a single check.

Rule object

You can use the Rule object, which represents an individual compliance check, to view check details and understand why a scan result passed or failed.

The Rule objects, which form the profiles, are also exposed as objects. You can use the Rule object to define your compliance check requirements and specify how a failed compliance check can be remediated.

Example Rule object

    apiVersion: compliance.openshift.io/v1alpha1
    checkType: Platform
    description: <description of the rule>
    id: xccdf_org.ssgproject.content_rule_configure_network_policies_namespaces
    instructions: <manual instructions for the scan>
    kind: Rule
    metadata:
      annotations:
        compliance.openshift.io/rule: configure-network-policies-namespaces
        control.compliance.openshift.io/CIS-OCP: 5.3.2
        control.compliance.openshift.io/NERC-CIP: CIP-003-3 R4;CIP-003-3 R4.2;CIP-003-3
          R5;CIP-003-3 R6;CIP-004-3 R2.2.4;CIP-004-3 R3;CIP-007-3 R2;CIP-007-3 R2.1;CIP-007-3
          R2.2;CIP-007-3 R2.3;CIP-007-3 R5.1;CIP-007-3 R6.1
        control.compliance.openshift.io/NIST-800-53: AC-4;AC-4(21);CA-3(5);CM-6;CM-6(1);CM-7;CM-7(1);SC-7;SC-7(3);SC-7(5);SC-7(8);SC-7(12);SC-7(13);SC-7(18)
      labels:
        compliance.openshift.io/profile-bundle: ocp4
      name: ocp4-configure-network-policies-namespaces
      namespace: openshift-compliance
    rationale: <description of why this rule is checked>
    severity: high
    title: <summary of the rule>

where:

checkType

Specifies the type of check this rule executes. Node profiles scan the cluster nodes and Platform profiles scan the Kubernetes platform. An empty value indicates there is no automated check.

id

Specifies the XCCDF name of the rule, which is parsed directly from the datastream.

severity

Specifies the severity of the rule when it fails.

The Rule object gets an appropriate label for an easy identification of the associated ProfileBundle object. The ProfileBundle also gets specified in the OwnerReferences of this object.

TailoredProfile object

You can use the TailoredProfile object to modify the default Profile object based on your organization requirements. You can enable or disable rules, set variable values, and provide justification for the customization.

After validation, the TailoredProfile object creates a ConfigMap, which can be referenced by a ComplianceScan object.

You can use the TailoredProfile object by referencing it in a ScanSettingBinding object. For more information about ScanSettingBinding, see ScanSettingBinding object.

Example TailoredProfile object

apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: rhcos4-with-usb
spec:
  extends: rhcos4-moderate
  title: <title of the tailored profile>
  disableRules:
    - name: <name of a rule object to be disabled>
      rationale: <description of why this rule is checked>
status:
  id: xccdf_compliance.openshift.io_profile_rhcos4-with-usb
  outputRef:
    name: rhcos4-with-usb-tp
    namespace: openshift-compliance
  state: READY

where:

spec.extends: Optional parameter. Specifies the name of the Profile object upon which the TailoredProfile is built. If no value is set, a new profile is created from the enableRules list.
status.id: Specifies the XCCDF name of the tailored profile.
status.outputRef.name: Specifies the ConfigMap name, which can be used as the value of the tailoringConfigMap.name attribute of a ComplianceScan.
status.state: Specifies the state of the object such as READY, PENDING, and FAILURE. If the state of the object is ERROR, then the attribute status.errorMessage provides the reason for the failure.

With the TailoredProfile object, you can create a new Profile object by using the TailoredProfile construct. To create a new Profile, set the following configuration parameters:

an appropriate title
extends value must be empty

scan type annotation on the TailoredProfile object:

compliance.openshift.io/product-type: Platform/Node

If you have not set the product-type annotation, the Compliance Operator defaults to Platform scan type. Adding the -node suffix to the name of the TailoredProfile object results in node scan type.

ScanSetting object

You can use the ScanSetting object to define and reuse the operational policies to run your scans, reducing configuration repetition across many scan bindings.

By default, the Compliance Operator creates the following ScanSetting objects:

default - Runs a scan every day at 1 AM on both control plane and worker nodes by using a 1Gi Persistent Volume (PV) and keeps the last three results. Remediation is neither applied nor updated automatically.
default-auto-apply - Runs a scan every day at 1 AM on both control plane and worker nodes by using a 1Gi Persistent Volume (PV) and keeps the last three results. Both autoApplyRemediations and autoUpdateRemediations are set to true.

Example ScanSetting object

apiVersion: compliance.openshift.io/v1alpha1
autoApplyRemediations: true
autoUpdateRemediations: true
kind: ScanSetting
maxRetryOnTimeout: 3
metadata:
  creationTimestamp: "2022-10-18T20:21:00Z"
  generation: 1
  name: default-auto-apply
  namespace: openshift-compliance
  resourceVersion: "38840"
  uid: 8cb0967d-05e0-4d7a-ac1c-08a7f7e89e84
rawResultStorage:
  nodeSelector:
    node-role.kubernetes.io/master: ""
  pvAccessModes:
  - ReadWriteOnce
  rotation: 3
  size: 1Gi
  tolerations:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
    operator: Exists
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  - effect: NoSchedule
    key: node.kubernetes.io/memory-pressure
    operator: Exists
roles:
- master
- worker
scanTolerations:
- operator: Exists
schedule: 0 1 * * *
showNotApplicable: false
strictNodeScan: true
timeout: 30m

where:

autoApplyRemediations

Set to true to enable auto remediations. Set to false to disable auto remediations.

autoUpdateRemediations

Set to true to enable auto remediations for content updates. Set to false to disable auto remediations for content updates.

rawResultStorage.rotation

Specifies the number of stored scans in the raw result format. The default value is 3. As the older results get rotated, the administrator must store the results elsewhere before the rotation happens. To disable the rotation policy, set the value to 0.

rawResultStorage.size

Specifies the storage size that must be created for the scan to store the raw results. The default value is 1Gi.

schedule

Specifies how often the scan must be run in cron format.

To disable the rotation policy, set the value to 0.

roles

Specifies the node-role.kubernetes.io label value to schedule the scan for Node type. This value must match the name of a MachineConfigPool.

ScanSettingBinding object

You can use the ScanSettingBinding object to specify your compliance requirements with reference to the Profile or TailoredProfile object.

The ScanSettingBinding object is linked to a ScanSetting object, which provides the operational constraints for the scan. Then the Compliance Operator generates the ComplianceSuite object based on the ScanSetting and ScanSettingBinding objects.

Example ScanSettingBinding object

apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: <name of the scan>
profiles:
  # Node checks
  - name: rhcos4-with-usb
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
  # Cluster checks
  - name: ocp4-moderate
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: my-companys-constraints
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1

where:

profiles: Specifies the details of Profile or TailoredProfile object to scan your environment.
settingsRef: Specifies the operational constraints, such as schedule and storage size.

The creation of ScanSetting and ScanSettingBinding objects results in the compliance suite. To get the list of compliance suite, run the following command:

$ oc get compliancesuites

If you delete ScanSettingBinding, then compliance suite also is deleted.

ComplianceSuite object

You can review a ComplianceSuite object to keep track of the state of the scans. The object has the raw settings to create scans and an overall result.

For Node type scans, map the scan to the MachineConfigPool, because the scan has the remediation for any issues. If you specify a label, ensure the label directly applies to a pool.

Example ComplianceSuite object

apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceSuite
metadata:
  name: <name_of_the_suite>
spec:
  autoApplyRemediations: false
  schedule: "0 1 * * *"
  scans:
    - name: workers-scan
      scanType: Node
      profile: xccdf_org.ssgproject.content_profile_moderate
      content: ssg-rhcos4-ds.xml
      contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
      rule: "xccdf_org.ssgproject.content_rule_no_netrc_files"
      nodeSelector:
        node-role.kubernetes.io/worker: ""
status:
  Phase: DONE
  Result: NON-COMPLIANT
  scanStatuses:
  - name: workers-scan
    phase: DONE
    result: NON-COMPLIANT

where:

spec.autoApplyRemediations: Set to true to enable auto remediations. Set to false to disable auto remediations.
spec.schedule: Specifies how often the scan should be run in cron format.
spec.scans: Specifies a list of scan specifications to run in the cluster.
status.Phase: Specifies the progress of the scans.
status.Result: Specifies the overall verdict of the suite.

The suite in the background creates the ComplianceScan object based on the scans parameter. You can programmatically fetch the ComplianceSuites events. To get the events for the suite, run the following command:

$ oc get events --field-selector involvedObject.kind=ComplianceSuite,involvedObject.name=<name of the suite>

You might create errors when you manually define the ComplianceSuite, since it contains the XCCDF attributes.

Advanced ComplianceScan Object

You can use the ComplianceScan object to configure advanced options such as custom result storage, debug pods, and scan suspension for troubleshooting and tool integration.

The Compliance Operator includes options for advanced users for debugging or integrating with existing tool. While Red Hat recommends that you should not create a ComplianceScan object directly, you can instead manage the object by using a ComplianceSuite object.

Example Advanced ComplianceScan object

apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceScan
metadata:
  name: <name_of_the_compliance_scan>
spec:
  scanType: Node
  profile: xccdf_org.ssgproject.content_profile_moderate
  content: ssg-ocp4-ds.xml
  contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
  rule: "xccdf_org.ssgproject.content_rule_no_netrc_files"
  nodeSelector:
    node-role.kubernetes.io/worker: ""
status:
  phase: DONE
  result: NON-COMPLIANT

where:

spec.scanType

Specifies either Node or Platform. Node profiles scan the cluster nodes and platform profiles scan the Kubernetes platform.

spec.profile

Specifies the XCCDF (Extensible Configuration Checklist Description Format) identifier of the profile that you want to run.

spec.contentImage

Specifies the container image that encapsulates the profile files.

spec.rule

Specifies the scan to run a single rule. This rule must be identified with the XCCDF ID, and must belong to the specified profile.

If you skip the rule parameter, then scan runs for all the available rules of the specified profile.

spec.nodeSelector

If you are on the OKD and wants to generate a remediation, the nodeSelector label must match the MachineConfigPool label.

If you do not specify nodeSelector parameter or match the MachineConfig label, scan will still run, but it will not create remediation.

status.phase

Specifies the current phase of the scan.

status.result

Specifies the verdict of the scan.

If you delete a ComplianceSuite object, then all the associated scans get deleted.

When the scan is complete, it generates the result as Custom Resources of the ComplianceCheckResult object. However, the raw results are available in ARF format. These results are stored in a Persistent Volume (PV), which has a Persistent Volume Claim (PVC) associated with the name of the scan. You can programmatically fetch the ComplianceScans events. To generate events for the suite, run the following command:

oc get events --field-selector involvedObject.kind=ComplianceScan,involvedObject.name=<name_of_the_compliance_scan>

ComplianceCheckResult object

When you run a scan with a specific profile, several rules in the profiles are verified. For each of these rules, a ComplianceCheckResult object is created, which provides the state of the cluster for a specific rule.

Example ComplianceCheckResult object

apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceCheckResult
metadata:
  labels:
    compliance.openshift.io/check-severity: medium
    compliance.openshift.io/check-status: FAIL
    compliance.openshift.io/suite: example-compliancesuite
    compliance.openshift.io/scan-name: workers-scan
  name: workers-scan-no-direct-root-logins
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ComplianceScan
    name: workers-scan
description: <description of scan check>
instructions: <manual instructions for the scan>
id: xccdf_org.ssgproject.content_rule_no_direct_root_logins
severity: medium
status: FAIL

where:

severity

Specifies the severity of the scan check.

status

Specifies the result of the check. The possible values are:

PASS: Specifies if the check was successful.
FAIL: Specifies if the check was unsuccessful.
INFO: Specifies if the check was successful and found something not severe enough to be considered an error.
MANUAL: Specifies if the check cannot automatically assess the status and manual check is required.
INCONSISTENT: Specifies that the different nodes are reporting different results.
ERROR: Specifies if the check ran successfully, but could not complete.
NOTAPPLICABLE: Specifies if the check did not run as it is not applicable.

To get all the check results from a suite, run the following command:

oc get compliancecheckresults \
-l compliance.openshift.io/suite=workers-compliancesuite

ComplianceRemediation object

If a Kubernetes fix is available, the Compliance Operator creates a ComplianceRemediation object, which you can use to determine a way to fix a problem described in a ComplianceCheckResult object.

For a specific check you can have a datastream specified fix. However, if a Kubernetes fix is available, then the Compliance Operator creates a ComplianceRemediation object.

Example ComplianceRemediation object

apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceRemediation
metadata:
  labels:
    compliance.openshift.io/suite: example-compliancesuite
    compliance.openshift.io/scan-name: workers-scan
    machineconfiguration.openshift.io/role: worker
  name: workers-scan-disable-users-coredumps
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ComplianceCheckResult
    name: workers-scan-disable-users-coredumps
    uid: <UID>
spec:
  apply: false
  object:
    current:
       apiVersion: machineconfiguration.openshift.io/v1
       kind: MachineConfig
       spec:
         config:
           ignition:
             version: 2.2.0
           storage:
             files:
             - contents:
                 source: data:,%2A%20%20%20%20%20hard%20%20%20core%20%20%20%200
               filesystem: root
               mode: 420
               path: /etc/security/limits.d/75-disable_users_coredumps.conf
    outdated: {}

where:

spec.apply: A true value specifies the remediation was applied. A false value indicates the remediation was not applied.
spec.object.current: Specifies the definition of the remediation.
spec.object.outdated: Specifies remediation that was before parsed from an earlier version of the content. The Compliance Operator still retains the outdated objects to give the administrator a chance to review the new remediations before applying them.

To get all the remediations from a suite, run the following command:

oc get complianceremediations \
-l compliance.openshift.io/suite=workers-compliancesuite

To list all failing checks that can be remediated automatically, run the following command:

oc get compliancecheckresults \
-l 'compliance.openshift.io/check-status in (FAIL),compliance.openshift.io/automated-remediation'

To list all failing checks that can be remediated manually, run the following command:

oc get compliancecheckresults \
-l 'compliance.openshift.io/check-status in (FAIL),!compliance.openshift.io/automated-remediation'

Additional resources

ComplianceAsCode community project