Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions modules/compliance-apply-remediation-for-customized-mcp.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
[id="compliance-operator-apply-remediation-for-customized-mcp"]
= Applying remediation when using customized machine config pools

[role="_abstract"]
When you create a custom `MachineConfigPool`, add a label to the `MachineConfigPool` so that `machineConfigPoolSelector` present in the `KubeletConfig` can match the label with `MachineConfigPool`.

[IMPORTANT]
Expand All @@ -18,7 +19,7 @@ Do not set `protectKernelDefaults: false` in the `KubeletConfig` file, because t
$ oc get nodes -n openshift-compliance
----
+
.Example output
The command output displays the nodes in the cluster:
+
[source,terminal]
----
Expand All @@ -30,7 +31,7 @@ ip-10-0-171-170.us-east-2.compute.internal Ready master 5h21m v1.35.4
ip-10-0-197-35.us-east-2.compute.internal Ready master 5h22m v1.35.4
----

. Add a label to nodes.
. Add a label to the target nodes:
+
[source,terminal]
----
Expand All @@ -39,14 +40,14 @@ label node ip-10-0-166-81.us-east-2.compute.internal \
node-role.kubernetes.io/<machine_config_pool_name>=
----
+
.Example output
The console verifies that the node has been successfully labeled:
+
[source,terminal]
----
node/ip-10-0-166-81.us-east-2.compute.internal labeled
----

. Create custom `MachineConfigPool` CR.
. Create the custom `MachineConfigPool` custom resource (CR). Ensure that the `labels` field defines the label name you intend to add for the machine config pool:
+
[source,yaml]
----
Expand All @@ -55,7 +56,7 @@ kind: MachineConfigPool
metadata:
name: <machine_config_pool_name>
labels:
pools.operator.machineconfiguration.openshift.io/<machine_config_pool_name>: '' <1>
pools.operator.machineconfiguration.openshift.io/<machine_config_pool_name>: ''
spec:
machineConfigSelector:
matchExpressions:
Expand All @@ -64,9 +65,8 @@ spec:
matchLabels:
node-role.kubernetes.io/<machine_config_pool_name>: ""
----
<1> The `labels` field defines label name to add for Machine config pool(MCP).

. Verify MCP created successfully.
. Verify that the machine config pool was created successfully:
+
[source,terminal]
----
Expand Down
28 changes: 18 additions & 10 deletions modules/compliance-applying.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,35 @@
[id="compliance-applying_{context}"]
= Applying a remediation

[role="_abstract"]
The boolean attribute `spec.apply` controls whether the remediation should be applied by the Compliance Operator. You can apply the remediation by setting the attribute to `true`:

.Procedure

. Apply the remediation by setting the `spec.apply` attribute to `true`:
+
[source,terminal]
----
$ oc -n openshift-compliance \
patch complianceremediations/<scan-name>-sysctl-net-ipv4-conf-all-accept-redirects \
--patch '{"spec":{"apply":true}}' --type=merge
----
+
After the Compliance Operator processes the applied remediation, the `status.ApplicationState` attribute changes to *Applied* (or to *Error* if the configuration is incorrect).
+
When a machine config remediation is applied, it is rendered along with all other applied remediations into a single composite `MachineConfig` object named `75-$scan-name-$suite-name`. This object is subsequently processed by the Machine Config Operator and applied to all pool nodes by the machine control daemon instance running on each node.
+
[NOTE]
====
When the Machine Config Operator applies a new `MachineConfig` object to a pool, all nodes belonging to that pool are rebooted. To prevent immediate reboots when applying multiple remediations, you can pause the machine config pool by setting the `.spec.paused` attribute of the corresponding `MachineConfigPool` object to `true`.
====

After the Compliance Operator processes the applied remediation, the `status.ApplicationState` attribute would change to *Applied* or to *Error* if incorrect. When a machine config remediation is applied, that remediation along with all other applied remediations are rendered into a `MachineConfig` object named `75-$scan-name-$suite-name`. That `MachineConfig` object is subsequently rendered by the Machine Config Operator and finally applied to all the nodes in a machine config pool by an instance of the machine control daemon running on each node.

Note that when the Machine Config Operator applies a new `MachineConfig` object to nodes in a pool, all the nodes belonging to the pool are rebooted. This might be inconvenient when applying multiple remediations, each of which re-renders the composite `75-$scan-name-$suite-name` `MachineConfig` object. To prevent applying the remediation immediately, you can pause the machine config pool by setting the `.spec.paused` attribute of a `MachineConfigPool` object to `true`.

The Compliance Operator can apply remediations automatically. Set `autoApplyRemediations: true` in the `ScanSetting` top-level object.

. Optional: To configure the Compliance Operator to apply remediations automatically without manual intervention, set `autoApplyRemediations: true` in your top-level `ScanSetting` object.
+
[WARNING]
====
Applying remediations automatically should only be done with careful consideration.
Applying remediations automatically should only be done with careful consideration and prior testing.
====

+
[IMPORTANT]
====
The Compliance Operator does not automatically resolve dependency issues that can occur between remediations. Users should perform a rescan after remediations are applied to ensure accurate results.
====
1 change: 1 addition & 0 deletions modules/compliance-custom-node-pools.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="compliance-custom-node-pools_{context}"]
= Scanning custom node pools

[role="_abstract"]
The Compliance Operator does not maintain a copy of each node pool configuration. The Compliance Operator aggregates consistent configuration options for all nodes within a single node pool into one copy of the configuration file. The Compliance Operator then uses the configuration file for a particular node pool to evaluate rules against nodes within that pool.

.Procedure
Expand Down
1 change: 1 addition & 0 deletions modules/compliance-evaluate-kubeletconfig-rules.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="compliance-evaluate-kubeletconfig-rules_{context}"]
= Evaluating KubeletConfig rules against default configuration values

[role="_abstract"]
{product-title} infrastructure might contain incomplete configuration files at run time, and nodes assume default configuration values for missing configuration options. Some configuration options can be passed as command-line arguments. As a result, the Compliance Operator cannot verify if the configuration file on the node is complete because it might be missing options used in the rule checks.

To prevent false negative results where the default configuration value passes a check, the Compliance Operator uses the Node/Proxy API to fetch the configuration for each node in a node pool, then all configuration options that are consistent across nodes in the node pool are stored in a file that represents the configuration for all nodes within that node pool. This increases the accuracy of the scan results.
Expand Down
1 change: 1 addition & 0 deletions modules/compliance-filtering-results.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="filtering-compliance-check-results_{context}"]
= Filters for compliance check results

[role="_abstract"]
By default, the `ComplianceCheckResult` objects are labeled with several useful labels that allow you to query the checks and decide on the next steps after the results are generated.

List checks that belong to a specific suite:
Expand Down
22 changes: 14 additions & 8 deletions modules/compliance-inconsistent.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,25 @@
[id="compliance-inconsistent_{context}"]
= Inconsistent ComplianceScan

[role="_abstract"]
The `ScanSetting` object lists the node roles that the compliance scans generated from the `ScanSetting` or `ScanSettingBinding` objects would scan. Each node role usually maps to a machine config pool.

[IMPORTANT]
====
It is expected that all machines in a machine config pool are identical and all scan results from the nodes in a pool should be identical.
====
.Procedure

. Review your scan results and identify nodes that differ from the most common state.
+
The `ScanSetting` object lists the node roles that the compliance scans generated from the `ScanSetting` or `ScanSettingBinding` objects would scan. Each node role usually maps to a machine config pool.
+
IMPORTANT: It is expected that all machines in a machine config pool are identical and all scan results from the nodes in a pool should be identical.
+
If some of the results are different from others, the Compliance Operator flags a `ComplianceCheckResult` object where some of the nodes will report as `INCONSISTENT`. All `ComplianceCheckResult` objects are also labeled with `compliance.openshift.io/inconsistent-check`.
+
Because the number of machines in a pool might be quite large, the Compliance Operator attempts to find the most common state and list the nodes that differ from the common state. The most common state is stored in the `compliance.openshift.io/most-common-status` annotation and the annotation `compliance.openshift.io/inconsistent-source` contains pairs of `hostname:status` of check statuses that differ from the most common status. If no common state can be found, all the `hostname:status` pairs are listed in the `compliance.openshift.io/inconsistent-source` annotation.
+
If possible, a remediation is still created so that the cluster can converge to a compliant status. However, this might not always be possible and correcting the difference between nodes must be done manually.

Because the number of machines in a pool might be quite large, the Compliance Operator attempts to find the most common state and list the nodes that differ from the common state. The most common state is stored in the `compliance.openshift.io/most-common-status` annotation and the annotation `compliance.openshift.io/inconsistent-source` contains pairs of `hostname:status` of check statuses that differ from the most common status. If no common state can be found, all the `hostname:status` pairs are listed in the `compliance.openshift.io/inconsistent-source annotation`.

If possible, a remediation is still created so that the cluster can converge to a compliant status. However, this might not always be possible and correcting the difference between nodes must be done manually. The compliance scan must be re-run to get a consistent result by annotating the scan with the `compliance.openshift.io/rescan=` option:

. Force the compliance scan to re-run and obtain a consistent result by annotating the scan with the `compliance.openshift.io/rescan=` option:
+
[source,terminal]
----
$ oc -n openshift-compliance \
Expand Down
1 change: 1 addition & 0 deletions modules/compliance-kubeletconfig-sub-pool-remediation.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="compliance-kubeletconfig-sub-pool-remediation_{context}"]
= Remediating `KubeletConfig` sub pools

[role="_abstract"]
`KubeletConfig` remediation labels can be applied to `MachineConfigPool` sub-pools.

.Procedure
Expand Down
1 change: 1 addition & 0 deletions modules/compliance-manual.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="compliance-manual_{context}"]
= Remediating a platform check manually

[role="_abstract"]
Checks for Platform scans typically have to be remediated manually by the administrator for two reasons:

* It is not always possible to automatically determine the value that must be set. One of the checks requires that a list of allowed registries is provided, but the scanner has no way of knowing which registries the organization wants to allow.
Expand Down
38 changes: 12 additions & 26 deletions modules/compliance-removing-kubeletconfig.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="compliance-removing-kubeletconfig_{context}"]
= Removing a KubeletConfig remediation

[role="_abstract"]
`KubeletConfig` remediations are included in node-level profiles. In order to remove a KubeletConfig remediation, you must manually remove it from the `KubeletConfig` objects. This example demonstrates how to remove the compliance check for the `one-rule-tp-node-master-kubelet-eviction-thresholds-set-hard-imagefs-available` remediation.

.Procedure
Expand All @@ -17,7 +18,6 @@
$ oc -n openshift-compliance get remediation \ one-rule-tp-node-master-kubelet-eviction-thresholds-set-hard-imagefs-available -o yaml
----
+
.Example output
[source,yaml]
----
apiVersion: compliance.openshift.io/v1alpha1
Expand All @@ -28,7 +28,7 @@ metadata:
creationTimestamp: "2022-01-05T19:52:27Z"
generation: 1
labels:
compliance.openshift.io/scan-name: one-rule-tp-node-master <1>
compliance.openshift.io/scan-name: one-rule-tp-node-master
compliance.openshift.io/suite: one-rule-ssb-node
name: one-rule-tp-node-master-kubelet-eviction-thresholds-set-hard-imagefs-available
namespace: openshift-compliance
Expand All @@ -50,59 +50,45 @@ spec:
spec:
kubeletConfig:
evictionHard:
imagefs.available: 10% <2>
imagefs.available: 10%
outdated: {}
type: Configuration
status:
applicationState: Applied
----
<1> The scan name of the remediation.
<2> The remediation that was added to the `KubeletConfig` objects.
+
[NOTE]
====
+ [NOTE]
If the remediation invokes an `evictionHard` kubelet configuration, you must specify all of the `evictionHard` parameters: `memory.available`, `nodefs.available`, `nodefs.inodesFree`, `imagefs.available`, and `imagefs.inodesFree`. If you do not specify all parameters, only the specified parameters are applied and the remediation will not function properly.
====

. Remove the remediation:

.. Set `apply` to false for the remediation object:
. Set the `apply` attribute to `false` for the remediation object to stop applying it:
+
[source,terminal]
----
$ oc -n openshift-compliance patch \
complianceremediations/one-rule-tp-node-master-kubelet-eviction-thresholds-set-hard-imagefs-available \
-p '{"spec":{"apply":false}}' --type=merge
----
+
.. Using the `scan-name`, find the `KubeletConfig` object that the remediation was applied to:

. Using the `scan-name` value obtained previously, find the `KubeletConfig` object that the remediation was applied to:
+
[source,terminal]
----
$ oc -n openshift-compliance get kubeletconfig \
--selector compliance.openshift.io/scan-name=one-rule-tp-node-master
----
+
.Example output
[source,terminal]
----
NAME AGE
compliance-operator-kubelet-master 2m34s
----
.. Manually remove the remediation, `imagefs.available: 10%`, from the `KubeletConfig` object:

. Manually remove the configuration line `imagefs.available: 10%` from the identified `KubeletConfig` object:
+
[source,terminal]
----
$ oc edit -n openshift-compliance KubeletConfig compliance-operator-kubelet-master
----
+
[IMPORTANT]
====
All affected nodes with the remediation will be rebooted.
====

[NOTE]
====
You must also exclude the rule from any scheduled scans in your tailored profiles that auto-applies the remediation, otherwise, the remediation will be re-applied during the next scheduled scan.
====

**Important:** All affected nodes with the remediation will be rebooted.
+
**Note:** You must also exclude the rule from any scheduled scans in your tailored profiles that auto-applies the remediation. Otherwise, the remediation will be re-applied during the next scheduled scan.
22 changes: 10 additions & 12 deletions modules/compliance-results.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@
[id="compliance-results_{context}"]
= Obtaining Compliance Operator raw results from a persistent volume

.Procedure

[role="_abstract"]
The Compliance Operator generates and stores the raw results in a persistent volume. These results are in Asset Reporting Format (ARF).

.Procedure

. Explore the `ComplianceSuite` object:
+
[source,terminal]
Expand All @@ -18,7 +19,6 @@ $ oc get compliancesuites nist-moderate-modified \
-o json -n openshift-compliance | jq '.status.scanStatuses[].resultsStorage'
----
+
.Example output
[source,json]
----
{
Expand All @@ -44,11 +44,10 @@ This shows the persistent volume claims where the raw results are accessible.
$ oc get pvc -n openshift-compliance rhcos4-moderate-worker
----
+
.Example output
[source,terminal]
----
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
rhcos4-moderate-worker Bound pvc-548f6cfe-164b-42fe-ba13-a07cfbc77f3a 1Gi RWO gp2 92m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
rhcos4-moderate-worker Bound pvc-548f6cfe-164b-42fe-ba13-a07cfbc77f3a 1Gi RWO gp2 92m
----

. Fetch the raw results by spawning a pod that mounts the volume and copying the results:
Expand All @@ -58,7 +57,6 @@ rhcos4-moderate-worker Bound pvc-548f6cfe-164b-42fe-ba13-a07cfbc77f3a 1Gi
$ oc create -n openshift-compliance -f pod.yaml
----
+
.Example pod.yaml
[source,yaml]
----
apiVersion: "v1"
Expand Down Expand Up @@ -93,15 +91,15 @@ spec:
----
$ oc cp pv-extract:/workers-scan-results -n openshift-compliance .
----
+
[IMPORTANT]
====
Spawning a pod that mounts the persistent volume will keep the claim as `Bound`. If the volume's storage class in use has permissions set to `ReadWriteOnce`, the volume is only mountable by one pod at a time. You must delete the pod upon completion, or it will not be possible for the Operator to schedule a pod and continue storing results in this location.
====

. After the extraction is complete, the pod can be deleted:
+
[source,terminal]
----
$ oc delete pod pv-extract -n openshift-compliance
----
+
[IMPORTANT]
====
Spawning a pod that mounts the persistent volume will keep the claim as `Bound`. If the volume's storage class in use has permissions set to `ReadWriteOnce`, the volume is only mountable by one pod at a time. You must delete the pod upon completion, or it will not be possible for the Operator to schedule a pod and continue storing results in this location.
====
Loading