Skip to content

cs_encrypt.md

Latest commit

 

History

History
388 lines (307 loc) · 25.6 KB

cs_encrypt.md

File metadata and controls

388 lines (307 loc) · 25.6 KB
copyright
years
2014, 2020
lastupdated 2020-08-26
keywords kubernetes, iks, encrypt, security, kms, root key, crk
subcollection containers

{:DomainName: data-hd-keyref="APPDomain"} {:DomainName: data-hd-keyref="DomainName"} {:android: data-hd-operatingsystem="android"} {:apikey: data-credential-placeholder='apikey'} {:app_key: data-hd-keyref="app_key"} {:app_name: data-hd-keyref="app_name"} {:app_secret: data-hd-keyref="app_secret"} {:app_url: data-hd-keyref="app_url"} {:authenticated-content: .authenticated-content} {:beta: .beta} {:c#: data-hd-programlang="c#"} {:codeblock: .codeblock} {:curl: .ph data-hd-programlang='curl'} {:deprecated: .deprecated} {:dotnet-standard: .ph data-hd-programlang='dotnet-standard'} {:download: .download} {:external: target="_blank" .external} {:faq: data-hd-content-type='faq'} {:fuzzybunny: .ph data-hd-programlang='fuzzybunny'} {:generic: data-hd-operatingsystem="generic"} {:generic: data-hd-programlang="generic"} {:gif: data-image-type='gif'} {:go: .ph data-hd-programlang='go'} {:help: data-hd-content-type='help'} {:hide-dashboard: .hide-dashboard} {:hide-in-docs: .hide-in-docs} {:important: .important} {:ios: data-hd-operatingsystem="ios"} {:java: #java .ph data-hd-programlang='java'} {:java: .ph data-hd-programlang='java'} {:java: data-hd-programlang="java"} {:javascript: .ph data-hd-programlang='javascript'} {:javascript: data-hd-programlang="javascript"} {:new_window: target="_blank"} {:note: .note} {:objectc data-hd-programlang="objectc"} {:org_name: data-hd-keyref="org_name"} {:php: data-hd-programlang="php"} {:pre: .pre} {:preview: .preview} {:python: .ph data-hd-programlang='python'} {:python: data-hd-programlang="python"} {:route: data-hd-keyref="route"} {:row-headers: .row-headers} {:ruby: .ph data-hd-programlang='ruby'} {:ruby: data-hd-programlang="ruby"} {:runtime: architecture="runtime"} {:runtimeIcon: .runtimeIcon} {:runtimeIconList: .runtimeIconList} {:runtimeLink: .runtimeLink} {:runtimeTitle: .runtimeTitle} {:screen: .screen} {:script: data-hd-video='script'} {:service: architecture="service"} {:service_instance_name: data-hd-keyref="service_instance_name"} {:service_name: data-hd-keyref="service_name"} {:shortdesc: .shortdesc} {:space_name: data-hd-keyref="space_name"} {:step: data-tutorial-type='step'} {:subsection: outputclass="subsection"} {:support: data-reuse='support'} {:swift: #swift .ph data-hd-programlang='swift'} {:swift: .ph data-hd-programlang='swift'} {:swift: data-hd-programlang="swift"} {:table: .aria-labeledby="caption"} {:term: .term} {:tip: .tip} {:tooling-url: data-tooling-url-placeholder='tooling-url'} {:troubleshoot: data-hd-content-type='troubleshoot'} {:tsCauses: .tsCauses} {:tsResolve: .tsResolve} {:tsSymptoms: .tsSymptoms} {:tutorial: data-hd-content-type='tutorial'} {:unity: .ph data-hd-programlang='unity'} {:url: data-credential-placeholder='url'} {:user_ID: data-hd-keyref="user_ID"} {:vb.net: .ph data-hd-programlang='vb.net'} {:video: .video}

Protecting sensitive information in your cluster

{: #encryption}

Protect sensitive information in your {{site.data.keyword.containerlong}} cluster to ensure data integrity and to prevent your data from being exposed to unauthorized users. {: shortdesc}

You can create sensitive data on different levels in your cluster that each require appropriate protection.

  • Cluster-level: Cluster configuration data is stored in the etcd component of your Kubernetes master. Data in etcd is stored on the local disk of the Kubernetes master and is backed up to {{site.data.keyword.cos_full_notm}}. Data is encrypted during transit to {{site.data.keyword.cos_full_notm}} and at rest. You can choose to enable encryption for your etcd data on the local disk of your Kubernetes master by enabling a key management service provider for your cluster.
  • App-level: When you deploy your app, do not store confidential information, such as credentials or keys, in the YAML configuration file, configmaps, or scripts. Instead, use Kubernetes secrets{: external}, such as an imagePullSecret for registry credentials. You can also encrypt data in Kubernetes secrets to prevent unauthorized users from accessing sensitive app information.

For more information about securing your cluster and personal information, see Security for {{site.data.keyword.containerlong_notm}} and Storing personal information.

Overview of cluster encryption

{: #encrypt_ov}

The following image and description outline default and optional data encryption for {{site.data.keyword.containerlong_notm}} clusters. {: shortdesc}

Overview of cluster encryption

Figure: Overview of data encryption in a cluster

  1. Kubernetes master control plane startup: Components in the Kubernetes master, such as etcd, boot up on a LUKS-encrypted drive by using an IBM-managed key.
  2. Bring your own key (BYOK): When you enable a key management service (KMS) provider in your cluster, you can bring your own root key to create data encryption keys (DEKs) that encrypt the secrets in your cluster. The root key is stored in the KMS instance that you control. For example, if you use {{site.data.keyword.keymanagementservicelong_notm}}, the root key is stored in a FIPS 120-3 Level 3 hardware security module (HSM).
  3. etcd data: Etcd is the component of the master that stores the configuration files of your Kubernetes resources, such as deployments and secrets. Data in etcd is stored on the local disk of the Kubernetes master and is backed up to {{site.data.keyword.cos_full_notm}}. Data is encrypted during transit to {{site.data.keyword.cos_full_notm}} and at rest. When you enable a KMS provider, a wrapped data encryption key (DEK) is stored in etcd. The DEK encrypts the secrets in your cluster that store service credentials and the LUKS key. Because the root key is in your KMS instance, you control access to your encrypted secrets. To unwrap the DEK, the cluster uses the root key from your KMS instance. For more information about how key encryption works, see Envelope encryption.
  4. Worker node disks: The primary disk contains the kernel images that are used to boot your worker node, and is unencrypted. The secondary disk hosts the container file system, stores locally pulled images, and is encrypted at rest in ways that vary by infrastructure provider.
    • VPC infrastructure provider icon VPC only: The secondary disk is AES-256 bit encrypted at rest by the underlying VPC infrastructure provider. You cannot manage the encryption on your own with a KMS provider.
    • Classic infrastructure provider icon Classic only: The secondary disk is AES 256-bit encrypted with an IBM-managed LUKS encryption key that is unique to the worker node and stored as a secret in etcd. When you reload or update your worker nodes, the LUKS keys are rotated. If you enable a KMS provider, the etcd secret that holds the LUKS key is encrypted by the root key and DEK of your KMS provider.
  5. Cluster secrets: By default, Kubernetes secrets{: external} are base64 encoded. To manage encryption of the Kubernetes secrets in your cluster, you can enable a KMS provider. The secrets are encrypted by KMS-provided encryption until their information is used. For example, if you update a Kubernetes pod that mounts a secret, the pod requests the secret values from the master API server. The master API server asks the KMS provider to use the root key to unwrap the DEK and encode its values to base64. Then, the master API server uses the KMS provider DEK that is stored in etcd to read the secret, and sends the secret to the pod by using TLS.
  6. Persistent storage encryption: You can choose to store data by setting up file, block, object, or software-defined Portworx persistent storage. If you store your data on file or block storage, your data is automatically encrypted at rest. If you use object storage, your data is also encrypted during transit. With Portworx, you can choose to set up volume encryption to protect your data during transit and at rest. The IBM Cloud infrastructure storage instances save the data on encrypted disks, so your data at rest is encrypted.
  7. Data-in-use encryption: For select, SGX-enabled classic worker node flavors, you can use {{site.data.keyword.datashield_short}} to encrypt data-in-use within the worker node.

Understanding Key Management Service (KMS) providers

{: #kms}

You can protect the etcd component in your Kubernetes master and Kubernetes secrets by using a Kubernetes key management service (KMS) provider{: external} that encrypts secrets with encryption keys that you control. {: shortdesc}

What KMS providers are available by default? Can I add other providers?
{{site.data.keyword.containerlong_notm}} supports the following KMS providers:

Because adding a different KMS provider requires updating the managed master default configuration, you cannot add other KMS providers to the cluster.

Can I change the KMS provider?
You can have one KMS provider enabled in the cluster. You can switch the KMS provider, but you cannot disable KMS provider encryption after it is enabled. For example, if you enabled {{site.data.keyword.keymanagementservicefull}} in your cluster, but want to use Hyper Protect Crypto Services instead, you can enable Hyper Protect Crypto Services as the KMS provider.

You cannot disable KMS provider encryption. Do not delete root keys in your KMS instance, even if you rotate to use a new key. If you delete a root key that a cluster uses, the cluster becomes unusable, loses all its data, and cannot be recovered.

Similarly, if you disable a root key, operations that rely on reading secrets fail. Unlike deleting a root key, however, you can reenable a disabled key to make your cluster usable again.

With a KMS provider, do I control the encryption in my cluster?
Yes. When you enable a KMS provider in your cluster, your own KMS root key is used to encrypt data in etcd, including the LUKS secrets. Using your own encryption root key adds a layer of security to your etcd data and Kubernetes secrets and gives you more granular control of who can access sensitive cluster information. For more information, see the overview and your KMS provider's documentation, such as {{site.data.keyword.keymanagementserviceshort}} envelope encryption.

Can I use all the features of the KMS provider with my cluster?
Review the following known limitations:

  • Customizing the IP addresses that are allowed to connect to your {{site.data.keyword.keymanagementservicefull}} instance is not supported.

Encrypting the Kubernetes master's local disk and secrets by using a KMS provider

{: #keyprotect}

Enable a Kubernetes key management service (KMS) provider{: external} such as {{site.data.keyword.keymanagementserviceshort}}{: external} to encrypt the Kubernetes secrets and etcd component of your Kubernetes master. {: shortdesc}

To rotate your encryption key, repeat the CLI or console steps to enable KMS provider encryption with a new root key ID. The new root key is added to the cluster configuration along with the previous root key so that existing encrypted data is still protected. To encrypt your existing secrets with the new root key, you must rewrite the secrets. {: note}

Prerequisites

{: #kms_prereqs}

Before you enable a key management service (KMS) provider in your cluster, create a KMS instance and complete the following steps. {: shortdesc}

  1. Create a KMS instance, such as {{site.data.keyword.keymanagementserviceshort}} or Hyper Protect Crypto Services{: external}.

  2. Create a customer root key (CRK) in your KMS instance, such as a {{site.data.keyword.keymanagementserviceshort}} root key. By default, the root key is created without an expiration date.

    Need to set an expiration date to comply with internal security policies? Create the root key by using the API and include the expirationDate parameter. Important: Before your root key expires, you must repeat these steps to update your cluster to use a new root key. Otherwise, you cannot decrypt your secrets. {: tip}

  3. Make sure that you have the correct permissions to enable KMS in your cluster.

  4. Enable KMS encryption through the CLI or console.

Enabling or rotating KMS encryption through the CLI

{: #kms_cli}

You can enable a KMS provider or update the instance or root key that encrypts secrets in the cluster through the CLI. {: shortdesc}

  1. Complete the prerequisite steps to create a KMS instance and root key.

  2. Get the ID of the KMS instance that you previously created.

    ibmcloud ks kms instance ls

    {: pre}

  3. Get the ID of the root key that you previously created.

    ibmcloud ks kms crk ls --instance-id <KMS_instance_ID>

    {: pre}

  4. Enable the KMS provider to encrypt secrets in your cluster. Fill in the flags with the information that you previously retrieved. The KMS provider's private service endpoint is used by default to download the encryption keys. To use the public service endpoint instead, include the --public-endpoint flag. The enablement process can take some time to complete.

    During the enablement, you might not be able to access the Kubernetes master such as to update YAML configurations for deployments.

    ibmcloud ks kms enable -c <cluster_name_or_ID> --instance-id <kms_instance_ID> --crk <root_key_ID> [--public-endpoint]

    {: pre}

  5. Verify that the KMS enablement process is finished. The process is finished when that the Master Status is Ready.

    ibmcloud ks cluster get -c <cluster_name_or_ID>

    {: pre}

    Example output when the enablement is in progress:

    Name:                   <cluster_name>   
ID:                     <cluster_ID>   
...
Master Status:          Key Protect feature enablement in progress.

    {: screen}

    Example output when the master is ready:

    Name:                   <cluster_name>   
ID:                     <cluster_ID>   
...
Master Status:          Ready (1 min ago)

    {: screen}

    After the KMS provider is enabled in the cluster, data in etcd and new secrets that are created in the cluster are automatically encrypted by using your root key. {: note}

  6. To encrypt existing secrets with the root key, rewrite the secrets.

    1. Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.
    2. With cluster-admin access, rewrite the secrets. 
      kubectl get secrets --all-namespaces -o json | kubectl replace -f -
      {: pre}

  7. Optional: Verify that your secrets are encrypted.

Do not delete root keys in your KMS instance, even if you rotate to use a new key. If you delete a root key that a cluster uses, the cluster becomes unusable, loses all its data, and cannot be recovered.

Similarly, if you disable a root key, operations that rely on reading secrets fail. Unlike deleting a root key, however, you can reenable a disabled key to make your cluster usable again.

Enabling or rotating KMS encryption through the console

{: #kms_ui}

You can enable a KMS provider or update the instance or root key that encrypts secrets in the cluster through the {{site.data.keyword.cloud_notm}} console. {: shortdesc}

  1. Complete the prerequisite steps to create a KMS instance and root key.

  2. From the Clusters{: external} console, select the cluster that you want to enable encryption for.

  3. From the Overview tab, in the Summary > Key management service section, click Enable. If you already enabled the KMS provider, click Update.

  4. Select the Key management service instance and Root key that you want to use for the encryption.

    During the enablement, you might not be able to access the Kubernetes master such as to update YAML configurations for deployments.

  5. Click Enable (or Update).

  6. Verify that the KMS enablement process is finished. From the Summary > Master status section, you can check the progress. Example output when the enablement is in progress:

    Master status   KMS feature enablement in progress.

    {: screen}

    Example output when the master is ready:

    Master status   Ready

    {: screen}

    After the KMS provider is enabled in the cluster, data in etcd and new secrets that are created in the cluster are automatically encrypted by using your root key. {: note}

  7. To encrypt existing secrets with the root key, rewrite the secrets.

    1. Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.
    2. With cluster-admin access, rewrite the secrets. 
      kubectl get secrets --all-namespaces -o json | kubectl replace -f -
      {: pre}

  8. Optional: Verify that your secrets are encrypted.

Do not delete root keys in your KMS instance, even if you rotate to use a new key. If you delete a root key that a cluster uses, the cluster becomes unusable, loses all its data, and cannot be recovered.

Similarly, if you disable a root key, operations that rely on reading secrets fail. Unlike deleting a root key, however, you can reenable a disabled key to make your cluster usable again.

Verifying secret encryption

{: #verify_kms}

After you enable a KMS provider in your cluster, you can verify that your cluster secrets are encrypted by querying information that is in etcd in the master. If the returned information is encrypted, you know that the KMS provider works in your cluster. {: shortdesc}

Before you begin: Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.

  1. Install the etcd CLI (etcdctl) version 3 or higher.

    1. Download the release package for your operating system from the etcd project{: external}.
    2. Extract and move the etcdctl binary file to the location of your binary files, such as the following example. 
      mv Downloads/etcd-v3.4.6-darwin-amd64/etcdctl /usr/local/bin/etcdctl
      {: pre}
    3. Verify that etcdctl is installed. 
      etcdctl version
      {: screen}

  2. Set your terminal session context to use the appropriate etcdctl API version.

    export ETCDCTL_API=3

    {: pre}

  3. Run the ibmcloud ks cluster config command and include the --admin option, which downloads the etcd certificates and keys for your cluster, and the --output zip > <cluster_name_or_ID>.zip option, which saves your cluster configuration files to a compressed folder.

    ibmcloud ks cluster config -c <cluster_name_or_ID> --admin --output zip > <cluster_name_or_ID>.zip

    {: pre}

  4. Unzip the compressed folder.

  5. Get the server field for your cluster. In the output, copy only the master URL, without the https:// and node port. For example, in the following output, copy the c2.us-south.containers.cloud.ibm.com master URL only.

    cat ./<cluster_name_or_ID>/kube-config.yaml | grep server

    {: pre}

    Example output:

        server: https://c2.us-south.containers.cloud.ibm.com:30426

    {: screen}

  6. Get the etcdPort for your cluster.

    ibmcloud ks cluster get -c <cluster_name_or_ID> --output json | grep etcdPort

    {: pre}

    Example output:

        "etcdPort": "31593",

    {: screen}

  7. Get the name of a secret in your cluster.

    kubectl get secrets [-n <namespace>]

    {: pre}

  8. Confirm that the Kubernetes secrets for the cluster are encrypted. Replace the secret_name, master_url, and etcd_port fields with the values that you previously retrieved, and replace <cluster_name_or_ID> with the name of the cluster in your compressed folder file path.

    etcdctl get /registry/secrets/<secret_namespace>/<secret_name> --endpoints https://<master_url>:<etcd_port> --key="./<cluster_name_or_ID>/admin-key.pem" --cert="./<cluster_name_or_ID>/admin.pem" --cacert="./<cluster_name_or_ID>/ca.pem"

    {: pre}

    The output is unreadable and scrambled, indicating that the secrets are encrypted. Example output of encrypted secrets:

    k8s:enc:kms:v1:ibm:...=a?u???T?fE?pC?/?f|???'?z
?nI?a,)?
        9??O?{2??]="g?۳o??\5
?,a??AW??6Mx??x?5???7       dwX@DG8Dd?԰?ۭ#??[Y?ρF??????a$??9????_ˌ??m??Ɵϭ8?7????????c4L??q1?$0? ??yfzgl?}
                    ??Aynw#?$?J???p?x??pΝ???]ؖE6I?ө?o??t]??p?s?#0%BׇB?????k*֊ؖ??~?B??????V??

    {: screen}

    If you see a context deadline exceeded error, you might have a temporary connectivity issue. Check that your local etcdctl version matches the remote etcd version. Run the etcdctl get commmand with the --debug=true flag to see any additional information. Then wait a few minutes and try again. {: tip}

Encrypting data in classic clusters by using IBM Cloud Data Shield

{: #datashield}

{{site.data.keyword.datashield_short}} is integrated with Intel® Software Guard Extensions (SGX) and Fortanix® technology so that the app code and data of your containerized workloads are protected in use. The app code and data run in CPU-hardened enclaves, which are trusted areas of memory on the worker node that protect critical aspects of the app, which helps to keep the code and data confidential and unmodified. {: shortdesc}

Classic infrastructure provider icon Applies to only classic clusters. VPC clusters cannot have bare metal worker nodes, which are required to use {{site.data.keyword.datashield_short}}. {: note}

When it comes to protecting your data, encryption is one of the most popular and effective controls. But, the data must be encrypted at each step of its lifecycle for your data to be protected. During its lifecycle, data has three phases. It can be at rest, in motion, or in use. Data at rest and in motion are generally the area of focus when you think of securing your data. But, after an application starts to run, data that is in use by CPU and memory is vulnerable to various attacks. The attacks might include malicious insiders, root users, credential compromise, OS zero-day, network intruders, and others. Taking that protection one step further, you can now encrypt data in use.

If you or your company require data sensitivity due to internal policies, government regulations, or industry compliance requirements, this solution might help you to move to the cloud. Example solutions include financial and healthcare institutions, or countries with government policies that require on-premises cloud solutions.

To get started, provision an SGX-enabled bare metal worker cluster with a supported flavor for {{site.data.keyword.datashield_short}}.