Create Kubernetes Secrets from HCP Vault Secrets with VSO

5min
|
HCP
Vault

The Vault Secrets Operator is a Kubernetes operator that continuously fetches secrets from HCP Vault Secrets and creates native Kubernetes secrets. Kubernetes workloads and users do not need to update workflows to adopt HCP Vault Secrets.

The Vault Secrets Operator syncs the secrets between HCP Vault Secrets and the Kubernetes secrets in a specified namespace. Within that namespace, applications have access to the secrets but the secrets are still managed by HCP Vault Secrets.

Prerequisites

An existing HCP account
Completed the previous HCP Vault Secrets tutorials
HCP service principal created at the org level with HCP_CLIENT_ID and HCP_CLIENT_SECRET available
minikube installed
Helm installed

Lab setup

To complete this tutorial, you must have information about your HCP Vault Secrets environment set as environment variables.

Verify you have HCP service principal credentials stored as environment variables from the Install HCP CLI for Vault Secrets tutorial.

$ echo ID=$HCP_CLIENT_ID\\nSecret=$HCP_CLIENT_SECRET

ID=FxqJRZabCd3fGh1jKaKVMybay3m
Secret=HoaxKFvG8QenprS3asam3stQnJbSXWWM5ab3rt4F2qWXbja33rnie

Retrieve the HCP organization ID and project ID selected during the hcp profile init process.

$ hcp profile display
name            = "default"
organization_id = "ab35ef-8d87-4443-a8a8-s3asam3st"
project_id      = "ab35ef-d3f4-4fda-b245-s3asam3st"

vault-secrets {
  app = "WebApplication"
}

Store the organization_id in a environment variable.

$ export HCP_ORG_ID=$(hcp profile display --format=json | jq -r .OrganizationID)

Store the project_id in a environment variable.

$ export HCP_PROJECT_ID=$(hcp profile display --format=json | jq -r .ProjectID)

Store the app name in a environment variable.

$ export APP_NAME=$(hcp profile display --format=json | jq -r .VaultSecrets.AppName)

Start minikube.

minikube is used to provision and manages the lifecycle of single-node Kubernetes cluster locally to allow you to follow the steps in this tutorial.

$ minikube start

😄  minikube v1.30.1 on Darwin 13.4 (arm64)
✨  Automatically selected the docker driver
📌  Using Docker Desktop driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🐳  Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
   ▪ Generating certificates and keys ...
   ▪ Booting up control plane ...
   ▪ Configuring RBAC rules ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
   ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🔎  Verifying Kubernetes components...
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

The initialization process takes several minutes as it retrieves any necessary dependencies and executes various container images.

Verify the status of the minikube cluster.

$ minikube status

minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured

Configure Kubernetes

Vault Secrets Operator is deployed using Helm to a Kubernetes environment. To use Vault Secrets Operator with HCP Vault Secrets you must have version 3.1 or higher installed.

Add the HashiCorp Helm repository.
```
$ helm repo add hashicorp https://helm.releases.hashicorp.com
```
Note
If you already have the HashiCorp Helm repository added, update the repository to ensure you have the latest version of the Vault Secrets Operator.
$ helm repo update hashicorp

Install the Vault Secrets Operator.

$ helm install vault-secrets-operator hashicorp/vault-secrets-operator \
     --namespace vault-secrets-operator-system \
     --create-namespace

Example output:

NAME: vault-secrets-operator
LAST DEPLOYED: Thu Sep 28 16:28:36 2023
NAMESPACE: vault-secrets-operator-system
STATUS: deployed
REVISION: 1

Create a Kubernetes secret for the HCP service principal credentials.

$ kubectl create secret generic vso-demo-sp \
    --namespace default \
    --from-literal=clientID=$HCP_CLIENT_ID \
    --from-literal=clientSecret=$HCP_CLIENT_SECRET

Example output:

secret/vso-demo-sp created

Configure Vault Secrets Operator with the HCP organization and project ID.

$ kubectl create -f - <<EOF
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: HCPAuth
metadata:
  name: default
  namespace: vault-secrets-operator-system
spec:
  organizationID: $HCP_ORG_ID
  projectID: $HCP_PROJECT_ID
  servicePrincipal:
    secretRef: vso-demo-sp
EOF

Example output:

hcpauth.secrets.hashicorp.com/default created

List the available Kubernetes secrets.
```
$ kubectl get secrets
NAME              TYPE     DATA   AGE
vso-demo-sp       Opaque   2      4m43s
```
The only secret that exists is the secret for the HCP service principal.
Create a Kubernetes secret from the HCP Vault Secret WebApplication created in a previous tutorial. This will include all key/value pairs in the HCP Vault Secrets application.
```
$ kubectl create -f - <<EOF
apiVersion: secrets.hashicorp.com/v1beta1
kind: HCPVaultSecretsApp
metadata:
  name: web-application
  namespace: default
spec:
  appName: $APP_NAME
  destination:
    create: true
    labels:
      hvs: "true"
    name: web-application
  refreshAfter: 1h
EOF
```
Example output:
```
hcpvaultsecretsapp.secrets.hashicorp.com/web-application created
```
For applications that do not support dynamically reloading rotated secrets, you can also include the rolloutRestartTargets parameter. This parameter provides the configuration required to perform a rollout-restart of the supported resources upon Vault Secret rotation.

List the available Kubernetes secrets.

$ kubectl get secrets
NAME              TYPE     DATA   AGE
vso-demo-sp       Opaque   2      4m43s
web-application   Opaque   2      1m24s

web-application is now listed.

Get the web-application secret.

$ kubectl get secrets web-application -o json
{
    "apiVersion": "v1",
    "data": {
        "_raw": "eyJzZWNyZXRzIjpbeyJjcm...snip...2ZXJzaW9uIjoiMSJ9fV19",
        "username": "ZGF0YWJhc2UtdXNlcg=="
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2023-09-29T17:52:14Z",
    ...snip...
    },
    "type": "Opaque"
}

The username secret created in the Create a secret in HCP Vault Secrets is listed as a base64 encoded value.

Retrieve and decode the username secret.
```
$ kubectl get secrets web-application -o jsonpath='{.data.username}' | base64 --decode
db-user
```
The value db-user, which was created in HCP Vault Secrets and synced to a Kubernetes secret using the Vault Secrets Operator, is displayed.
Your applications can now consume secrets natively in Kubernetes such as mounting the secret in a data volume or as an environment variable.

Help and reference

Access secrets with Terraform

Explore tutorial library