ConfigMaps and Secrets

Settings are objects in the cluster, handed to pods

ConfigMap api-configLOG_LEVEL: infoDB_HOST: dbFEATURE_X: "on"Secret db-loginpassword: ********Pod apienvironment variablesLOG_LEVEL=info DB_HOST=dbas variablesfiles/etc/secrets/passwordas a mounted volumeSettings live in the cluster as objects of their own, and are handed to pods at start.A ConfigMap holds ordinary settings and a Secret holds sensitive ones. A pod can receive eitheras environment variables or as files in a directory. Mounted files are refreshed when the objectchanges; environment variables are fixed for the life of the container.

A ConfigMap and a Secret, passed in as variables or files.

ConfigMaps and Secrets · Going deeper · 1 of 2

Lab: settings from a ConfigMap and a Secret

TypeLook forkubectl create configmap api-config --from-literal=LOG_LEVEL=infokubectl create secret generic db-login --from-literal=password=s3cr3tkubectl get secret db-login -o yamlecho czNjcjN0 | base64 -dkubectl apply -f api.yamlkubectl exec api -- env | grep -e LOG -e DB_configmap/api-config createdsecret/db-login createdpassword: czNjcjN0s3cr3t: encoded, not encryptedpod/api createdLOG_LEVEL=info and DB_PASSWORD=s3cr3tLab: give a pod its settings from a ConfigMap and a Secret.Save the manifest from the slide as api.yaml with "image: nginx", because acme/api is invented.Then "kubectl describe pod api" shows the requests, the limits and the pod's QoS class.

Create both, decode the Secret, then read the variables inside the pod.

When settings update · Going deeper · 2 of 2

Four ways in, and only one of them follows a change

The way inWritten in the pod asThe program seesAfter the ConfigMap changesOne variableenv: valueFrom: configMapKeyRef$LOG_LEVELno change until the pod is replacedEvery key as a variableenvFrom: configMapRefone variable per keyno change until the pod is replacedA mounted volumevolumes: configMap, volumeMountsone file per keyfiles updated, usually within a minute or twoOne file, by subPathvolumeMounts: subPatha single filenever updatedThe volume form: in the pod specand in the containervolumes:- name: config configMap: { name: api-config }volumeMounts:- name: config mountPath: /etc/configEnvironment variables are read once, at start. Only a mounted volume follows a change, and not through subPath.To make running pods take a new value, replace them: kubectl rollout restart deployment/api. A pod whose variables name aConfigMap or key that does not exist never starts, and shows CreateContainerConfigError. A ConfigMap or Secret holds at most1 MiB, and immutable: true locks one against edits, which also spares the kubelet from watching it.

Variables are fixed at start. A mounted volume is refreshed. A subPath file is not.

The pod manifest

A manifest names its settings; it does not contain them

apiVersion: v1kind: Podmetadata: name: apispec: containers: - name: api image: acme/api:2.3 envFrom: - configMapRef: { name: api-config } env: - name: DB_PASSWORD valueFrom: secretKeyRef: { name: db-login, key: password } resources: requests: { cpu: 250m, memory: 256Mi } limits: { memory: 512Mi }the image holds no settings at allevery key of the ConfigMap becomes a variableone variable, filled from the Secretrequests: what is set aside for itlimits: the most it may ever useA pod manifest names its settings. It does not contain them.The same manifest works in every namespace, each holding its own api-config and db-login.

envFrom a ConfigMap, a secretKeyRef, then requests and limits.

Secrets

A Secret is not secret by itself

password: czNjcjN0what the Secret storesbase64 -ds3cr3tthe password✗base64 is a way of writing bytes, not a lock: whoever can read the Secret can read the value✓access rules: let only the pods and people who need a Secret read it✓encryption at rest: have the API server encrypt Secrets before they reach etcd✓keep the real values in a secret manager, and never commit them to GitA Secret is not secret by itself. It is a place to which you can restrict access.The value of a Secret over a ConfigMap is that the cluster treats it more carefully: separateaccess rules, kept in memory on the node and never written to its disk, hidden in most output.The protection comes from what you configure around it.

base64 is not encryption. Access rules and encryption at rest are.

Secret types · Going deeper

A Secret has a type, and the type fixes the keys inside

typeKeys it must holdMade withOpaque (the default)anykubectl create secret generic db-login ...kubernetes.io/tlstls.crt, tls.keykubectl create secret tls web-tls --cert=... --key=...kubernetes.io/dockerconfigjson.dockerconfigjsonkubectl create secret docker-registry regcred ...kubernetes.io/basic-authusername, passworda manifestkubernetes.io/ssh-authssh-privatekeya manifestkubernetes.io/service-account-tokentoken, ca.crt, namespacea manifest: a long-lived token, rarely needed nowUsed by a pod: pull from a private registryUsed by an Ingress: HTTPSspec: imagePullSecrets: - name: regcredspec: tls: - { hosts: [shop.example.com], secretName: web-tls }The type is a promise about the keys inside, so that an Ingress controller or a kubelet knows what to read.In a manifest, values under data must be base64. Values under stringData are plain text, and are encoded for you on the wayin. A pull from a private registry without its Secret fails with ErrImagePull, and the pod's events say the pull was denied.

Opaque for anything, tls for certificates, dockerconfigjson for registries.

Requests and limits

A request is a promise; a limit is a ceiling

request 256Milimit 512Miuses 200Mi within its request: always availableuses 400Mi over its request: fine while the node has roomuses 540Mi over its limit: the container is killed (OOMKilled)A request is what a container is promised. A limit is the most it may take.The scheduler places a pod only on a node with enough unpromised capacity for its requests.Limits are enforced while it runs. Memory cannot be taken back, so a container over its memorylimit is killed and restarted. Processor time can, so one over its CPU limit is only slowed.

Under the request: guaranteed. Over the limit: killed.

Namespace limits · Going deeper

A namespace can supply defaults and cap the total

LimitRange: each containerResourceQuota: the whole namespaceapiVersion: v1kind: LimitRangemetadata: name: defaults namespace: shopspec: limits: - type: Container defaultRequest: { cpu: 100m, memory: 128Mi } default: { cpu: 500m, memory: 256Mi } max: { memory: 1Gi }apiVersion: v1kind: ResourceQuotametadata: name: team namespace: shopspec: hard: requests.cpu: "4" requests.memory: 8Gi limits.memory: 16Gi pods: "20"defaultRequest and default fill in what a container leaves out;default is the limit. max refuses anything larger.The sums over every pod in shop may not pass these.Used and hard: kubectl describe quota -n shop.A LimitRange fills in and bounds each container. A ResourceQuota caps the total for the namespace.Once a quota counts requests.cpu, a pod that states no CPU request is refused outright (403 Forbidden), unless a LimitRangesupplies the default. Both act only when a pod is created: pods already running are not touched. Units: 100m is a tenth ofa core, and 128Mi is 128 x 1,048,576 bytes. Memory written as 128m means 0.128 of a byte, a classic typing slip.

A LimitRange fills in each container. A ResourceQuota caps the sum.

Quality of service

When a node runs out, the pods that promised least go first

PodMemory request (solid) and limit (pale)ClassWhen the node runs outcachenothing setBestEffortevicted firstweb256Mi of 512MiBurstableevicted nextdb512Mi = 512MiGuaranteedevicted lastWhen a node runs out of memory, the pods that promised least are removed first.Kubernetes sorts pods into three classes from their requests and limits. A pod that sets none canuse anything spare, and is the first to go. A pod whose requests equal its limits is guaranteedits share. Always set requests for anything that matters.

BestEffort goes first, Burstable next, Guaranteed last.

Lesson complete

You finished “Configuration and secrets”

Nicely done.

Open in a new tab