Observability

Three signals answer three different questions

METRICS: NUMBERS OVER TIMEerrors per minuteIs something wrong?LOGS: WHAT HAPPENED14:02:07 ERROR db timeout14:02:07 WARN retry 1 of 314:02:09 ERROR db timeoutWhat exactly happened?TRACES: ONE REQUEST, END TO ENDapi, 400 msauth, 30 msdb, 340 msWhere is the time going?Observability rests on three signals: metrics, logs and traces.A metric shows that errors jumped. The logs say what failed. A trace follows one request acrossservices and shows which hop was slow. You need all three, linked together.

Metrics say something is wrong, logs say what, traces say where.

Observability · Going deeper

The observability tools

ToolSignalWhat it doesPrometheusmetricsScrapes and stores time series; queried with PromQLGrafanadashboardsDraws charts from Prometheus and many other sourcesFluentd, Fluent BitlogsCollect logs from every node and forward themJaegertracesStores and shows distributed tracesOpenTelemetryall threeOne standard and SDK to produce and collect telemetryOpenCostcostShows what each namespace and workload costsmetrics-serverCPU, memoryFeeds kubectl top and the autoscalers; not for historyPrometheus, Fluentd, Jaeger and OpenTelemetry are all CNCF projects.Telemetry is the data a system emits. Observability is being able to explain the system from it.Cost is treated as one more signal: FinOps is the practice of watching and owning cloud spend.

Prometheus, Grafana, Fluentd, Jaeger and OpenTelemetry.

Prometheus

Prometheus pulls metrics from every target

pod, serving /metricspod, serving /metricspod, serving /metricsPrometheuspulls the numbers,stores time seriesscrapes on a scheduleGrafanadashboards, using PromQL queriesAlertmanagergroups alerts, notifies on-callPrometheus pulls: it visits each target and collects its numbers on a schedule.Applications only expose a /metrics page. Prometheus finds the targets through the Kubernetes API,scrapes them, and stores each number with labels and a timestamp. Rules turn queries into alerts.

Prometheus scrapes /metrics. Grafana draws it. Alertmanager sends the alerts.

Prometheus · Going deeper

SLI, SLO, SLA and the roles

TermWhat it isExampleSLIindicator: a measured numbershare of requests answered within 300 msSLOobjective: the target for it99.9% over 30 daysSLAagreement: a promise to customersa refund when a month falls below 99.5%Error budgetwhat the SLO lets you miss0.1% of requests: spend it on changeRoles the exam namesSite reliability engineer (SRE)Owns reliability: SLOs, error budgets, on-call, automationPlatform engineerBuilds the internal platform that developers deploy ontoDevOps engineerJoins development and operations: pipelines and toolingSecurity engineer, DevSecOpsBuilds security into every stage of deliveryCloud architect, FinOpsDesigns the systems; watches and owns the cloud billAn SLI is measured, an SLO is aimed for, an SLA is promised.

An SLI is measured, an SLO is the target, an SLA is the promise.

Check yourself

api400 msauth30 msdb340 msone request, timed across three services

What is this, and what does it tell you?

Autoscaling

More pods, bigger pods, or more nodes

Horizontal Pod Autoscalermore podspodpodpodpodpodpodVertical Pod Autoscalerbigger podscpu 250mcpu 500mCluster Autoscalermore nodesnodenodenodenodenodeThree autoscalers: more pods, bigger pods, more nodes.The Horizontal Pod Autoscaler changes the replica count to hold a target, such as 60% CPU. TheVertical Pod Autoscaler changes the requests of each pod. The Cluster Autoscaler adds a node whenpods are Pending for lack of room, and removes nodes that sit empty.

HPA adds pods, VPA resizes them, the Cluster Autoscaler adds nodes.

Autoscaling · Going deeper

Autoscalers compared

AutoscalerChangesReacts toHorizontal Pod Autoscaler (HPA)the number of podsCPU, memory or custom metricsVertical Pod Autoscaler (VPA)the requests of each podwhat the pods have really usedCluster Autoscalerthe number of nodespods that cannot be scheduledKarpenterthe number and type of nodesthe same, choosing machines to fitKEDApods, from zero to manyevents: queue length, messages, schedulesKnativepods, from zero to manyHTTP requests: serverless on KubernetesThe HPA needs metrics-server, and every pod it scales needs resource requests.Horizontal scaling adds copies and suits stateless services. Vertical scaling makes one copy larger.The pod autoscalers and the node autoscaler work as a pair: more pods, then more room for them.

KEDA scales on events and Knative on requests, both from zero.

The CNCF

Projects climb from Sandbox to Graduated

Sandboxearly and experimentalIncubatingused in production, growingTOC voteGraduatedproven, audited, widely adoptedTOC voteKubernetes, Prometheus, Helm, ArgoA CNCF project climbs three levels: Sandbox, Incubating, Graduated.The Cloud Native Computing Foundation, part of the Linux Foundation, is the neutral home of theseprojects. Its Technical Oversight Committee (TOC) admits them and votes on each move up. Thelevel tells you how mature a project is, and so how safe it is to depend on.

Sandbox, Incubating, Graduated: the TOC votes on each step.

The CNCF · Going deeper

Who is who in the community

Who and whatCNCFVendor-neutral foundation that hosts the projects; part of the Linux FoundationTechnical Oversight Committee (TOC)Admits projects and decides their maturity levelGoverning BoardBudget, marketing and direction of the foundationTechnical Advisory Groups (TAGs)Experts who advise the TOC in an area, such as securitySpecial Interest Groups (SIGs)Own one part of Kubernetes: SIG Network, SIG StorageKubernetes Enhancement Proposal (KEP)The written proposal every Kubernetes feature starts asEnd usersCompanies that use the projects and do not sell themCNCF LandscapeThe map of projects and products, by categoryOpen governance: no single company controls a CNCF project.Decisions are made in public, in meetings, mailing lists and pull requests that anyone can join.

The TOC, TAGs, Kubernetes SIGs and KEPs.

Check yourself

Usersmany companies run it in productionAuditit has passed an independent security auditVotethe TOC has approved the top level

Which maturity level fits this project?

Serverless

Serverless follows the load, down to zero

instancesrequests/s002695200031A fixed fleet of 3 runs the whole time: 36 instance-minutesServerless runs only what the load needs: 11 instance-minutesServerless follows the load, all the way down to zero.You hand over code or a container, and the platform runs copies only while there is work, thennone. You pay for use, not for waiting. The price is a cold start: the first request after zerowaits for an instance. Knative and KEDA bring this to Kubernetes.

Run only while there is work, then scale to zero. Knative and KEDA do it on Kubernetes.

Serverless · Going deeper · 1 of 2

What "cloud native" means, and open standards

What "cloud native" means (CNCF definition)The goalScalable applications in public, private and hybrid cloudsThe techniquesContainers, service meshes, microservices, immutable infrastructure, declarative APIsThe resultLoosely coupled systems that are resilient, manageable and observableWith automationFrequent, predictable changes with little toilOpen standards keep it portableOCIImage, runtime and distribution specifications for containersCRI, CNI, CSIHow Kubernetes plugs in runtimes, networks and storageOpenTelemetryHow telemetry is produced and shippedCloudEventsA common format for describing eventsOpen standards mean you can change vendor without rewriting your application.

Resilient, manageable, observable, and built on open standards.

Serverless · Going deeper · 2 of 2

Clue words for cloud native architecture

When the question saysThe answer isRemember"numbers over time", "PromQL"Prometheuspull model, /metrics"follow one request across services"Tracing: Jaegerspans make a trace"vendor-neutral telemetry standard"OpenTelemetrymetrics, logs, traces"collect and forward logs"Fluentd, Fluent Bitruns on every node"visualise metrics in dashboards"Grafanareads many sources"add pods when CPU is high"Horizontal Pod Autoscalerneeds metrics-server"add nodes for Pending pods"Cluster Autoscalerremoves empty nodes too"scale on queue length, to zero"KEDAevent-driven"serverless on Kubernetes"Knativescale to zero"highest CNCF maturity level"Graduatedafter Incubating"decides project maturity"TOCTechnical Oversight Committee"propose a Kubernetes feature"KEPowned by a SIGDomain 4 is 12% of the exam, and most of it is knowing which project does what.

Prometheus, OpenTelemetry, the autoscalers and the CNCF levels.

Check yourself

Podsmore replicas were requested as load roseStatusthe new pods are Pending: no node has roomNeedsmore machines, added without a person

Which autoscaler solves this?

Lesson complete

You finished “Cloud native architecture”

Answer the questions along the way to see your score here.

Open in a new tab