All of the catalog
Scenario
Observability & SRE
Full observability stack with log aggregation, distributed tracing, alerting rules, and SLO dashboards.
ObservabilityVerifiedk3dkind
Definition on GitHubWhat you'll do
- Aggregate application logs in Loki and query them from Grafana
- Export traces through a collector (Grafana Alloy) into Tempo, instead of pointing the app at the backend
- Correlate the three signals: jump from a log line to its trace, and from a span back to that pod's logs
- Drive the stack with real load from k6 and compare client-side against server-side request metrics
- Alert on high error rate / latency and track SLOs on dashboards
Stages
- 1log-aggregation
Ship and store application logs
lokipromtail - 2tracing
Trace collector (Alloy) and trace backend (Tempo)
tempoalloywire-app-to-collector - 3alerting-and-slo
Alert rules and SLO dashboards
alerting-rulesslo-dashboards
Prerequisites
These are installed into the lab cluster for you — listed so you know what the scenario actually depends on.
ingressmonitoring/metricsmonitoring/grafanago-api
Related scenarios
- Autoscaling Under LoadWatch KEDA scale go-api on Prometheus RPS: a spike drives it from 1 to several replicas, then cooldown brings it back. The flagship 'autoscaling actually works' demo, verified under traffic from the load generator.
- Chaos EngineeringFailure injection with Chaos Mesh — pod kills, network delays, CPU/memory stress, partitions. Correlate chaos experiments with application metrics via Grafana dashboard.
- Cost & Capacity: Right-Sizing Over-Provisioned ResourcesDeploy go-api with deliberately over-provisioned CPU and memory requests, observe the inflated cost in OpenCost, then right-size the requests to a realistic baseline. Checks verify that requests were reduced to the target thresholds while go-api stays healthy under traffic. Teaches the observe-measure-reduce loop that keeps platform costs rational.