PromQL Query Basics

Observability / Guide 37 of 46 / Beginner

• Kubernetes1.36.4
• Cluster4 nodes
• Runtimecontainerd 2.2.6
• CNICalico v3.32.1
• TimeAbout 20 min

Metric names are stable across these versions; the numbers in the output are from this cluster and yours will differ.

• Chartkube-prometheus-stack 88.5.3
• Prometheus Operatorv0.93.1

This guide includes

Use this when Prometheus is running and you need to ask it something useful. This matters because the first questions worth asking are about Prometheus itself - which targets it is failing to scrape - and PromQL is how you ask them.

• querying the HTTP API from a Pod inside the cluster rather than through kubectl port-forward
• using up to count scrape targets and how many of them are failing
• grouping those failures by job, because a count of seven is not something anyone can act on
• writing two queries that answer questions about the workload rather than about Prometheus

Before you start

• kube-prometheus-stack running in a monitoring namespace. The install guide covers it.
• No port-forward needed - every query here runs from a Pod inside the cluster.

1. 1

   A Pod to query from

2. 2

   How many targets, and how many are broken

3. 3

   Group them, because a count is not a diagnosis

4. 4

   Two queries that answer real questions

Official sources

• Querying basics
• Aggregation operators
• HTTP API
• kube-state-metrics documentation