Domain 1: Fundamentals of Observability
- Observability is the ability to ask new questions of a running system without shipping new code to answer them. Monitoring tells you whether known failure modes are occurring; observability lets you investigate a failure nobody predicted.
- OpenTelemetry exists to make instrumentation vendor-neutral. You instrument once against the OpenTelemetry API and choose a backend afterwards, which is what removes the lock-in of reinstrumenting every service to change observability vendor.
- The three signals are traces, metrics and logs. A trace shows the path and timing of one request across services, a metric is an aggregated numeric measurement over time, and a log is a timestamped record of a discrete event. Each answers a different question.
- Use metrics to notice that something is wrong and how widely, traces to find where in the request path it is wrong, and logs to find out exactly what happened at that point. Reaching for logs first is the common and expensive mistake.
- A resource describes the entity producing telemetry - the service, its version, the host, the container, the deployment environment - and it is attached to every signal that entity emits. service.name is the attribute everything else is grouped by, and it is required.
- Semantic conventions are the agreed names and meanings for attributes, so that http.request.method means the same thing regardless of which language or library produced it. They are what makes telemetry from different services comparable.
- Follow the conventions rather than inventing attribute names, because backends, dashboards and processors are built around them. A custom attribute that duplicates a conventional one produces data nothing else can use.
- Instrumentation comes in three forms: automatic, where an agent or library instruments common frameworks without code changes; library-native, where the library emits OpenTelemetry itself; and manual, where you write spans and measurements for your own business logic.
- Start with automatic instrumentation for breadth and add manual instrumentation for the parts that matter to your domain. Automatic instrumentation gives you the HTTP and database calls; only you can add the span that says which tenant the request was for.
- The specification, the API, the SDK and the contrib components are distinct. The API is what your code depends on, the SDK implements it and is configured at startup, and contrib holds the components maintained alongside the core.
- Signals reach stability independently and per language, so a signal that is stable in one SDK may be in beta in another. Check the status for the language you are using rather than assuming parity.
- A library instrumented with the OpenTelemetry API but running in an application that never configures an SDK emits nothing, at near-zero cost. That no-op default is deliberate: it makes it safe for libraries to depend on the API.
- Good telemetry has outcomes attached to it. Service level indicators and objectives turn signals into a target, and the RED method for request-driven services - rate, errors, duration - is the usual starting set of metrics to derive from traces.
- Cardinality is the constraint that shapes metric design: every distinct combination of attribute values is a separate time series. Putting a user ID or a request ID on a metric attribute is what makes a metrics bill unbounded.
Domain 2: The OpenTelemetry API and SDK
- The API and the SDK are separate on purpose. Application and library code calls the API; the application wires up the SDK once at startup to decide what actually happens to the data. That separation is why the API can be a safe dependency for libraries.
- A TracerProvider, MeterProvider and LoggerProvider are the SDK entry points. You obtain a tracer, meter or logger from the provider, and the provider holds the configuration - resource, processors, exporters and samplers.
- A span represents one operation with a start and end time, a name, a kind, a status, attributes, events and links. Spans form a trace through parent-child relationships carried in the context.
- Span kind describes the role in a call: SERVER and CLIENT for synchronous request-response, PRODUCER and CONSUMER for asynchronous messaging, and INTERNAL for work that does not cross a boundary. Backends use it to build service maps, so setting it wrongly distorts them.
- Span status is Unset by default, set to Error when the operation failed, and Ok only when something explicitly determines success. Recording an exception adds an event; it does not by itself set the status to Error.
- Span attributes describe the operation and should be set as early as possible, ideally at span creation, because a sampler that runs at start time cannot see attributes added later.
- Span events are timestamped points within a span, useful for marking something that happened during the operation. Span links relate a span to others outside its parent chain, which is how a batch consumer links to the many producer spans it processes.
- Always end every span, and prefer the language construct that guarantees it - a using block, a context manager, a defer. An unended span is never exported, which produces a trace that is silently missing work.
- Metric instruments differ by how a value is reported: a Counter only increases, an UpDownCounter can go both ways, a Histogram records a distribution of values, and Gauge records a current value. Choosing the wrong one produces meaningless aggregation.
- Synchronous instruments are called inline where the event happens; asynchronous or observable instruments register a callback that the SDK invokes at collection time, which is how you report a value that is expensive or continuous to read.
- A View reshapes what the SDK produces: renaming an instrument, changing its aggregation, dropping it entirely, or limiting which attribute keys are kept. Dropping attribute keys with a View is the supported way to control cardinality without changing application code.
- Aggregation temporality decides what an exported data point means: cumulative reports the total since start, delta reports the change since the last export. Prometheus expects cumulative, and several other backends expect delta, which is why the exporter dictates the choice.
- The logs signal is designed around a bridge rather than a new logging API for applications: you keep your existing logging library and attach an OpenTelemetry appender, so log records gain trace context and resource attributes automatically.
- Correlating logs with traces is the point of that bridge. A log record emitted inside an active span carries the trace and span IDs, which is what lets a backend jump from a slow trace straight to the log lines from that exact request.
- The SDK pipeline for traces is span processor then exporter. The simple processor exports each span as it ends, which suits development; the batch processor buffers and exports in batches, which is what production should use.
- The metrics pipeline uses a metric reader with an exporter, most commonly a periodic exporting reader that collects and pushes on an interval. Pull-based exporters such as Prometheus invert this and are scraped instead.
- Sampling decides which traces are recorded. Head sampling decides at the root before the trace exists, is cheap, and is configured in the SDK - ParentBased with a TraceIdRatioBased root is the usual production setting.
- A ParentBased sampler respects the upstream decision so a trace is sampled consistently across services. Different ratios configured in different services without ParentBased is how traces end up half-recorded.
- Context is the mechanism that carries the current span across function calls, and in most SDKs it is implicit - stored in a thread local, async local or equivalent - so instrumentation does not have to thread it through every signature.
- Baggage is separate from trace context: it carries arbitrary key-value pairs alongside the trace across service boundaries. It is not automatically added to spans as attributes, and because it travels over the wire to every downstream service, it should never carry sensitive data.
- Configure the SDK by environment variable where possible - OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_TRACES_SAMPLER, OTEL_RESOURCE_ATTRIBUTES - so the same build behaves correctly in every environment without code changes.
- Composability is a design principle of the SDK: samplers, processors, exporters, propagators and resource detectors are interfaces you can replace or wrap. A custom span processor that redacts an attribute is a supported extension point, not a hack.
- OTLP is the native protocol and the default choice, over gRPC or HTTP. Exporting directly to a vendor-specific protocol from the SDK is possible but reintroduces the coupling OpenTelemetry exists to remove - the Collector is the better place for that translation.
- Zero-code or agent-based instrumentation attaches at startup - a Java agent, a Python or Node wrapper, an operator-injected sidecar - and instruments supported libraries without touching source. It is the fastest route to coverage across an existing estate.
Domain 3: The OpenTelemetry Collector
- The Collector is a separate process that receives, processes and exports telemetry. It exists so that applications can emit OTLP and forget about backends, retries, batching, sanitisation and vendor formats - none of which belong in application code.
- A Collector configuration has receivers, processors, exporters, optionally connectors and extensions, and a service section. Nothing is active until it is referenced in a pipeline under service, which is the single most common configuration mistake.
- Pipelines are declared per signal - traces, metrics, logs - and each names its receivers, processors and exporters. Processors run in the order they are listed, which matters a great deal for memory limiting and batching.
- Receivers bring data in: the OTLP receiver for OpenTelemetry data over gRPC or HTTP, the Prometheus receiver to scrape existing endpoints, the filelog receiver to tail log files, and Jaeger and Zipkin receivers to accept legacy formats during a migration.
- Exporters send data out - OTLP to another Collector or a backend, plus vendor and Prometheus-format exporters. A single pipeline can fan out to several exporters, which is how you send the same telemetry to two backends during a migration.
- The batch processor groups telemetry before export to reduce request count and overhead, and it should be in essentially every production pipeline. Place it after memory_limiter and after any sampling.
- The memory_limiter processor protects the Collector from running out of memory by refusing data when it approaches a threshold, and it must be the first processor in the pipeline so it can act before memory is consumed by later stages.
- Attributes, resource and transform processors modify data in flight: adding an attribute, deleting one containing personal data, renaming to match semantic conventions. Redaction in the Collector is how you fix sensitive data without redeploying every service.
- OTTL, the OpenTelemetry Transformation Language, is what the transform and filter processors use to express conditions and statements over telemetry, so complex reshaping is configuration rather than a custom build.
- Tail sampling makes the keep-or-drop decision after a trace is complete, so it can keep every trace containing an error or exceeding a latency threshold - which head sampling cannot, because it decides before any of that is known.
- Tail sampling requires every span of a trace to reach the same Collector instance, which is why a scaled deployment puts a layer of load-balancing exporters in front, routing by trace ID.
- The two deployment patterns are agent and gateway. An agent runs next to the workload - as a sidecar or a DaemonSet - collecting local telemetry and host metadata; a gateway is a standalone scalable service that centralises processing, sampling and egress.
- Most production estates use both: agents close to workloads for fast local handoff and resource detection, forwarding to a gateway pool that does the expensive work and holds the backend credentials.
- Scale a gateway horizontally behind a load balancer, and remember which processors are stateful. Batching and attribute processing scale freely; tail sampling and span-to-metrics connectors need trace-aware routing.
- Connectors join two pipelines, consuming from one and emitting into another. The span metrics connector generates request rate, error and duration metrics from spans, which is how you get RED metrics without instrumenting them separately.
- Extensions add capabilities that are not part of a pipeline - health check endpoints, pprof profiling, zpages for live diagnostics, and authenticators for securing receivers.
- Secure the Collector as an ingress point: enable TLS on receivers, require authentication where it accepts data from outside a trusted network, and never expose an unauthenticated OTLP receiver to the internet - anyone can then write telemetry into your backend.
- The Collector comes in distributions. The core build carries a small supported component set and contrib carries a much larger one; the OpenTelemetry Collector Builder produces a custom binary containing only the components you actually use.
- Resource detection processors enrich telemetry with environment metadata - cloud provider, region, Kubernetes namespace and pod - which is generally better done in an agent near the workload than guessed at a central gateway.
- Filtering in the Collector is how you drop telemetry you have decided not to pay for: noisy health check spans, debug-level logs in production, or metrics nobody queries. Dropping at the agent saves the most, because the data never crosses the network.
Domain 4: Maintaining and Debugging Observability Pipelines
- Debug a pipeline in stages: confirm the application is producing data, confirm the Collector is receiving it, confirm it is passing through the pipeline, and confirm the exporter is succeeding. Guessing which stage is at fault is what makes these problems take hours.
- The debug exporter prints telemetry to the Collector log, which is the fastest way to prove data is arriving and see its shape. Add it to a pipeline temporarily; leaving it on in production is expensive and noisy.
- The zpages extension serves live diagnostic pages from the running Collector, showing active and recently completed spans and pipeline state, so you can inspect behaviour without restarting or exporting anywhere.
- The Collector emits its own internal telemetry - how many spans, metrics and log records were accepted, refused, sent and failed. Those counters are the primary evidence for where data is being lost.
- Refused records mean the Collector rejected data, usually because memory_limiter engaged. The fix is more Collector capacity or less data, not a bigger queue - the queue is already the thing under pressure.
- Exporters buffer with a sending queue and retry with backoff. When a backend is slow or down, the queue fills and then data is dropped, so a persistent export failure shows up as data loss some time after the failure began.
- Distinguish retryable from permanent errors. A timeout or a 503 is worth retrying; a 400 or an authentication failure will fail identically forever, and retrying it just delays the moment someone notices the configuration is wrong.
- When no data reaches the backend at all, check in this order: is the SDK configured with an endpoint, can it reach the Collector, is the receiver listening on that protocol and port, is the receiver referenced in a pipeline, and is the exporter authenticating successfully.
- A trace that stops at a service boundary is nearly always a propagation problem: the caller did not inject context, the callee did not extract it, a proxy stripped the traceparent header, or the two services are configured with different propagators.
- Spans appearing as roots when they should have parents means incoming context was not extracted. Every service in a trace must agree on the propagator, and the default W3C trace context is what everything should be using unless a legacy format is being migrated.
- Inconsistent sampling across services produces traces that are partly present. Use ParentBased sampling so the root decision is honoured downstream, rather than each service sampling independently.
- Context loss inside a single service is a different fault with the same symptom: crossing a thread pool, an async boundary or a callback without propagating context breaks the parent chain even though nothing left the process.
- Missing telemetry from short-lived processes is usually an export that never flushed. A batch job that exits immediately must shut the provider down so buffered spans are exported before the process ends.
- Schema URLs record which version of the semantic conventions a piece of telemetry follows, which is what lets a consumer interpret attribute names that have changed between versions.
- Semantic conventions evolve, and attribute renames are the disruptive part. Schema files describe the transformations between versions so telemetry emitted against an older version can still be understood, rather than every dashboard breaking on an upgrade.
- Plan a convention upgrade rather than doing it per service: agree the target version, migrate dashboards and alerts alongside the instrumentation, and use Collector transform rules to bridge the gap while both old and new attribute names are in flight.
OTCA exam tips
- The API and SDK domain is 46% - almost half the exam. Spans, metric instruments, the log bridge, processors, exporters, samplers, propagators and views are where the marks are, so start there rather than with observability theory.
- The Collector is 26% and is the part candidates most often under-prepare. Know the configuration shape (receivers, processors, exporters, connectors, extensions, service), that nothing runs until it is referenced in a pipeline, and that processors execute in listed order.
- Two ordering facts recur: memory_limiter must be the first processor, and batch should come after it and after any sampling. Questions frequently show a config and ask what is wrong.
- Know head sampling against tail sampling cold. Head sampling is in the SDK, decides before the trace exists, and is cheap; tail sampling is in the Collector, decides after the trace is complete, can keep all errors and slow traces, and needs every span on one instance.
- Learn the metric instruments by their semantics rather than their names: Counter only goes up, UpDownCounter goes both ways, Histogram records a distribution, Gauge records a current value. Then learn cumulative against delta temporality and which backends expect which.
- For any broken-trace scenario the cause is almost always propagation: context not injected, not extracted, stripped by a proxy, mismatched propagators, or lost across an async boundary inside one process.
- Remember that baggage is not automatically copied onto spans as attributes, and that it travels to every downstream service - so it is both a common trick question and a real security consideration.
- Cardinality is the recurring cost theme. High-cardinality attributes on metrics create unbounded time series, and a View that drops attribute keys is the supported fix that needs no application change.
Study guide FAQ
What is the format of the OTCA exam?
It is a 90-minute online proctored multiple-choice exam costing US$250, or US$495 bundled with a THRIVE-ONE annual subscription. Unlike the Kubernetes administrator exams it is not hands-on, so you are answering questions about OpenTelemetry rather than configuring anything in a live environment.
Which domain carries the most weight?
The OpenTelemetry API and SDK at 46%, followed by the Collector at 26%. Fundamentals of Observability is 18% and Maintaining and Debugging Observability Pipelines is 10%. Together the API/SDK and Collector domains are nearly three quarters of the exam.
Do I need to write code to pass OTCA?
You do not write code in the exam, but you need to understand what instrumentation code does. Expect questions about creating and ending spans, setting span kind and status, choosing a metric instrument, configuring a span processor and exporter, and wiring a propagator - so having actually instrumented an application in some language helps considerably more than reading about it.
What is the difference between the API and the SDK?
The API is the surface your application and library code calls - creating spans, recording measurements, emitting log records. The SDK is the implementation that decides what happens to that data: sampling, batching, exporting and the resource attached to it, all configured once at application startup. The split matters because a library can depend on the API safely: if the running application never configures an SDK, the API calls become near-zero-cost no-ops rather than errors.
Why would I run a Collector instead of exporting straight to a backend?
To keep backend concerns out of application code. A Collector gives you one place to batch, retry, sample, redact sensitive attributes, enrich with environment metadata, filter out telemetry you do not want to pay for, and translate to whatever formats your backends need - including fanning out to two backends during a migration. It also means changing any of that does not require redeploying every service.