Domain 1: Essential AI Knowledge
- Artificial intelligence is the outer field, machine learning is a family of methods within it, and deep learning is a family within that. Generative AI sits inside deep learning. Keeping that nesting straight is the fastest way to eliminate wrong options on definition questions.
- Supervised learning uses examples that already carry the correct answer, unsupervised learning finds structure in data with no labels, and reinforcement learning improves a policy from rewards received while acting. Regression predicts a number; classification predicts a category; clustering groups unlabelled data.
- Training adjusts weights from data and is periodic and expensive; inference applies the finished model to new inputs and runs continuously. Over a model's life the accumulated inference cost normally exceeds the cost of training it, which is why so much optimisation effort concentrates on serving.
- Overfitting shows as training error still falling while validation error turns upward. Dropout, data augmentation, regularisation and early stopping are the standard defences. Accuracy is misleading on imbalanced data, so precision, recall and the confusion matrix they come from are worth knowing precisely.
- The NVIDIA software stack layers upwards: the driver and CUDA expose the hardware, CUDA-X libraries such as cuDNN and RAPIDS accelerate common work, frameworks sit above those, and TensorRT, Triton, NeMo and NIM package models for optimisation and serving. NGC is the catalog they are distributed through.
- Transformers dominate modern AI because attention lets a whole sequence be processed in parallel and the core computation is dense matrix multiplication, which is exactly what accelerators are built for. Tokens are the unit of both context limits and cost, and the key value cache grows as generation proceeds.
- Retrieval augmented generation keeps knowledge outside the weights, so documents can be updated without retraining and answers can cite their source. Fine tuning changes the weights themselves, and parameter efficient methods train a small set of added weights over a frozen base model.
- Trustworthy AI in this exam means transparency to the people affected, bias that enters through unrepresentative data rather than through hardware, explainability of individual outputs, meaningful human oversight, and privacy obligations that attach to training data and to logged predictions alike.
Domain 2: AI Infrastructure
- A CPU has a few powerful cores optimised for one instruction stream; a GPU has many simpler cores that maximise total arithmetic across thousands of threads. The two are complementary, and a host that cannot prepare data fast enough will starve expensive accelerators.
- Tensor Cores perform matrix multiply and accumulate operations, which is why reduced precision formats such as FP16, BF16 and FP8 run so much faster than FP32. BF16 keeps the exponent range of FP32 with fewer mantissa bits, which is why it usually trains without loss scaling.
- Device memory is small and extremely fast because high bandwidth memory is stacked beside the processor. Parameters, activations and optimiser state all compete for it, so reducing batch size, gradient checkpointing, quantisation and splitting a model across devices are the levers when a job will not fit.
- NVLink and NVSwitch connect accelerators inside a system at bandwidths PCI Express cannot approach, which is why the first rule of scaling is to fill a node before spanning several. Data parallelism replicates the model, tensor parallelism splits a layer, and pipeline parallelism splits the layer sequence.
- Cluster fabrics carry collective traffic, so what matters is consistent low latency including at the tail and high bandwidth between many node pairs at once. InfiniBand is lossless with native remote direct memory access; tuned Ethernet with RoCE and platforms such as Spectrum-X serve the same role.
- Storage for training is sized on sustained parallel read throughput rather than capacity, and checkpoints arrive as large synchronised write bursts that average figures never predict. Millions of small files collapse throughput, so datasets are packed into large sequential shards.
- Sharing accelerators has two forms: hardware partitioning, which gives each instance its own compute and memory and therefore predictable performance and fault isolation, and software time slicing, which raises utilisation for cooperating users but provides no isolation at all.
- Accelerated racks draw tens of kilowatts, so power and cooling exhaust before floor space does. Direct-to-chip liquid cooling raises achievable density at the cost of plumbing and leak procedures, and power usage effectiveness measures the building rather than whether the hardware is doing useful work.
Domain 3: AI Operations
- The system management interface reports device utilisation, memory, temperature, power and the processes holding each device, and DCGM collects the same telemetry across a fleet. Reported utilisation counts intervals when any work was resident, so pair it with arithmetic activity before calling a device busy.
- Low accelerator utilisation with saturated host processors means the data pipeline is the limit, not the compute. Re-running against synthetic in-memory data isolates it in a single run: if throughput jumps, the input path was the bottleneck.
- Workload managers queue jobs and allocate resources. A declared time limit lets shorter work backfill gaps, gang scheduling starts every worker of a distributed job together, and preemption is only tolerable when the interrupted job checkpoints and resumes.
- Accelerator aware orchestration adds what a general purpose container platform lacks: gang start guarantees, fair share between teams, quotas and fractional device allocation. A node still needs a matching host driver and a container runtime configured to expose the devices.
- Troubleshooting order is measure, isolate, then change. A job that fails only on some nodes points at those nodes; a job that scales poorly points at placement or a silent fallback from remote direct memory access to a slower transport; a driver and toolkit mismatch fails immediately at start up.
- MLOps applies software delivery practice to models, with the extra difficulty that behaviour depends on data and can degrade with no code change. Retraining is triggered by detected drift or by schedule, and promotion is gated on a comparison against the incumbent on a fixed evaluation set.
- Production accuracy is usually unmeasurable in real time because true outcomes arrive late or never, so input distribution and confidence distribution serve as proxy signals. Logging the model version with every prediction is what makes a disputed answer explainable months later.
- Operating the platform means routine unglamorous work: access reviews, credential rotation, patching planned around drains rather than applied blindly, tested restores rather than untested backups, and follow-up actions from incidents tracked to completion by a named owner.
NVIDIA NCA-AIIO exam tips
- Learn where each NVIDIA product sits in the stack rather than what it does in detail. A large share of items are answered by knowing that CUDA is the platform, cuDNN and RAPIDS are libraries, TensorRT optimises for inference, Triton serves models, NeMo builds and customises language models, NIM packages them as microservices and NGC is the catalog.
- For any performance question, ask which resource is the limit before considering the answer. Low device utilisation with busy host processors is a data pipeline problem; poor multi node scaling is an interconnect or placement problem; a job that will not fit is a memory problem. The exam tests that diagnostic habit repeatedly.
- Memorise the difference between scaling up and scaling out, and the reason the first is preferred. Traffic inside a chassis over NVLink is faster and cheaper than traffic between chassis over the fabric, which is why you fill a node before spanning nodes.
- Know the three parallelism strategies by what they split: data parallelism splits the batch, tensor parallelism splits a layer, pipeline parallelism splits the sequence of layers. Tensor parallelism stays inside a node because it communicates several times per layer.
- Be able to separate hardware partitioning from software time slicing. Partitioning gives isolated compute and memory with predictable performance; time slicing takes turns on the whole device and offers no isolation. Questions about noisy neighbours and multi-tenant serving turn on that distinction.
- Treat power and cooling as first-class constraints. Plan racks in kilowatts rather than rack units, expect facility work to take longer than hardware procurement, and remember that power usage effectiveness rates the building while performance per watt rates the work.
- For operations items, prefer the answer that measures before it changes and restores service before it investigates. Profile before buying hardware, compare a suspect node against a healthy peer, and roll back a bad release first and analyse afterwards.
- Distinguish data drift, where the inputs change but the rule holds, from concept drift, where the relationship between inputs and the correct answer has itself moved. Both are detected through monitoring rather than through the model reporting a problem.
Study guide FAQ
What is the NCA-AIIO exam format and passing score?
NCA-AIIO is 50 multiple-choice questions in 60 minutes, delivered online with remote proctoring and available in English. NVIDIA does not publish a numeric passing score, so your result is reported simply as a pass or a fail. The certification is valid for two years, and NVIDIA suggests a basic understanding of data centre infrastructure as preparation.
How are the exam topics weighted?
NVIDIA publishes three topic areas: Essential AI Knowledge at 38 percent, AI Infrastructure at 40 percent and AI Operations at 22 percent. The weights sum to exactly 100, so roughly 19 of the 50 questions come from infrastructure, 19 from essential AI knowledge and 11 from operations. Our practice sets draw to the same distribution.
Do I need to be able to write code or train models to pass?
No. The exam is aimed at infrastructure and operations people rather than model developers, so it asks you to recognise concepts, identify which component does what, and reason about which resource is limiting a workload. You will not be asked to write CUDA kernels, define a network architecture or debug a training script.
How does NCA-AIIO differ from NCA-GENL?
NCA-GENL is the generative AI and large language model track, covering prompting, experimentation, data analysis and the GenAI software stack. NCA-AIIO covers the infrastructure underneath: GPU architecture, interconnect, networking, storage, the data centre, and the operations practices that keep a cluster and its models running. They overlap on essential AI concepts and the NVIDIA software stack.
How much detail about specific GPU models do I need?
Less than candidates expect. You should recognise architecture generation names and the broad product families, and understand which characteristics suit training against inference and against the edge, but the exam is not a specification quiz. Understanding why very large device memory and fast device-to-device links matter for training is far more useful than memorising figures.
What is the most common reason candidates fail?
Preparing only for the AI concepts and neglecting the infrastructure and operations halves, which together are 62 percent of the exam. Candidates from a data science background typically lose marks on networking, storage, power and cooling, and cluster operations; candidates from an infrastructure background lose them on training and inference fundamentals. Work through whichever half is less familiar.