Domain 1: Design and implement an MLOps infrastructure
- Creating an Azure Machine Learning workspace provisions an associated storage account, key vault and Application Insights instance. To reuse an existing storage account you must supply it at workspace creation time, not afterwards.
- A datastore is a saved connection to storage whose secrets are held in the workspace key vault, so pipeline code references the datastore rather than embedding credentials.
- Identity-based access uses a managed identity and stores no secret at all, which is the preferred pattern over credential-based datastore access.
- A compute instance with idle shutdown suits interactive notebook work: it scales to zero after the idle period and compute billing stops.
- Data assets and environments are both versioned workspace assets, so a run can record exactly which data version and which environment it used.
- A registry shares assets - models, components, environments - across workspaces and regions, which is how a promotion path between dev, test and production is built.
- Parameterise environment-specific values and reuse a single template rather than maintaining one template per environment.
- For CI/CD, federate the pipeline identity to GitHub via OIDC and grant it an RBAC role on the workspace, so no long-lived secret is stored in the repository.
- Private endpoints combined with public network access disabled isolate the workspace; the managed virtual network then needs explicit outbound rules for anything it must reach.
- The managed network requires a private endpoint outbound rule to reach a storage account, and an FQDN outbound rule to reach an allowed public hostname.
- The AzureML Data Scientist role permits submitting and managing jobs without granting workspace administration.
- Keep code in Git and record the commit a run used, so a result can be traced back to the exact source that produced it.
- A component packages a reusable pipeline step together with its interface and its environment, which is what makes steps shareable across pipelines.
- To use internal container images, reference the registry image and grant the workspace managed identity the AcrPull role.
- A pipeline parameter declared without a default forces the caller to supply a value, which prevents a job silently running against the wrong dataset.
- Mount exposes a folder as a filesystem path and streams files on demand, which suits large datasets where download would be wasteful.
- An mltable adds a schema and load steps on top of paths, and can span multiple paths - use it when the data needs interpretation rather than just access.
- The az ml CLI extension is the command-line surface for Azure Machine Learning and is what pipelines typically invoke.
Domain 2: Implement machine learning model lifecycle and operations
- mlflow.autolog() captures framework parameters, metrics and the model itself without explicit logging calls, which is the fastest way to get a run fully tracked.
- The Azure Machine Learning workspace acts as an MLflow tracking server, so standard MLflow client code works against it unchanged.
- Tracking gives you cross-run comparison and model-to-run lineage - the ability to say which run produced a registered model and on what data.
- Accuracy rewards predicting the majority class on imbalanced data, so it is the wrong primary metric there; choose a metric that reflects the minority class.
- Both an experiment timeout and a maximum number of trials bound a hyperparameter search, and either can be the constraint that stops it.
- Bandit early-termination cancels trials trailing the best run by more than the configured slack, which concentrates compute on promising configurations.
- Grid sampling covers every combination of a discrete search space exhaustively, which is appropriate only when that space is small.
- Take data locations as job arguments rather than hardcoding them, so the caller decides where data comes from.
- Declare the distribution configuration in the job and write the training code to be distributed - both halves are required.
- Checkpoint to persistent storage and resume from the last checkpoint, so a pre-empted low-priority node does not cost the whole run.
- The MLmodel file carries the flavor and the signature that deployment relies on.
- A model signature declares input and output schemas, which is what lets a deployment validate requests rather than failing deep in the scoring script.
- Archiving a model version retains it but excludes it from normal listings - it is not a delete, and the version can still be referenced.
- Every registered version is retained, so a deployment can be repointed at an earlier version as a rollback.
- The Responsible AI dashboard combines fairness, error analysis and explanation views over a single model and dataset.
- Error analysis exposes cohorts and feature ranges where the model fails, which is what turns an aggregate score into something actionable.
- A feature store records the feature computation itself, so the transformation used at inference matches the one used in training.
- Record the resolved data versions and the exact environment with each run, because "latest" is not reproducible.
- Pipeline step reuse returns the cached output when inputs, code and environment are unchanged, which is why deterministic steps matter.
- A managed online endpoint serves low-latency real-time requests; a batch endpoint on a scale-to-zero cluster fits high-volume offline scoring.
Domain 3: Design and implement a GenAIOps infrastructure
- The common structure is one Azure AI Foundry resource with one project per team, because connections and evaluation results are project-scoped.
- Authenticate applications with a managed identity holding a data-plane role on the resource rather than with an API key.
- The Cognitive Services OpenAI User role permits inference without granting management rights over the resource.
- Add a private endpoint and disable public network access to isolate the resource; deployments then reach it over the private network only.
- Deploy resources with parameterised Bicep modules from a pipeline, and declare model deployments in the template so they are reproducible rather than clicked.
- Serverless (standard) endpoints bill per token with no compute to manage; provisioned throughput reserves dedicated capacity for predictable latency and volume.
- Managed compute is required when a model is not offered as a serverless endpoint.
- Choose a model on measured quality for your own task plus cost and latency at your own volume - published benchmarks do not settle it.
- Version prompts in Git and evaluate each revision against a fixed dataset, so behaviour is traceable to a revision and reviewable before it ships.
- Run a new prompt against the regression dataset before rolling it out, and treat variants as things to be compared on the same evaluation data.
- Pin a model version so provider updates do not silently change behaviour; when you move version, re-evaluate first and then roll out gradually.
- Create connections using managed identity authentication where supported, and keep any remaining credentials in Key Vault so rotation is centralised.
- Either accept cross-region latency or deploy in the caller's region - there is no third option, and the choice belongs in the design.
- Model quota is per subscription, per region and per model, and a deployment failure with capacity available elsewhere usually means that regional quota is exhausted.
- Rate limits are expressed as tokens per minute and requests per minute, and exceeding either returns a throttling error that the client must handle.
Domain 4: Implement generative AI quality assurance and observability
- Groundedness measures whether the response is supported by the retrieved context; relevance measures whether it answers the question. Both are reference-free, needing no ground-truth answer.
- An evaluation dataset provides the inputs and, where used, the ground truth, mapped to the fields each evaluator expects.
- An LLM-as-a-judge evaluator prompts a model to score a response against stated criteria, and must be calibrated against human ratings on a sample before its scores are trusted.
- When judge scores are inconsistent between runs, check the judge's temperature and the clarity of the rubric before concluding the system changed.
- Risk and safety evaluators cover the defined harm categories and are scored separately from quality.
- An average hides the rare severe failures that matter most - look at the distribution and the worst cases, not the mean.
- Hold the evaluation set fixed and include past failures alongside typical cases, so regressions in previously fixed behaviour are caught.
- Wire evaluation into CI as a pull request check with a failing threshold, and agree that bar for the use case before results are seen.
- A per-request trace shows the steps that produced an answer; a span is one timed unit of work inside that trace, and OpenTelemetry provides the model and instrumentation.
- Capture the retrieved chunks with their scores and the assembled prompt, or a bad answer cannot be diagnosed after the fact.
- Redact sensitive fields before traces are stored, because traces contain the user's input and the retrieved content.
- Score a representative sample of production traffic rather than every request, and track that score over time to see drift.
- A rising rate of ungrounded answers usually means retrieval quality has degraded and worse context is being supplied, not that the model changed.
- An answer that reads well but is not supported by the retrieved context is exactly what the groundedness evaluator is for.
- The latency measures that matter for a streaming experience are time to first token and total completion time, and they behave differently.
Domain 5: Optimize generative AI systems and model performance
- When answers are cut off mid-rule, enlarge the chunks or add overlap so context is not severed at a boundary. Chunks that are too large dilute the embedding and consume context budget - both directions have a cost.
- When irrelevant passages are retrieved, raise the similarity threshold or rerank the candidates rather than simply returning more of them.
- Hybrid search fuses vector similarity with keyword matching, which is what recovers exact identifiers - part numbers, error codes - that carry little semantic content for a vector search.
- Measure retrieval quality on labelled queries from your own corpus. Recall at k and mean reciprocal rank measure retrieval itself, separately from the generated answer.
- If retrieval metrics are healthy and answers are still wrong, the problem is in generation or context assembly, not in the index.
- Changing the embedding model without rebuilding the index leaves old and new vectors incompatible, and the failure is silent rather than an error.
- Rewrite or expand the query before retrieving when user vocabulary differs from the corpus, and resolve a conversational follow-up into a standalone query first.
- A reranker rescores candidates by examining the query and document together, which is more accurate than embedding similarity and is applied to a shortlist for cost reasons.
- Fine-tune for behaviour that needs many examples or high-volume consistency - a required output format, a house style, a classification convention.
- Fine-tuning encodes behaviour, not facts that change weekly. Changing knowledge belongs in retrieval.
- The strongest fine-tuning data is consistent demonstration of the exact target format; training on another model's output propagates that generator's errors and biases.
- Catastrophic forgetting is specialising at the cost of general ability, so evaluate a fine-tuned model on held-out data and check for regression outside the target task.
- LoRA trains small added matrices while freezing the base weights, which lowers training and storage cost and reduces capability loss relative to full fine-tuning.
- To cut cost and latency, send fewer but better-ranked chunks rather than trimming the response or lowering the model tier first.
- Filter at retrieval using the user's own permissions - never rely on instructing the model to withhold content it has already been given.
AI-300: Operationalizing ML and Gen AI exam tips
- Decide first whether a scenario is an infrastructure question or a lifecycle question. Workspace, networking, identity and compute belong to domain 1; anything about runs, metrics, registration and deployment belongs to domain 2, and they have different correct answers for similar-sounding problems.
- Prefer managed identity over keys in every scenario where both appear. Identity-based datastore access, OIDC federation for pipelines, and data-plane RBAC on the AI resource are all the exam's preferred answers.
- For a bad generative answer, work the pipeline in order: was the right passage retrieved, was it assembled into the prompt, and only then did the model reason poorly. Retrieval metrics healthy plus wrong answers means the problem is downstream of the index.
- Groundedness and relevance are reference-free; anything requiring ground truth needs a labelled dataset. Knowing which evaluators need what is a recurring distinction.
- Fine-tuning is for behaviour, retrieval is for knowledge. Any scenario where the facts change on a weekly cadence points to the retrieval pipeline, never to a training run.
Study guide FAQ
How is the AI-300 exam scored and structured?
A scaled score of 700 or greater out of 1000 is required to pass, with about 120 minutes for the exam. Questions are multiple-choice and multiple-select and may include case studies, drag-and-drop ordering and code-completion items typical of Microsoft role-based exams.
Which domain should I focus on most?
Implement machine learning model lifecycle and operations is the largest domain, followed by designing and implementing a GenAIOps infrastructure. Together they cover MLflow tracking, model registration and deployment, and the Azure AI Foundry side of the exam, which is where most of the questions concentrate.
Do I need to know both classical ML and generative AI?
Yes. Two domains cover Azure Machine Learning workspaces, MLflow, hyperparameter tuning and endpoints, and three cover generative AI - Foundry infrastructure, evaluation and tracing, and retrieval and model optimisation. Preparing for only one half leaves most of the exam unaddressed.
How much of the exam is code?
You are not asked to write substantial code, but you are expected to recognise SDK and CLI usage - MLflow logging calls, az ml commands, job and component YAML, and Bicep for deploying AI resources. Reading them accurately matters more than writing them from memory.