Domain 1: Set Up and Configure an Azure Databricks Environment
- Choose the compute type by workload rather than by habit: job compute is created for a job run and terminated afterwards, which is the cheapest option for scheduled work; all-purpose or classic compute stays up for interactive development; SQL warehouses serve BI and SQL queries; and serverless removes the startup wait and the capacity management entirely.
- Never run production jobs on all-purpose compute. It costs more per unit, stays running between runs, and couples unrelated workloads together - job compute or serverless is the expected answer for scheduled pipelines.
- Configure performance settings against the shape of the work: node type for the CPU and memory balance the workload needs, a minimum and maximum node count for autoscaling, and an auto-termination timeout so idle interactive clusters stop costing money.
- Autoscaling adds workers when tasks queue and removes them when they are idle, which suits variable workloads. A fixed-size cluster is more predictable for a steady job, and a pool of pre-warmed instances cuts cluster start time for frequent short jobs.
- Photon is a vectorised query engine that accelerates SQL and DataFrame operations, so it is worth enabling for scan- and aggregation-heavy work. Choose the Databricks Runtime version deliberately - an LTS release for stability, and the ML runtime only when the libraries it bundles are actually needed.
- Install libraries at the right scope: cluster-scoped libraries for something every job on that compute needs, notebook-scoped installs for experimentation, and a Databricks Asset Bundle or an init script for anything that must be reproducible in production.
- Control compute access with permissions: who can attach to a cluster, who can restart it, and who can manage it. Cluster policies constrain what users may create, which is how you stop a developer provisioning a hundred-node cluster by accident.
- Unity Catalog uses a three-level namespace of catalog, schema and table, replacing the old two-level hive_metastore. A fully qualified name is catalog.schema.object, and the metastore sits above catalogs at the account level.
- Use naming conventions to encode isolation: a catalog per environment such as dev, test and prod is the standard pattern, because catalog is the boundary at which access and sharing are most naturally granted.
- Volumes govern non-tabular files - landing-zone CSVs, images, models - inside Unity Catalog, so file access is governed by the same permission model as tables. Managed volumes are stored by Databricks; external volumes point at an existing storage location.
- Know what each object type gives you: a table stores data, a view is a stored query evaluated at read time, and a materialized view stores computed results and refreshes them, which trades storage and freshness for query speed.
- A foreign catalog surfaces an external system - another database or warehouse - inside Unity Catalog through a connection, so it can be queried and governed alongside native data without copying it first.
- Managed tables have their data and lifecycle owned by Unity Catalog, so dropping the table deletes the data. External tables register data at a location you control, so dropping the table leaves the files behind. That difference decides most DDL questions.
- Configure AI/BI Genie instructions so natural-language questions resolve correctly: supply the business context, the joins that make sense, and example questions, because Genie answers only as well as the metadata and instructions it is given.
Domain 2: Secure and Govern Unity Catalog Objects
- Unity Catalog privileges are granted to principals - users, groups or service principals - on securable objects, and inherit down the hierarchy: a grant on a catalog applies to its schemas and tables. Grant to groups, not individuals, so membership changes handle access changes.
- Reaching data needs more than one grant. A principal must have USE CATALOG on the catalog and USE SCHEMA on the schema before SELECT on the table means anything - a missing USE grant is the most common cause of an unexpected permission denied.
- Implement column-level access control with a dynamic view or a column mask so restricted users see a redacted value rather than an error, and row-level security with a row filter so each principal sees only the rows they are entitled to.
- Row filters and column masks are functions attached to a table, which means the policy lives with the data and applies to every query path rather than depending on users going through a particular view.
- Attribute-based access control uses tags on objects and policies that reference those tags, so a rule such as "mask everything tagged PII from this group" applies automatically to new columns that carry the tag. That scales where per-object grants do not.
- Read secrets from Azure Key Vault through a secret scope backed by the vault, and reference them with dbutils.secrets rather than embedding them in a notebook. Secret values are redacted in notebook output, which is why printing them is not a way around it.
- Authenticate to external data with a service principal for automated workloads and a managed identity where the Azure resource can carry its own identity, so no credential is stored at all. Storage credentials and external locations are how Unity Catalog holds that access centrally.
- Populate table and column comments and descriptions, because they are what makes a catalog discoverable and what AI/BI Genie and search rely on. Preserve them through DDL changes rather than losing them on every table rebuild.
- Data lineage is tracked automatically and viewed in Catalog Explorer: which tables and columns a table was derived from, which notebooks and jobs read and wrote it, and who owns it. Lineage is how you assess the blast radius of a schema change.
- Apply data retention policies so data is not kept longer than policy allows, and remember that Delta time travel and unvacuumed files keep historical versions available - retention has to account for those, not just the current table.
- Configure audit logging so access to governed data is recorded, then route the logs somewhere they can be queried. Audit answers who read what and when, which lineage and permissions alone do not.
- Delta Sharing shares live data with recipients outside your workspace or organisation without copying it. Design it securely: share the minimum objects, prefer recipient identities managed through the sharing identity federation, and apply row filters and column masks so the share inherits the restrictions.
- Ownership matters in Unity Catalog: the owner of an object can always grant on it. Set ownership to a group rather than an individual, so an object does not become unmanageable when its creator leaves.
Domain 3: Prepare and Process Data
- Design ingestion around the source: whether extraction is full or incremental, what file format arrives, how late and how often data lands, and whether the source can supply a change feed. Those answers decide the tool more than any preference does.
- Choose the ingestion tool by the source and the operating model: Lakeflow Connect for managed connectors to SaaS and database sources, notebooks with Auto Loader for files landing in storage, and Azure Data Factory when orchestration spans systems beyond Databricks.
- Choose batch when the requirement is periodic and cost-sensitive, and streaming when latency matters or the source is continuous. Structured Streaming with a trigger of AvailableNow gives incremental processing with batch-like economics, which is often the right middle ground.
- Delta Lake is the default table format and the one the exam assumes: ACID transactions, schema enforcement, time travel and efficient upserts on top of Parquet. Plain Parquet, CSV and JSON are interchange formats; Iceberg is supported for interoperability.
- Prefer liquid clustering over static partitioning for new Delta tables. Partitioning by a high-cardinality column creates small files and skew, and the partition scheme cannot be changed without rewriting - liquid clustering can be changed as query patterns evolve.
- Z-ordering colocates related data within files to improve data skipping on filtered columns, and deletion vectors let deletes and updates be recorded without rewriting whole files, which speeds up MERGE-heavy workloads.
- Know the slowly changing dimension types: type 1 overwrites and keeps no history, type 2 adds a new row with effective dates and a current flag, and type 3 keeps a previous-value column. Type 2 is the answer whenever the requirement is to report on history as it was.
- Choose granularity deliberately: the grain of a fact table is the single most consequential modelling decision, because aggregating later is easy and disaggregating later is impossible without reloading.
- A temporal or history table records how a row changed over time. Delta time travel gives you version history for recovery and auditing, but it is bounded by retention and is not a substitute for a modelled history table.
- Ingest with SQL where SQL is enough: CREATE TABLE AS SELECT for a one-off load, CREATE OR REPLACE TABLE for a full refresh that keeps the table's identity, and COPY INTO for idempotent incremental file loading that tracks which files it has already processed.
- Auto Loader incrementally processes new files as they arrive in storage, tracking state so files are not reprocessed, and scales to very large directories through file notification mode rather than repeatedly listing the directory.
- Ingest change data capture feeds with APPLY CHANGES in Lakeflow Spark Declarative Pipelines, which handles the out-of-order and delete semantics that a hand-written MERGE gets wrong. Structured Streaming reads Azure Event Hubs through the Kafka-compatible endpoint.
- Profile data before transforming it: summary statistics, distributions, distinct counts and null rates. That is how you discover that a supposedly unique key is not, before the MERGE that assumes it fails or silently duplicates.
- Resolve data quality problems explicitly rather than implicitly - deduplicate with a defined key and ordering, decide whether a null means unknown or zero, and choose column data types that match the domain rather than defaulting everything to string.
- Transform with the operators the exam names and know how they differ: join combines columns, union stacks rows, intersect keeps rows in both, and except keeps rows in the first and not the second. Watch whether duplicates are preserved.
- Denormalise for query performance, and pivot or unpivot to reshape between long and wide formats. Denormalisation trades storage and update complexity for read speed, which is usually the right trade in an analytics table and the wrong one in a source of truth.
- Load with the operation that matches the intent: append for immutable events, insert for new rows, and MERGE for upserts where a matching key should update and a new key should insert. MERGE is what makes an idempotent reload safe.
- Implement quality constraints rather than hoping: nullability and range checks, cardinality checks on keys, and data type checks at the boundary. Catching bad data at ingestion is far cheaper than finding it in a report.
- Delta enforces schema on write, so a mismatched column type fails rather than corrupting the table. Manage deliberate change with schema evolution, and handle schema drift from a source explicitly rather than by enabling permissive merging everywhere.
- Pipeline expectations in Lakeflow Spark Declarative Pipelines declare quality rules with an action: warn and record the violation, drop the offending rows, or fail the pipeline. Choosing the action per rule is what makes a quality strategy rather than a switch.
Domain 4: Deploy and Maintain Data Pipelines and Workloads
- Design the order of operations before building: source to bronze as raw ingested data, bronze to silver as cleansed and conformed, silver to gold as business-level aggregates. The medallion shape exists so each stage can be reprocessed independently.
- Choose between notebooks and Lakeflow Spark Declarative Pipelines by what you want to manage. Declarative pipelines infer the dependency graph, handle incremental processing and enforce expectations; notebooks give full control and are the right answer when logic does not fit the declarative model.
- Design job task logic as small tasks with explicit dependencies rather than one long notebook, so a failure can be repaired and rerun from the failed task instead of from the beginning.
- Implement error handling deliberately: fail fast on data quality violations that make downstream results wrong, and retry transient failures. A pipeline that swallows errors and completes successfully is worse than one that fails.
- Configure job triggers to match the requirement: scheduled on a cron expression, file arrival when new data lands in a location, continuous for always-on streaming, or triggered by another job's completion.
- Configure alerting on job outcomes - failure, success where it matters, and duration exceeding a threshold - and route them somewhere that is actually monitored. A long-running job that has not failed can still be the incident.
- Set automatic retries and restarts for transient failures, but bound them. Unlimited restarts on a deterministic failure burn compute all night and hide the problem rather than solving it.
- Use Git folders to keep notebooks and pipeline code in version control, working on branches and merging through pull requests. Production jobs should run from a specific branch or tag, not from a developer's working copy.
- Test data pipelines at more than one level: unit tests for transformation functions, integration tests for a pipeline stage against known input, end-to-end tests over a full run, and user acceptance testing against business expectations.
- Databricks Asset Bundles package notebooks, jobs, pipelines and their configuration as code in a YAML definition, with per-target overrides for dev, test and production. They are the supported way to promote work between environments.
- Deploy a bundle with the Databricks CLI - validate, then deploy, then run - or through the REST API from a CI/CD pipeline. That is what makes a deployment reproducible rather than a sequence of manual steps in the workspace.
- Monitor and manage cluster consumption so cost stays visible: watch utilisation against the configured size, use auto-termination, and check whether jobs are running on the compute type they should be. Idle all-purpose clusters are the usual source of surprise cost.
- Repair a failed Lakeflow job run rather than rerunning it whole: repair reruns only the failed tasks and their dependants, which saves both time and compute on a long pipeline.
- Diagnose Spark problems from the DAG in the Spark UI and the query profile, which show where time and data actually went. Look at stage durations, task skew within a stage, and shuffle read and write volumes before changing anything.
- Recognise the four classic Spark problems by their signature: skew, where a few tasks take far longer than the rest because a key dominates; spill, where data exceeds memory and goes to disk; shuffle, where a wide transformation moves data across the cluster; and caching, where recomputing the same dataset repeatedly costs more than persisting it.
- Fix skew by salting the key, by broadcasting the small side of a join, or by letting adaptive query execution handle it, rather than by adding nodes - more workers do not help when one task holds most of the data.
- Maintain Delta tables with OPTIMIZE to compact small files and VACUUM to remove files no longer referenced. VACUUM permanently removes the ability to time travel beyond the retention threshold, so it is not a purely cosmetic operation.
- Small files are the usual cause of a Delta table that gets slower over time, because every query pays the per-file overhead. Streaming ingestion with a short trigger interval is the usual cause of the small files.
- Stream Databricks logs to Log Analytics in Azure Monitor when the operations team monitors Azure centrally, and configure Azure Monitor alerts alongside the job-level alerts so platform problems and pipeline problems both surface.
DP-750: Azure Databricks Data Engineer exam tips
- The last two domains are 30-35% each, so roughly two thirds of the exam is preparing data and then running the pipelines that do it. Environment setup and Unity Catalog governance are 15-20% each.
- Unity Catalog's three-level namespace - catalog, schema, object - underpins most governance answers, and the commonest permission trap is forgetting that USE CATALOG and USE SCHEMA are needed before SELECT does anything.
- Know the managed against external table distinction cold. Dropping a managed table deletes the data; dropping an external table leaves the files. It decides DDL questions, migration questions and several governance questions.
- For new Delta tables, liquid clustering is the modern answer and static partitioning is usually the distractor - especially where the proposed partition column is high cardinality.
- Learn the four Spark performance problems by their symptoms: skew, spill, shuffle and caching. The exam describes a symptom in the Spark UI or DAG and expects you to name the cause and the right fix.
- Never run scheduled production work on all-purpose compute. Job compute or serverless is the expected answer whenever a question mentions a scheduled pipeline and cost in the same breath.
- Databricks Asset Bundles are the published way to package and promote work between environments, deployed with the CLI or REST API. Expect them in any question about development lifecycle or CI/CD.
- Distinguish the ingestion mechanisms precisely: COPY INTO for idempotent incremental file loads, Auto Loader for continuously arriving files, APPLY CHANGES for CDC feeds, and Lakeflow Connect for managed connectors to SaaS and database sources.
Study guide FAQ
What score do I need to pass DP-750?
A score of 700 or greater is required to pass, on Microsoft's scaled 1-1000 range, and it is not a simple percentage of questions answered correctly. Microsoft does not publish a fixed question count or duration for DP-750 on the study guide, and both can vary between forms. There is no penalty for a wrong answer.
How much Python and SQL do I need?
Enough to read and write both comfortably. Microsoft states that you need to know how to ingest and transform data using SQL and Python, and the published objectives include specific SQL forms such as CREATE TABLE AS SELECT, COPY INTO and MERGE, plus PySpark and Structured Streaming patterns. You are more often asked to pick or diagnose code than to write it from a blank page.
Which domain carries the most weight?
Prepare and process data, and deploy and maintain data pipelines and workloads, are tied at 30-35% each - about two thirds of the exam between them. Set up and configure an Azure Databricks environment and secure and govern Unity Catalog objects are 15-20% each.
What is the difference between a managed and an external table?
Unity Catalog owns both the metadata and the data lifecycle of a managed table, so dropping the table deletes the underlying files. An external table registers data at a storage location you control, so dropping the table removes only the metadata and leaves the files in place. Managed tables are the default recommendation because Databricks can then optimise and clean up the storage; external tables are for data that must remain accessible to other systems or is owned elsewhere.
Should I use notebooks or Lakeflow Spark Declarative Pipelines?
Declarative pipelines when the work fits a declarative dataset-and-dependency model: they infer the execution graph, handle incremental processing, and support expectations for data quality with warn, drop or fail actions. Notebooks in a Lakeflow Job when you need full procedural control, unusual logic, or orchestration steps that are not dataset transformations. Both are examinable, and the question usually signals which by whether it emphasises data quality and incremental refresh or bespoke logic.