Domain 1: Databricks Machine Learning
- The Databricks Machine Learning Runtime preinstalls common ML libraries (scikit-learn, XGBoost, TensorFlow, MLflow); choose it over the standard runtime for ML work, and use a GPU runtime for deep learning.
- Use a single-node cluster for single-machine libraries (scikit-learn, pandas) and a multi-node (standard) cluster for distributed training with Spark ML.
- MLflow Tracking records an ML run: mlflow.start_run() opens a run, then log_param/log_metric/log_metrics/set_tag record inputs and results, and mlflow.<flavor>.log_model logs the trained model as an artifact.
- MLflow autologging (mlflow.autolog() or a flavor-specific mlflow.sklearn.autolog()) automatically captures parameters, metrics, and the model without explicit logging calls.
- An experiment groups runs; mlflow.set_experiment selects it, the MLflow UI compares runs side by side, and mlflow.search_runs(filter_string=..., order_by=...) queries runs programmatically to find the best one.
- Databricks AutoML is glass-box automated ML: it trains and tunes many models for classification, regression, or forecasting, and produces a data-exploration notebook plus an editable notebook for the best trial so you can reproduce and improve it.
- Retrieve AutoML results from the returned summary object (for example summary.best_trial and its model), and use AutoML as a strong baseline rather than a black box.
- Feature Engineering in Unity Catalog (the Feature Store) creates reusable feature tables keyed by primary keys, so features are consistent between training and serving and can be shared across models.
- Build a training set by joining features to labels with FeatureLookup and create_training_set; use point-in-time lookups with a timestamp key so a model only sees feature values available at prediction time (avoiding leakage).
- Register a model in the MLflow Model Registry or Models in Unity Catalog (mlflow.register_model or registered_model_name); manage versions with stages (Staging/Production/Archived) or aliases (Champion/Challenger), and load one with a URI like models:/name/version or models:/name@alias.
Domain 2: ML Workflows
- Start with exploratory data analysis: pandas .describe(), Spark .summary()/.describe(), and dbutils.data.summarize() give summary statistics, distributions, and a first read on data quality.
- Detect missing values (isNull/isna) and decide to drop or impute; impute with scikit-learn SimpleImputer or Spark ML Imputer using a strategy such as mean, median, or most_frequent (mode).
- Handle outliers (detection and capping) and remove duplicate rows with dropDuplicates (Spark) or drop_duplicates (pandas) as part of cleaning.
- Encode categorical features: Spark ML uses StringIndexer to index categories and OneHotEncoder to one-hot them; scikit-learn uses OneHotEncoder or OrdinalEncoder. Use one-hot for nominal categories so the model does not infer a false order.
- Scale numeric features when the algorithm is sensitive to magnitude (distance- or gradient-based): StandardScaler (zero mean, unit variance), MinMaxScaler (0-1), or RobustScaler (outlier-resistant).
- Bin or bucket continuous features with Bucketizer or QuantileDiscretizer, and apply log or power transforms to reduce skew.
- Spark ML estimators require a single feature vector column: assemble input columns into it with VectorAssembler before fitting.
- Split data into train/validation/test before fitting: scikit-learn train_test_split (test_size, stratify, random_state) or Spark randomSplit([0.8, 0.2], seed=...); always set the seed so the split is reproducible.
- Avoid target leakage: fit imputers, encoders, and scalers on the training set only, then apply them to validation/test - fitting on all the data leaks information.
- Move between engines as needed: toPandas() collects a Spark DataFrame to pandas (only when it fits in memory), and the pandas API on Spark (import pyspark.pandas as ps) scales pandas-style code across the cluster.
Domain 3: Model Development
- scikit-learn models follow fit(X, y) then predict(X); use single-node scikit-learn when the data fits on one machine.
- In Spark ML, a transformer has .transform() and an estimator has .fit() that returns a fitted Model (a transformer); a Pipeline chains transformer stages and a final estimator, and pipeline.fit() returns a PipelineModel used to transform new data.
- Frame the task correctly: regression predicts a continuous value (LinearRegression, RandomForestRegressor), classification predicts a class (LogisticRegression, DecisionTreeClassifier).
- Evaluate regression with RMSE, MAE, and R2, and classification with accuracy, precision, recall, F1, and ROC AUC; on imbalanced classes, accuracy misleads, so prefer F1, precision/recall, or AUC.
- Use the right Spark ML evaluator: RegressionEvaluator(metricName='rmse'|'r2'|'mae'), MulticlassClassificationEvaluator(metricName='accuracy'|'f1'), BinaryClassificationEvaluator(metricName='areaUnderROC').
- Cross-validation gives a more robust estimate than a single split: Spark ML CrossValidator takes an estimator, estimatorParamMaps, an evaluator, and numFolds (k-fold), while TrainValidationSplit uses a single trainRatio split.
- A grid of hyperparameters is built with ParamGridBuilder; k-fold CV over an N-combination grid trains k times N models, so grids and folds multiply cost.
- Tune hyperparameters efficiently with Hyperopt: fmin(fn, space, algo=tpe.suggest, max_evals, trials) searches a space (hp.uniform, hp.quniform, hp.choice), and the objective returns {'loss': ..., 'status': STATUS_OK} - since fmin minimizes, return the negative of a score you want to maximize.
- Use Hyperopt SparkTrials to distribute tuning of a single-machine model (for example scikit-learn) across the cluster, but use plain Trials when the model itself is already distributed (Spark ML).
- Ensembles usually beat a single tree: bagging (Random Forest) trains many trees in parallel to reduce variance, while boosting (gradient-boosted trees) trains sequentially to reduce bias; scale single-node training across groups with pandas function APIs (applyInPandas) or pandas UDFs.
Domain 4: Model Deployment
- Match the deployment to the requirement: batch inference for high-throughput, latency-tolerant scoring; streaming inference for continuous near-real-time scoring; real-time (online) serving for low-latency per-request predictions.
- For batch inference, load a logged or registered model with mlflow.pyfunc.load_model(model_uri), or create a Spark UDF with mlflow.pyfunc.spark_udf(spark, model_uri) and apply it to a Spark DataFrame to score at scale.
- Reference a model by URI: runs:/<run_id>/model for a run artifact, or models:/<name>/<version> and models:/<name>@<alias> for a registered version, then write predictions to a Delta table.
- For streaming inference, apply the model UDF inside a Structured Streaming query (readStream then writeStream) to score records continuously.
- For real-time inference, deploy a registered model version to a Databricks Model Serving endpoint - a serverless REST endpoint that returns low-latency predictions per request.
- Manage the production lifecycle in the Model Registry / Models in Unity Catalog: promote a version through stages (Staging to Production) or set aliases (Champion/Challenger), and load 'the Production' or 'the Champion' model by stage or alias rather than a hardcoded version.
- Roll back by pointing the Production stage or Champion alias at a previous version, and support champion/challenger or A/B comparison by serving multiple versions.
- Monitor deployed models for data drift and performance decay over time (comparing incoming data and prediction quality against training), and retrain when drift or metric degradation is detected.
- Log predictions and inputs from serving so monitoring and drift detection have data to compare against.
- Choose Model Serving over batch when an application needs an immediate single prediction, and batch (Spark UDF) when scoring large datasets on a schedule.
Databricks Certified Machine Learning Associate exam tips
- Weight your prep by the blueprint: Databricks Machine Learning (38%) and Model Development (31%) are two-thirds of the exam, so be fluent with MLflow tracking/autologging, AutoML, the Feature Store, Models in Unity Catalog, Spark ML pipelines, evaluation, cross-validation, and Hyperopt before the smaller ML Workflows (19%) and Model Deployment (12%) areas.
- Know the transformer-vs-estimator rule cold: in Spark ML a transformer has .transform() and an estimator has .fit() that returns a fitted Model; a Pipeline needs its inputs assembled into one features vector with VectorAssembler, and pipeline.fit() returns a PipelineModel.
- Master Hyperopt: fmin minimizes, so the objective returns {'loss': -score, 'status': STATUS_OK}; use SparkTrials to distribute tuning of a single-node (scikit-learn) model, but plain Trials for an already-distributed Spark ML model.
- Be precise about leakage and reproducibility: fit imputers/encoders/scalers on the training split only, use point-in-time Feature Store lookups so a model only sees features available at prediction time, and always pass a seed to randomSplit / random_state to train_test_split.
- For deployment, map the scenario to the paradigm - batch (mlflow.pyfunc.spark_udf over a Spark DataFrame) for throughput, Structured Streaming for continuous scoring, and Model Serving endpoints for low-latency real-time - and load production models by stage/alias (models:/name@Champion), never a hardcoded version.
Study guide FAQ
What is the format of the Databricks ML Associate exam?
It is 48 scored multiple-choice questions in 90 minutes, delivered online-proctored, with all code shown in Python. Databricks does not publish a fixed passing percentage, and the certification is valid for two years.
How much experience does it assume?
Databricks recommends about six months of hands-on experience performing ML tasks on Databricks: writing Python, using MLflow and AutoML, working with the Feature Store, and training models with scikit-learn and Spark ML. It is an associate (entry) level certification, but it is practical and API-focused.
Do I need to know Spark ML or is scikit-learn enough?
You need both. The exam covers single-node scikit-learn (fit/predict, Hyperopt with SparkTrials) and distributed Spark ML (transformers vs estimators, Pipeline, VectorAssembler, evaluators, CrossValidator). Knowing when to use single-node vs distributed training is itself tested.
What is the difference between this and the Data Engineer Associate?
The Data Engineer Associate is about building data pipelines (Delta Lake, ELT, Delta Live Tables, Unity Catalog governance). The Machine Learning Associate is about doing ML: MLflow, AutoML, the Feature Store, model training and tuning, and deploying models for inference. They are separate credentials in the same Databricks family.
Is CertGrid's practice official Databricks material?
No. CertGrid is an independent practice platform and is not affiliated with or endorsed by Databricks. These questions are original and written to mirror the current exam guide's four domains and Python, applied style so you can rehearse the objectives. Always confirm the current guide on the official Databricks certification page before your exam.