DATABRICKS-MACHINE-LEARNING-ASSOCIATE Online Practice Questions and Answers

Correct Answer: D

In the context of Spark MLlib, an estimator refers to an algorithm which can be "fit" on a DataFrame to produce a model (referred to as a Transformer), which can then be used to transform one DataFrame into another, typically adding

predictions or model scores. This is a fundamental concept in machine learning pipelines in Spark, where the workflow includes fitting estimators to data to produce transformers.

References:

Spark MLlib Documentation:https://spark.apache.org/docs/latest/ml-pipeline.html#estimators

Correct Answer: E

The Root Mean Squared Error (RMSE) is a standard and widely used metric for evaluating the accuracy of regression models. The statement that it is invalid is incorrect. Here's a breakdown of why the other statements are or are not valid:

Transformations and RMSE Calculation:If the model predictions were transformed (e.g., using log), they should be converted back to their original scale before calculating RMSE to ensure accuracy in the evaluation. Missteps in this

conversion process can lead to misleading RMSE values. Accuracy of Models:Without additional information, we can't definitively say which model is more accurate without considering their RMSE values properly scaled back to the original

price scale.

Appropriateness of RMSE:RMSE is entirely valid for regression problems as it provides a measure of how accurately a model predicts the outcome, expressed in the same units as the dependent variable.

References:

"Applied Predictive Modeling" by Max Kuhn and Kjell Johnson (Springer, 2013), particularly the chapters discussing model evaluation metrics.

Correct Answer: A

The code block provided by the machine learning engineer will perform the desired inference when the Feature Store feature set was logged with the model at model_uri. This ensures that all necessary feature transformations and metadata

are available for the model to make predictions. The Feature Store in Databricks allows for seamless integration of features and models, ensuring that the required features are correctly used during inference.

References:

Databricks documentation on Feature Store: Feature Store in Databricks

Correct Answer: C

Pandas API on Spark (previously known as Koalas) provides a pandas-like API on top of Apache Spark. It allows users to perform pandas operations on large datasets using Spark's distributed compute capabilities. Internally, it uses Spark DataFrames and adds metadata that facilitates handling operations in a pandas-like manner, ensuring compatibility and leveraging Spark's performance and scalability.

References: pandas API on Spark documentation:https://spark.apache.org/docs/latest/api/python/user_guide/pandas _on_spark/index.html

Correct Answer: D

In the provided code block, theImputerobject is created but not fitted on the data to generate anImputerModel. Thetransformmethod is being called directly on the Imputerobject, which does not yet contain the fitted median values needed for imputation. The correct approach is to fit the imputer on the dataset first. Corrected code: imputer = Imputer( strategy="median", inputCols=input_columns, outputCols=output_columns ) imputer_model = imputer.fit(features_df)# Fit the imputer to the dataimputed_features_df = imputer_model.transform(features_df)# Transform the data using the fitted imputer References: PySpark ML Documentation

Correct Answer: C

The code block to compute the root mean-squared error (RMSE) for a linear regression model in Spark ML should use theRegressionEvaluatorclass withmetricNameset to "rmse". Given the schema ofpreds_dfwith

columnspredictionandactual, the correct evaluator setup will specifypredictionCol="prediction"andlabelCol="actual". Thus, the appropriate code block (Option C in your list) that usesRegressionEvaluatorto compute the RMSE is the correct

choice. This setup correctly measures the performance of the regression model using the predictions and actual outcomes from the DataFrame.

References:

Spark ML documentation (Using RegressionEvaluator to Compute RMSE).

Correct Answer: A

TheSparkTrials()is used to distribute trials of hyperparameter tuning across a Spark cluster. If the environment does not support Spark or if the user prefers not to usedistributed computing for this purpose, switching toTrials()would be

appropriate.Trials() is the standard class for managing search trials in Hyperopt but does not distribute the computation. If the user is encountering issues withSparkTrials()possibly due to an unsupported configuration or an error in the cluster

setup, usingTrials()can be a suitable change for running the optimization locally or in a non-distributed manner.

References:

Hyperopt documentation: http://hyperopt.github.io/hyperopt/

Correct Answer: B

To organize MLflow runs with one parent run for the tuning process and a child run for each unique combination of hyperparameter values, the data scientist can specifynested=Truewhen starting the child run. This approach ensures that

each child run is properly nested under the parent run, maintaining a clear hierarchical structure for the experiment. This nesting helps in tracking and comparing different hyperparameter combinations within the same tuning

process.References:

MLflow Documentation (Managing Nested Runs).

Correct Answer: C

Random Forest is a machine learning algorithm that typically uses bagging (Bootstrap Aggregating). Bagging involves training multiple models independently on different random subsets of the data and then combining their predictions.

Random Forests consist of many decision trees trained on random subsets of the training data and features, and their predictions are averaged to improve accuracy and control overfitting. This method enhances model robustness and

predictive performance.References:

Ensemble Methods in Machine Learning (Understanding Bagging and Random Forests).

Correct Answer: C

The method that transforms categorical features into a series of binary indicator variables is known as one-hot encoding. This technique converts each categorical value into a new binary column, which is essential for models that require

numerical input. One-hot encoding is widely used because it helps to handle categorical data without introducing a false ordinal relationship among categories.References:

Feature Engineering Techniques (One-Hot Encoding).