Databricks Databricks-Certified-Data-Engineer-Associate Databricks Certified Data Engineer Associate Exam Exam Practice Test

When Automatically detect column types is disabled during file upload, Databricks does not infer specialized data types from the CSV values. Instead, every uploaded column is created as STRING. This behavior prevents values that resemble numbers, dates, or Boolean values from being automatically assigned potentially incorrect types. The engineer can subsequently transform or cast individual columns into the required target types as part of a controlled Silver-layer process. Option B is incorrect because partial numeric inference does not occur when automatic detection is disabled. Option C is incorrect because Databricks can create the table without manually supplied types by using STRING for every column. Option D is also incorrect because disabling detection determines the schema behavior for the table creation operation. Therefore, option A accurately describes the result.

================

Option B reads the relational source through JDBC and writes the resulting Spark DataFrame directly as a registered Unity Catalog table using its three-level name: catalog, schema, and table. Once written with saveAsTable, the data is immediately discoverable and queryable through Databricks SQL, subject to Unity Catalog permissions. Because the read and write operations occur through supported Databricks workloads, Unity Catalog can capture lineage between the source operation, notebook or job, and destination table. Writing files only to DBFS or an external path does not by itself create the required governed table. Introducing a separate staging and loading job adds unnecessary processing and operational complexity. A Pandas extraction also reduces scalability and does not satisfy the direct managed-table requirement. Therefore, option B is correct.

================

The Continuous Pipeline Mode for Delta Live Tables allows the pipeline to run continuously and process data as it arrives. This mode is suitable for streaming ingest and CDC workloads that require low-latency updates. The Development mode for Delta Live Tables allows the pipeline to run on a dedicated cluster that is not shared with other pipelines. This mode is useful for testing and debugging the pipeline logic before deploying it to production. Therefore, the correct answer is B, because the pipeline will run continuously on a dedicated cluster until it is manually stopped, and the compute resources will be released only after the pipeline is shut down. References: Databricks Documentation - Configure pipeline settings for Delta Live Tables, Databricks Documentation - Continuous vs. triggered pipeline execution, Databricks Documentation - Development vs. production mode.

Structured Streaming uses checkpointing and write-ahead logs to record the offset range of the data being processed in each trigger. This ensures that the engine can reliably track the exact progress of the processing and handle any kind of failure by restarting and/or reprocessing. Checkpointing is the mechanism of saving the state of a streaming query to fault-tolerant storage (such as HDFS) so that it can be recovered after a failure. Write-ahead logs are files that record the offset range of the data being processed in each trigger and are written to the checkpoint location before the processing starts. These logs are used to recover the query state and resume processing from the last processed offset range in case of a failure. References: Structured Streaming Programming Guide, Fault Tolerance Semantics

 A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. By using a data lakehouse, both the data analysis and data engineering teams can access the same data sources and formats, ensuring data consistency and quality across their reports. A data lakehouse also supports schema enforcement and evolution, data validation, and time travel to old table versions, which can help resolve data conflicts and errors1. References: 1: What is a Data Lakehouse? - Databricks 2: What is a data lakehouse? | IBM

In Python, the == operator is used to compare the values of two variables, while the = operator is used to assign a value to a variable. Therefore, option A and E are incorrect, as they use the = operator for comparison. Option B and C are also incorrect, as they compare the review_period variable to a string value  " True " , which is different from the boolean value True. Option D is the correct answer, as it uses the == operator to compare the day_of_week variable to the integer value 1, and the and operator to check if both conditions are true. If both conditions are true, then the final block of the Python program will be executed. References: [Python Operators], [Python If … Else]

Option D satisfies the requirements because a Lakeflow Connect managed SaaS connector provides source-specific authentication, incremental ingestion, schema evolution, and automated retries as managed capabilities. Managed connectors use serverless infrastructure and publish governed destination tables that can be processed downstream with Lakeflow Spark Declarative Pipelines. This avoids custom code for OAuth token handling, API pagination, CDC state management, retry behavior, and schema-change recovery. Option A still requires the engineering team to implement and maintain CDC and operational logic. Options B and C place most authentication, schema evolution, change tracking, and failure recovery responsibilities in custom notebooks or jobs, conflicting with the minimal-maintenance requirement. Therefore, the fully managed connector described in option D is the only solution that delivers the requested ingestion and operational behavior as an integrated platform capability.

================

 In a medallion architecture, a common data design pattern for lakehouses, data flows from Bronze to Silver to Gold layer tables, with each layer progressively improving the structure and quality of data. Bronze tables store raw data ingested from various sources, while Silver tables apply minimal transformations and cleansing to create an enterprise view of the data. Silver tables can also join and enrich data from different Bronze tables to provide a more complete and consistent view of the data. Therefore, option D is the correct answer, as Silver tables contain a more refined and cleaner view of data than Bronze tables. Option A is incorrect, as it is the opposite of the correct answer. Option B is incorrect, as Silver tables do not necessarily contain aggregates, but can also store detailed records. Option C is incorrect, as Silver tables may contain less data than Bronze tables, depending on the transformations and cleansing applied. Option E is incorrect, as Silver tables may contain more data than Bronze tables, depending on the joins and enrichments applied. References: What is a Medallion Architecture?, Transforming Bronze Tables in Silver Tables, What is the medallion lakehouse architecture?

A SQL Serverless Warehouse is the most suitable option for running SQL-based exploratory data analysis at scale while minimizing operational and cost overhead. Serverless SQL warehouses are fully managed by Databricks , meaning Databricks automatically provisions and scales compute resources, applies Photon acceleration , and optimizes query execution without requiring the user to manage infrastructure. This makes them ideal for workloads involving large numbers of ad hoc or interactive SQL queries , which is common during EDA. Because the data resides in a custom-defined cloud network , serverless SQL warehouses can still securely access governed data when Unity Catalog and networking requirements are satisfied. All-purpose clusters (option A) are designed for collaborative development and mixed workloads, not cost-efficient high-concurrency SQL querying. Classic SQL warehouses (option D) are legacy and lack the performance and cost optimizations of newer serverless offerings. Pro SQL warehouses (option B) require user-managed sizing decisions, whereas serverless warehouses optimize both performance and cost automatically. Databricks documentation recommends SQL Serverless Warehouses for fast, scalable, and cost-effective SQL analytics.

=========

The SQL keyword that can be used to convert a table from a long format to a wide format is PIVOT. The PIVOT clause is used to rotate the rows of a table into columns of a new table1. The PIVOT clause can aggregate the values of a column based on the distinct values of another column, and use those values as the column names of the new table1. The PIVOT clause can be useful for transforming data from a long format, where each row represents an observation with multiple attributes, to a wide format, where each row represents an observation with a single attribute and multiple values2. For example, the PIVOT clause can be used to convert a table that contains the sales of different products by different regions into a table that contains the sales of each product by each region as separate columns1.

The other options are not suitable for converting a table from a long format to a wide format. CONVERT is a function that can be used to change the data type of an expression3. WHERE is a clause that can be used to filter the rows of a table based on a condition4. TRANSFORM is a keyword that can be used to apply a user-defined function to a group of rows in a table5. SUM is a function that can be used to calculate the total of a numeric column.

1: PIVOT | Databricks on AWS

2: Reshaping Data - Long vs Wide Format | Databricks on AWS

3: CONVERT | Databricks on AWS

4: WHERE | Databricks on AWS

5: TRANSFORM | Databricks on AWS

[SUM | Databricks on AWS]

he approach that can be used to identify the owner of new_table is to review the Owner field in the table’s page in Data Explorer. Data Explorer is a web-based interface that allows users to browse, create, and manage data objects such as tables, views, and functions in Databricks1. The table’s page in Data Explorer provides various information about the table, such as its schema, partitions, statistics, history, and permissions2. The Owner field shows the name and email address of the user who created or owns the table3. The data engineer can use this information to contact the table owner and request for permission to access the table.

The other options are not correct or reliable for identifying the owner of new_table. Reviewing the Permissions tab in the table’s page in Data Explorer can show the users and groups who have access to the table, but not necessarily the owner4. Reviewing the Owner field in the table’s page in the cloud storage solution can be misleading, as the owner of the data files may not be the same as the owner of the table5. There is a way to identify the owner of the table, as explained above, so option E is false.

1: Data Explorer | Databricks on AWS

2: Table details | Databricks on AWS

3: Set owner when creating a view in databricks sql - Databricks - 9978

4: Table access control | Databricks on AWS

5: External tables | Databricks on AWS

The array functions from Spark SQL are a subset of the collection functions that operate on array columns1. They provide an ability to work with complex, nested data ingested from JSON files or other sources2. For example, the explode function can be used to transform an array column into multiple rows, one for each element in the array3. The array_contains function can be used to check if a value is present in an array column4. The array_join function can be used to concatenate all elements of an array column with a delimiter. These functions can be useful for processing JSON data that may have nested arrays or objects. References: 1: Spark SQL, Built-in Functions - Apache Spark 2: Spark SQL Array Functions Complete List - Spark By Examples 3: Spark SQL Array Functions - Syntax and Examples - DWgeek.com 4: Spark SQL, Built-in Functions - Apache Spark : Spark SQL, Built-in Functions - Apache Spark : [Working with Nested Data Using Higher Order Functions in SQL on Databricks - The Databricks Blog]

Databricks Asset Bundles are a Databricks CLI feature that lets teams define Databricks projects “as code” using YAML configuration files . In a bundle, metadata is expressed in YAML (for example, a required databricks.yml at the root), and the configuration can declare resources (such as Databricks Jobs/workflows and other supported assets), artifacts , and deployment targets . After defining these YAML manifests, teams use the Databricks CLI to validate, deploy, and run the bundle into a Databricks workspace, which matches the scenario described (YAML + CLI deployment). This approach standardizes how workflows and related resources are packaged and promoted across environments (dev/test/prod) while keeping definitions version-controlled and reproducible. GitHub is only a hosting/version-control platform (not the YAML deployment feature itself), Terraform is a separate IaC tool, and “DataOps” is a methodology rather than a specific Databricks feature. ( Databricks Documentation )

Option A is correct because Lakeflow Connect provides a built-in, fully managed Salesforce connector. It ingests Salesforce data into Databricks, supports incremental processing, and publishes governed destination tables in Unity Catalog. Managed SaaS connectors handle source-specific authentication, incremental reads, schema evolution, and automated retries, reducing the custom engineering required for API pagination, state tracking, and failure recovery. Calling the Salesforce REST API from a notebook would require the team to develop and maintain those capabilities itself. Auto Loader is designed to discover and incrementally process files arriving in cloud object storage; it does not connect directly to Salesforce. Using an exported Amazon S3 path would also require a separate export system outside the requested managed Databricks ingestion experience. Therefore, option A is the low-code capability that directly satisfies the Salesforce ingestion requirement.

================

To minimize the total running time of the SQL endpoint used in the refresh schedule of a dashboard in Databricks, the most effective approach is to utilize the Auto Stop feature. This feature allows the SQL endpoint to automatically stop after a period of inactivity, ensuring that it only runs when necessary, such as during the dashboard refresh or when actively queried. This minimizes resource usage and associated costs by ensuring the SQL endpoint is not running idle outside of these operations.

The best action that the data engineer can perform to improve the start up time for the clusters used for the Job is to use clusters that are from a cluster pool. A cluster pool is a set of idle clusters that can be used by jobs or interactive sessions. By using a cluster pool, the data engineer can avoid the cluster creation time and reduce the latency of the tasks. Cluster pools also offer cost savings and resource efficiency, as they can be shared by multiple users and jobs.

Option A is not relevant, as endpoints available in Databricks SQL are used for creating and managing SQL analytics workloads, not for improving cluster start up time.

Option B is not correct, as jobs clusters and all-purpose clusters have similar start up times. Jobs clusters are clusters that are dedicated to run a single job and are terminated when the job is completed. All-purpose clusters are clusters that can be used for multiple purposes, such as interactive sessions, notebooks, or multiple jobs. Both types of clusters can benefit from using a cluster pool.

Option C is not advisable, as configuring the clusters to be single-node will reduce the parallelism and performance of the tasks. Single-node clusters are clusters that have only one worker node and are typically used for testing or development purposes. They are not suitable for running production jobs that require high scalability and fault tolerance.

Option E is not helpful, as configuring the clusters to autoscale for larger data sizes will not affect the start up time of the clusters. Autoscaling is a feature that allows clusters to dynamically adjust the number of worker nodes based on the workload. It can help optimize the resource utilization and cost efficiency of the clusters, but it does not speed up the cluster creation process.

Cluster Pools

Jobs

Clusters

[Databricks Data Engineer Professional Exam Guide]

Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as cloning, committing, pushing, pulling, and branch management. Databricks Notebooks versioning is a legacy feature that allows users to link notebooks to GitHub repositories and perform basic Git operations. However, Databricks Notebooks versioning does not support the use of multiple branches for development work, which is an advantage of using Databricks Repos. With Databricks Repos, users can create and manage branches for different features, experiments, or bug fixes, and merge, rebase, or resolve conflicts between them. Databricks recommends using a separate branch for each notebook and following data science and engineering code development best practices using Git for version control, collaboration, and CI/CD. References: Git integration with Databricks Repos - Azure Databricks | Microsoft Learn, Git version control for notebooks (legacy) | Databricks on AWS, Databricks Repos Is Now Generally Available - New ‘Files’ Feature in …, Databricks Repos - What it is and how we can use it | Adatis.

In a Databricks customer ' s cloud account, the primary elements stored include:

Data: This is the central type of content stored in the customer ' s cloud account. Data might include various datasets, tables, and files that are used and managed through Databricks platforms.

Notebooks: These are also stored within a customer ' s cloud account. Notebooks include scripts, notes, and other information necessary for data analysis and processing tasks.

Cluster management metadata is indeed managed through the cloud, but it ' s primarily handled by Databricks rather than stored directly in the customer ' s account. The Databricks web application itself is not stored within the customer ' s cloud account; rather, it ' s a service provided by Databricks.

Bronze tables are the first layer of a medallion architecture, which is a data design pattern used to organize data in a lakehouse. Bronze tables contain raw data ingested from various sources, such as RDBMS data, JSON files, IoT data, etc. The table structures in this layer correspond to the source system table structures “as-is”, along with any additional metadata columns that capture the load date/time, process ID, etc. The only transformation applied to the raw data in this layer is to apply a schema, which defines the column names and data types of the table. The schema can be inferred from the data source or specified explicitly. Applying a schema to the raw data enables the use of SQL and other structured query languages to access and analyze the data. Therefore, option E is the correct answer. References: What is a Medallion Architecture?, Raw Data Ingestion into Delta Lake Bronze tables using Azure Synapse Mapping Data Flow, Apache Spark + Delta Lake concepts, Delta Lake Architecture & Azure Databricks Workspace.

To sync a Databricks Repo with the changes from a central Git repository, the data engineer needs to run the Git pull operation. This operation fetches the latest updates from the remote repository and merges them with the local repository. The data engineer can use the Pull button in the Databricks Repos UI, or use the git pull command in a terminal session. The other options are not relevant for this task, as they either push changes to the remote repository (Push), combine two branches (Merge), save changes to the local repository (Commit), or create a new local repository from a remote one (Clone). References:

Run Git operations on Databricks Repos

Git pull

Auto Loader is a tool that can incrementally and efficiently process new data files as they arrive in cloud storage without any additional setup. Auto Loader provides a Structured Streaming source called cloudFiles, which automatically detects and processes new files in a given input directory path on the cloud file storage. Auto Loader also tracks the ingestion progress and ensures exactly-once semantics when writing data into Delta Lake. Auto Loader can ingest various file formats, such as JSON, CSV, XML, PARQUET, AVRO, ORC, TEXT, and BINARYFILE. Auto Loader has support for both Python and SQL in Delta Live Tables, which are a declarative way to build production-quality data pipelines with Databricks. References: What is Auto Loader?, Get started with Databricks Auto Loader, Auto Loader in Delta Live Tables

Databricks Asset Bundles use YAML configuration, and Databricks documentation specifies that bundle configuration is expressed in a root databricks.yml file. The resources mapping is used to define supported objects such as jobs , and job definitions include keys like name and tasks. Tasks in a job are defined under the tasks key, and a notebook task is represented with notebook_task and notebook_path. Option A is the only answer written in the correct general YAML bundle structure and matches the documented style of defining bundle resources. Options B and C use JSON- and Python-style syntax rather than bundle YAML. Option D uses Terraform/HCL-style syntax, which is not the Databricks Asset Bundle format. While a full production bundle often includes more top-level settings such as bundle, targets, or workspace, the question asks which format is valid, and only option A matches the documented Databricks Asset Bundles configuration model.

=========

https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html

 The keyword that can be used to compact the small files associated with a Delta table is OPTIMIZE. The OPTIMIZE command performs file compaction on a Delta table by rewriting a set of small files into a set of larger files1. This can improve the performance of queries that scan the table by reducing the number of files that need to be read and the amount of metadata that needs to be processed1. The OPTIMIZE command can also optionally sort the data within each file by a given set of columns, which can further improve the query performance by enabling data skipping and predicate pushdown1. The OPTIMIZE command can be applied to the whole table or to a specific partition of the table1.

The other keywords are not suitable for compacting the small files associated with a Delta table. REDUCE is a keyword used in the SQL syntax for aggregating data using a user-defined function2. COMPACTION is not a valid keyword in SQL or Python. REPARTITION is a keyword used in the Python syntax for changing the number of partitions of a DataFrame or an RDD3. VACUUM is a keyword used to remove files that are no longer referenced by a Delta table and are older than a retention threshold4.

1: OPTIMIZE | Databricks on AWS

2: REDUCE | Databricks on AWS

3: repartition | Databricks on AWS

4: VACUUM | Databricks on AWS

To create a table in Databricks using data from an SQLite database, the correct syntax involves specifying the format of the data source. The format in the case of using JDBC (Java Database Connectivity) with SQLite is specified by the org.apache.spark.sql.jdbc format. This format allows Spark to interface with various relational databases through JDBC. Here is how the command should be structured:

CREATE TABLE jdbc_customer360

USING org.apache.spark.sql.jdbc

OPTIONS (

url " jdbc:sqlite:/customers.db " ,

dbtable " customer360 "

)

The USING org.apache.spark.sql.jdbc line specifies that the JDBC data source is being used, enabling Spark to interact with the SQLite database via JDBC.

According to the medallion lakehouse architecture, gold tables are the final layer of data that powers analytics, machine learning, and production applications. They are often highly refined and aggregated, containing data that has been transformed into knowledge, rather than just information. Silver tables, on the other hand, are the intermediate layer of data that represents a validated, enriched version of the raw data from the bronze layer. They provide an enterprise view of all its key business entities, concepts and transactions, but they may not have all the aggregations and calculations that are required for specific use cases. Therefore, gold tables are more likely to contain aggregations than silver tables. References:

What is the medallion lakehouse architecture?

What is a Medallion Architecture?

Option A uses the supported ALTER TABLE ... SET MANAGED conversion for a Unity Catalog external Delta table. Databricks documents that this conversion retains the table name, permissions, dependent objects, settings, and table history while minimizing downtime for readers and writers. After conversion, Databricks manages the table’s storage lifecycle, satisfying the requirement for platform-controlled cleanup when the managed table is dropped. Dropping the original table with PURGE destroys the object instead of migrating it. SET LOCATION changes the storage path referenced by an external table but does not convert the table into a managed table. Creating a replacement table with CREATE TABLE AS SELECT introduces a new object and requires manual migration of permissions, dependencies, and workloads. Therefore, ALTER TABLE catalog.schema.sales SET MANAGED is the correct action.

================

The query performing a streaming hop from raw data to a Bronze table is identified by using the Spark streaming read capability and then writing to a Bronze table. Let ' s analyze the options:

Option A: Utilizes .writeStream but performs a complete aggregation which is more characteristic of a roll-up into a summarized table rather than a hop into a Bronze table.

Option B: Also uses .writeStream but calculates an average, which again does not typically represent the raw to Bronze transformation, which usually involves minimal transformations.

Option C: This uses a basic .write with .mode( " append " ) which is not a streaming operation, and hence not suitable for real-time streaming data transformation to a Bronze table.

Option D: It employs spark.readStream.load() to ingest raw data as a stream and then writes it out with .writeStream, which is a typical pattern for streaming data into a Bronze table where raw data is captured in real-time and minimal transformation is applied. This approach aligns with the concept of a Bronze table in a modern data architecture, where raw data is ingested continuously and stored in a more accessible format.

The MERGE INTO command is used to perform upserts, which are a combination of insertions and updates, based on a source table into a target Delta table1. The MERGE INTO command can handle scenarios where the target table cannot contain duplicate records, such as when there is a primary key or a unique constraint on the target table. The MERGE INTO command can match the source and target rows based on a merge condition and perform different actions depending on whether the rows are matched or not. For example, the MERGE INTO command can update the existing target rows with the new source values, insert the new source rows that do not exist in the target table, or delete the target rows that do not exist in the source table1.

The INSERT INTO command is used to append new rows to an existing table or create a new table from a query result2. The INSERT INTO command does not perform any updates or deletions on the existing target table rows. The INSERT INTO command can handle scenarios where the location of the data needs to be changed, such as when the data needs to be moved from one table to another, or when the data needs to be partitioned by a certain column2. The INSERT INTO command can also handle scenarios where the target table is an external table, such as when the data is stored in an external storage system like Amazon S3 or Azure Blob Storage3. The INSERT INTO command can also handle scenarios where the source table can be deleted, such as when the source table is a temporary table or a view4. The INSERT INTO command can also handle scenarios where the source is not a Delta table, such as when the source is a Parquet, CSV, JSON, or Avro file5.

1: MERGE INTO | Databricks on AWS

2: [INSERT INTO | Databricks on AWS]

3: [External tables | Databricks on AWS]

4: [Temporary views | Databricks on AWS]

5: [Data sources | Databricks on AWS]

DESCRIBE HISTORY my_table returns the Delta table’s operation history. Its result includes information such as the table version, operation timestamp, user identity when available, operation name, parameters, notebook or job details, and associated metrics. The engineer can filter the returned timestamps to analyze operations performed during the last seven days. SHOW TBLPROPERTIES displays configured table properties rather than transactional history. DESCRIBE DETAIL returns the current table metadata and storage characteristics, not a chronological audit of changes. Directly querying the _delta_log path is unnecessary, depends on file-level implementation details, and the displayed path is not the supported SQL statement for this audit requirement. Therefore, option C provides the supported and structured record of past Delta table operations.

================

A webhook alert destination is a notification destination that allows Databricks to send HTTP POST requests to a third-party endpoint when an alert is triggered. This enables the data engineer to integrate Databricks alerts with their preferred messaging or collaboration platform, such as Slack, Microsoft Teams, or PagerDuty. To set up a webhook alert destination, the data engineer needs to create and configure a webhook connector in their messaging platform, and then add the webhook URL to the Databricks notification destination. After that, the data engineer can create an alert for their Databricks SQL query, and select the webhook alert destination as the notification destination. The alert can be configured with a custom condition, such as when the number of stores with $0 in sales is greater than zero, and a custom message template, such as “Alert: {number_of_stores} stores have $0 in sales”. The alert can also be configured with a recurrence interval, such as every hour, to check the query result periodically. When the alert condition is met, the data engineer and their team will receive a notification via the messaging webhook, with the custom message and a link to the Databricks SQL query. The other options are either not suitable for sending notifications via a messaging webhook (A, B, E), or not suitable for sending recurring notifications ©. References: Databricks Documentation - Manage notification destinations, Databricks Documentation - Create alerts for Databricks SQL queries, Databricks Documentation - Configure alert conditions and messages.

The small country lookup DataFrame should be broadcast to the executors. A broadcast join copies the small side of the join to each worker, allowing every executor to join its partition of the billion-row events DataFrame locally. This avoids shuffling the very large DataFrame across the network and is effective when one relation is small enough to fit comfortably in executor memory. Broadcasting events_df would be impractical because it contains billions of rows. Caching or persisting both DataFrames can help when data is reused, but neither action by itself changes the join strategy or removes the expensive shuffle. Databricks exposes the PySpark broadcast function specifically to mark a DataFrame as suitable for a broadcast join. Therefore, option D is the correct optimization.

================

One of the benefits of the Databricks Lakehouse Platform embracing open source technologies is that it avoids vendor lock-in. This means that customers can use the same open source tools and frameworks across different cloud providers, and migrate their data and workloads without being tied to a specific vendor. The Databricks Lakehouse Platform is built on open source projects such as Apache Spark™, Delta Lake, MLflow, and Redash, which are widely used and trusted by millions of developers. By supporting these open source technologies, the Databricks Lakehouse Platform enables customers to leverage the innovation and community of the open source ecosystem, and avoid the risk of being locked into proprietary or closed solutions. The other options are either not related to open source technologies (A, B, C, D), or not benefits of the Databricks Lakehouse Platform (A, B). References: Databricks Documentation - Built on open source, Databricks Documentation - What is the Lakehouse Platform?, Databricks Blog - Introducing the Databricks Lakehouse Platform.

Option D provides a practical balance between the 15-minute latency objective and compute cost. A scheduled Lakeflow Job running every 12 minutes leaves a small margin for startup and processing while avoiding an always-running stream. Trigger.AvailableNow processes all records available when the query starts in one or more incremental micro-batches and then terminates, allowing the associated compute to be released after the backlog is consumed. Running the job every minute would introduce unnecessary job-start and storage-check overhead. A continuous trigger would keep compute active even when little or no data is arriving. Triggering a separate run for every individual file can also create excessive runs and operational overhead. Therefore, the scheduled AvailableNow strategy in option D supports bounded latency while minimizing idle compute consumption.

================

The engineer should use Repair run . A repair reruns unsuccessful tasks as part of the original job run while preserving the successful status of tasks that do not need to be executed again. In this scenario, Tasks 1 and 2 remain successful, Task 3 is rerun after its problem is corrected, and Task 4 can execute after its dependency succeeds. Run now starts a separate job run and normally executes the workflow from its beginning. Cloning a task creates another task definition and does not resume the failed run. Reset job is not the appropriate Lakeflow Jobs recovery operation. Databricks also notes that repaired tasks should use idempotent write logic where possible because a failed task might have produced partial output before failing. Therefore, option C correctly resumes the workflow without unnecessarily repeating the successful upstream tasks.

================

COPY INTO is idempotent by default and tracks files that have already been loaded into the target table. A corrected file uploaded under the same name and path can therefore be treated as previously processed and skipped. The reliable pattern is to publish the correction under a new unique filename so that ingestion detects it as a new file. The pipeline can then use a business key with MERGE, UPDATE, or equivalent transformation logic to apply the corrected values to the target Delta table. Overwriting or recreating the complete table risks unnecessary data loss, increases processing cost, and disrupts downstream consumers. Reloading every historical source file is also inefficient and can introduce duplicates. Therefore, option B supports incremental ingestion while applying corrections safely and efficiently.

================

For small-scale, ad hoc Python workloads that run frequently and should start quickly and stop without leaving idle infrastructure running, serverless compute is the best fit. Databricks documentation explains that serverless compute allows users to run workloads without provisioning compute resources manually , and Databricks automatically allocates and manages the required resources. This improves startup and scaling speed and minimizes idle time, which directly matches the requirement that compute should “wind down quickly” after the work completes. Databricks also recommends serverless compute broadly when the workload is supported because it is the simplest and most reliable option. An all-purpose cluster is better suited to interactive collaborative development and can remain running longer than needed. A job cluster is ephemeral and often a good batch choice, but it still requires more explicit configuration than serverless and is not as optimized for minimal operational overhead. A SQL warehouse is intended for SQL workloads, not general ad hoc Python script execution. Therefore, serverless compute is the most appropriate choice.

A fan-in dependency pattern makes the dashboard task depend on all three parallel tasks. After the entry notebook completes, Databricks can execute the cleansing, feature-engineering, and model-update tasks concurrently because none depends on another. The dashboard refresh remains blocked until every configured upstream dependency has completed successfully. In Lakeflow Jobs, task dependencies form a directed acyclic graph, and Databricks runs as many eligible upstream tasks in parallel as possible. The default All succeeded Run if condition ensures that the downstream dashboard task runs only when all three dependencies succeed. Sequentially chaining the parallel tasks would unnecessarily increase the total execution time. Depending on only one task could refresh the dashboard before the other outputs are ready. Therefore, option B correctly implements the required fan-out and fan-in DAG.

================

Delta Lake is an open source project that provides ACID transactions, time travel, and other features on top of Apache Parquet, a columnar file format that is widely used for big data analytics. Delta Lake uses versioned Parquet files to store your data in your cloud storage, along with JSON files as transaction logs and checkpoint files to track the changes and ensure data integrity. Delta Lake is compatible with any Apache Hive compatible file format, such as CSV, JSON, or AVRO, but it primarily stores data as Parquet files for better performance and compression. References: How to Create Delta Lake tables, 5 reasons to choose Delta Lake format (on Databricks), Parquet vs Delta format in Azure Data Lake Gen 2 store, What is Delta Lake? - Azure Databricks, Lakehouse and Delta tables - Microsoft Fabric

 The VACUUM command is used to remove files that are no longer referenced by a Delta table and are older than the retention threshold1. The default retention period is 7 days2, but it can be changed by setting the delta.logRetentionDuration and delta.deletedFileRetentionDuration configurations3. If the VACUUM command was run on the table with a retention period shorter than 3 days, then the data files that were needed to restore the table to a 3-day-old version would have been deleted. The other commands do not delete data files from the table. The TIME TRAVEL command is used to query a historical version of the table4. The DELETE HISTORY command is not a valid command in Delta Lake. The OPTIMIZE command is used to improve the performance of the table by compacting small files into larger ones5. The HISTORY command is used to retrieve information about the operations performed on the table. References: 1: VACUUM | Databricks on AWS 2: Work with Delta Lake table history | Databricks on AWS 3: [Delta Lake configuration | Databricks on AWS]  4: Work with Delta Lake table history - Azure Databricks 5: [OPTIMIZE | Databricks on AWS] : [HISTORY | Databricks on AWS]

In the Medallion Architecture promoted by Databricks, the Bronze layer represents the raw ingestion layer where data is stored in its original form with minimal transformation. The primary goal is to ensure data fidelity and traceability , not to apply business logic or heavy transformations. Typical operations in the Bronze layer include schema inference (or enforcement), ingestion of raw data, and the addition of metadata columns such as ingestion timestamp, load date/time, and process identifiers. These metadata fields help track lineage, enable debugging, and support incremental processing downstream. Extensive data cleansing (A), aggregations (B), and business transformations (C) are intentionally deferred to the Silver and Gold layers , where data is refined and structured for analytics. By keeping Bronze data close to the source and append-only, organizations maintain a reliable audit trail and can reprocess data if needed. Therefore, adding operational metadata like load timestamps and process IDs is the correct characteristic of Bronze layer transformations.

=========

 Option A is the correct answer because it uses the FILTER higher-order function correctly to filter out employees with more than 5 years of experience from the array column “employees”. It applies a lambda function i - > i.years_exp > 5 that checks if the years of experience of each employee in the array is greater than 5. If this condition is met, the employee is included in the new array column “exp_employees”.

The use of higher-order functions like FILTER can be referenced from Databricks documentation on Higher-Order Functions.

The workspace administrator is the only individual who can transfer ownership of the Delta tables in Data Explorer, assuming the original data engineer no longer has access. The workspace administrator has the highest level of permissions in the workspace and can manage all resources, users, and groups. The other options are either not possible or not sufficient to perform the ownership transfer. The Databricks account representative is not involved in the workspace management. The transfer is possible and not dependent on the original data engineer. The new lead data engineer may not have the necessary permissions to access or modify the Delta tables, unless granted by the workspace administrator or the original data engineer before leaving. References: Workspace access control, Manage Unity Catalog object ownership.

The COPY INTO statement is an idempotent operation, which means that it will skip any files that have already been loaded into the target table1. This ensures that the data is not duplicated or corrupted by multiple attempts to load the same file. Therefore, if the data engineer runs the same command every day without specifying the names of the files to be copied with the FILES keyword or a glob pattern with the PATTERN keyword, the statement will only copy the first file that matches the source location and ignore the rest. To avoid this problem, the data engineer should either use the FILES or PATTERN keywords to filter the files to be copied based on the date or some other criteria, or delete the files from the source location after they are copied into the table2. References: 1: COPY INTO | Databricks on AWS 2: Get started using COPY INTO to load data | Databricks on AWS

The < = > operator performs null-safe equality in Databricks SQL. It returns TRUE when both operands contain the same non-NULL value and also when both operands are NULL. It returns FALSE when only one operand is NULL. By contrast, the standard equality operator returns UNKNOWN when either side is NULL, so it cannot match two NULL values in a WHERE condition. Option C implements inequality logic and does not provide null-safe equality. Option D can incorrectly treat a genuine empty string as equivalent to NULL, changing the meaning of the data. Therefore, option A is the only predicate that preserves SQL values while implementing the required NULL comparison behavior. This question was also checked against the existing Word question bank.

================

The command databricks bundle deploy deploys the bundle’s local files, artifacts, and resource definitions to the configured remote Databricks workspace. Databricks documents this command as the deployment operation for Declarative Automation Bundles, and a specific target can be selected with the -t or --target option. The validate command checks whether the bundle configuration is valid but does not create or update workspace resources. The run command executes a resource that has already been deployed; it is not the deployment step. The init command creates a new bundle project from a template, while bundle deployment is not a valid command combination. Because the question asks for the single command that sends local bundle assets to the target workspace, option C is correct.

================

Databricks documents that Delta Sharing does not require data replication , but cloud providers can still charge data egress fees when data is shared across clouds or across geographic regions . Databricks specifically states that sharing within the same region incurs no egress cost , while cross-cloud or cross-region transfers can create additional charges from the underlying cloud provider. That means option A is correct. Options B and C describe same-cloud or same-region use cases that do not trigger the cross-region/cross-cloud egress pattern Databricks calls out. Option D is unrelated to the main documented billing driver for Delta Sharing costs. The important principle is that Delta Sharing itself is designed to avoid replication overhead, but the physical movement of data between cloud boundaries or regions can still result in vendor networking charges. Therefore, when sharing data with external partners in other regions or clouds, engineers should plan for possible egress costs and monitor them accordingly.

=========

 To grant full permissions on a table to a user or a group, you can use the GRANT ALL PRIVILEGES ON TABLE statement. This statement will grant all the possible privileges on the table, such as SELECT, CREATE, MODIFY, DROP, ALTER, etc. Option A is the only code block that follows this syntax correctly. Option B is incorrect, as it does not grant all the possible privileges on the table, but only a subset of them. Option C is incorrect, as it only grants the SELECT privilege on the table, which is not enough to fully manage the project. Option D is incorrect, as it grants the USAGE privilege on the table, which is not a valid privilege for tables. Option E is incorrect, as it grants all the privileges on the table team to the user or group sales, which is the opposite of what the question asks. References: Grant privileges on a table using SQL | Databricks on AWS, Grant privileges on a table using SQL - Azure Databricks, SQL Privileges - Databricks

 Create a table using SQL | Databricks on AWS, Create a table using SQL - Azure Databricks, Delta Lake Quickstart - Azure Databricks

 Option C is the correct answer because Parquet files have a well-defined schema that is embedded within the data itself. This means that the data types and column names of the Parquet files are automatically detected and preserved when creating an external table from them. This also enables the use of SQL and other structured query languages to access and analyze the data. CSV files, on the other hand, do not have a schema embedded in them, and require specifying the schema explicitly or inferring it from the data when creating an external table from them. This can lead to errors or inconsistencies in the data types and column names, and also increase the processing time and complexity.

 CREATE TABLE AS SELECT, Parquet Files, CSV Files, Parquet vs. CSV

The OPTIMIZE command in Delta Lake is designed to compact small files into larger files to improve query performance and reduce file system overhead. Importantly, OPTIMIZE is considered an idempotent operation when run on unchanged data. This means that once files have been compacted optimally, running OPTIMIZE again does not produce additional changes unless new small files have been added to the table. Databricks ensures that file compaction is efficient and avoids unnecessary rewrites, which helps control compute costs and maintain stable performance. While features like ZORDER can be applied during OPTIMIZE to improve data skipping, re-running the same command without new data will not further reorganize files. Options A, B, and C incorrectly imply additional restructuring or changes on repeated execution. According to Delta Lake best practices, OPTIMIZE should be run periodically after data ingestion events rather than repeatedly on unchanged datasets. This behavior reinforces predictable performance and efficient resource utilization in Databricks environments.

=========

An If/else condition task is specifically designed to branch a Lakeflow Job based on a Boolean expression. The condition can reference the upstream value as {{tasks.check_volume.values.record_count}} and compare it with 1000000. The true branch can invoke high_volume_pipeline, while the false branch invokes normal_pipeline. Run if conditions evaluate task outcomes such as success or failure; they do not compare a calculated record count. A For each task iterates over an input array and is not appropriate for selecting one of two execution paths. Retries are intended to recover from unsuccessful task attempts, not to wait for a business metric to cross a threshold. Therefore, option A implements the required value-based branching. This question was retained after comparison with the existing Word question bank.

================