Confluent CCDAK Confluent Certified Developer for Apache Kafka Certification Examination Exam Practice Test

According to the official Apache Kafka consumer semantics documentation, at-most-once delivery means that records may be lost but must never be processed more than once. To achieve this guarantee, a consumer must commit offsets before processing records. If the application crashes after committing but before processing, messages are skipped rather than reprocessed, which satisfies at-most-once semantics.

Option B correctly implements this pattern. It disables automatic offset commits (enable.auto.commit=false) and explicitly calls commitAsync() immediately after polling and before processing records. This ensures that offsets are advanced regardless of whether processing succeeds, eliminating the possibility of duplicate processing.

Option C represents at-least-once semantics, because offsets are committed after processing, which can lead to duplicates on failure. Options A and D rely on automatic commits or incorrect ordering and do not provide strict at-most-once guarantees.

The Kafka documentation explicitly states that manual offset management with commits before processing is required for at-most-once delivery.

Therefore, Option B is the best and officially documented configuration and code skeleton to avoid duplicate message processing.

The Apache Kafka consumer configuration max.poll.records explicitly controls the maximum number of records returned in a single call to Consumer.poll(). This is clearly documented in the official Kafka consumer configuration reference. It allows developers to limit the size of the batch returned to the application, helping to control processing latency, memory usage, and CPU load.

This setting does not affect how much data the broker sends to the consumer; instead, it limits how many records the consumer returns to the application layer from the fetched data. This makes it especially useful for applications where processing each record is expensive and needs predictable batch sizes.

Option B (max.records.consumer) does not exist in Kafka. Option C (fetch.max.records) is also invalid; Kafka uses fetch.max.bytes and related byte-based configurations, not record-count limits, at the fetch level. Option D (max.poll.records.interval) is not a valid Kafka configuration; the correct related setting is max.poll.interval.ms, which controls liveness, not batch size.

Thus, max.poll.records is the correct and officially documented configuration.

According to the official Apache Kafka Connect REST API documentation, restarting a connector can be done at different granularities: connector-level, task-level, or both. By default, calling POST /connectors/{name}/restart only restarts the connector instance, not its tasks.

To restart the connector and all of its tasks, the request must explicitly include the query parameter includeTasks=true. This instructs Kafka Connect to restart the connector and all associated tasks, regardless of whether they are currently running or failed.

Option B (onlyFailed=true) limits the restart to only failed tasks, which does not meet the requirement to restart all tasks. Option C restarts only the connector instance, leaving tasks untouched. Option D restarts a single specific task and does not restart the connector itself.

Therefore, the correct and officially documented REST request to restart the connector instance and all tasks is:

POST /connectors/s3-connector/restart?includeTasks=true

This ensures a full restart of the connector and its execution context across the Kafka Connect cluster.

For real-time applications, the key metric to monitor is consumer lag — the difference between the latest offset on the broker and the last committed offset by the consumer.

From Kafka Monitoring Guide:

“Consumer lag is the most important metric for real-time applications. It tells you how far behind the consumer is from the latest data in Kafka.”

B relates to replication, not client responsiveness.

C tracks group membership stability, not data delay.

D tracks input throughput, not processing latency.

Table → Product, Customers

Stream → Orders_Placed, Shipment_Of_Orders

Tables represent slowly changing, reference data — e.g., customer profiles, product info.

Streams represent real-time event data — e.g., orders placed, shipments processed.

From Kafka Streams Documentation:

“Use KTable for reference datasets, and KStream for event-based processing such as orders or logs.”

In the Kafka Streams API, operations are classified as stateless or stateful based on whether they require maintaining local state stores. According to the official Kafka Streams documentation, stateless operations process each record independently, without storing or accessing prior records.

Filter is a stateless operation because it evaluates each record individually and decides whether to pass it downstream. It does not require any state or historical context.

GroupBy is also considered stateless because it merely repartitions the stream by assigning a new key and forwarding records to downstream processors. While it triggers the creation of an internal repartition topic, the GroupBy operation itself does not maintain a state store.

In contrast, Reduce is a stateful operation because it aggregates records over time and requires maintaining intermediate results in a state store. Similarly, Join operations are stateful because Kafka Streams must buffer and store records from one or both input streams or tables to perform the join within a defined time window.

Thus, the correct stateless operations are Filter and GroupBy, as documented in the Kafka Streams developer guide.

A 409 Conflict response in Kafka Connect REST API usually means:

The connector name provided already exists (i.e., a connector with that name is already deployed).

The Kafka Connect cluster is temporarily unavailable to accept new tasks due to a rebalance in progress.

From the Confluent Developer documentation and Kafka Connect REST API spec:

“409 – Conflict. The connector name is already in use or the worker group is in the process of rebalancing and cannot complete the request at this time.”

Option A is incorrect because invalid configurations return 400 Bad Request, not 409.

Option B is invalid because Kafka Connect does not have a strict concept of "capacity" that prevents connector creation with a 409.

unclean.leader.election.enable=false → Prevents data loss by only considering in-sync replicas when rebalancing.

replication.factor → Specifies how many redundant copies of partitions are distributed across brokers.

min.insync.replicas=2 → Sets the standard for the number of partition instances that must keep up with the latest committed message.

unclean.leader.election.enable=false ensures that only in-sync replicas can be elected as leaders. If disabled, an out-of-sync replica may become leader, potentially leading to data loss.

replication.factor defines how many brokers will maintain copies of each partition, directly impacting durability and availability.

min.insync.replicas determines how many replicas must acknowledge a write when acks=all is used, enforcing write durability.

The Apache Kafka Streams documentation defines a co-partitioning requirement for KStream–KTable and KStream–KStream joins. Both input topics must have the same number of partitions and the same key partitioning strategy.

One valid solution is to use a GlobalKTable (Option A). A GlobalKTable is fully replicated to every Kafka Streams instance, removing the co-partitioning requirement. This approach is recommended when the reference data is relatively small and changes infrequently.

Another valid solution is to repartition one topic so that both topics have the same number of partitions (Option B). Kafka Streams provides repartition topics specifically for this purpose, allowing proper KStream–KTable joins.

Option C does not resolve the partition mismatch, as increasing instances does not change partitioning. Option D is incorrect because Kafka Streams does not automatically repartition both topics for joins; repartitioning must be explicitly configured.

Therefore, the correct and officially supported solutions are using a GlobalKTable and explicitly repartitioning one topic.

=========================

The onCompletion() method of Kafka’s Callback interface is executed once the broker acknowledges the message. This includes success or failure, and it is invoked asynchronously by the producer.

From Kafka Java Client API Documentation:

“The onCompletion method will be called when the record sent to the server has been acknowledged, or when an error occurs.”

This ensures that the record was sent and acknowledged, not just added to a batch or local buffer.

Kafka message headers are the right place to include metadata such as:

Payload format (e.g., schema version)

Timestamps (custom)

Trace or correlation IDs

From Kafka Producer API Docs:

“Headers are a map of key-value pairs that can be used to include metadata alongside the message, without modifying the message body.”

Keys determine partitioning and ordering

Values are the message payload

Brokers do not store arbitrary metadata fields

These three metrics are critical for cluster health:

OfflinePartitionsCount: Indicates partitions without active leaders — a sign of broker failure.

ActiveControllerCount: There should be exactly one active controller. A count ≠ 1 signals controller failure.

UncleanLeaderElectionsPerSec: Tracks leader elections where out-of-sync replicas were selected — risky for data loss.

From Kafka Monitoring Documentation:

“Offline partitions and unclean leader elections should trigger alerts. Also, ensure a single active controller is running.”

A is about topics pending deletion — not critical.

E is a per-topic record metric, not broker-level.

The official CLI utility for consuming messages from Kafka topics is kafka-console-consumer.sh. It connects to the broker, consumes messages, and prints them to standard output.

From Kafka CLI Tools Documentation:

"kafka-console-consumer.sh is used to read data from a Kafka topic and write it to standard output."

The other options are not valid Kafka CLI tools.

This scenario describes vertical scaling, as defined in the Apache Kafka Streams documentation and general distributed systems terminology. Vertical scaling refers to adding more resources (CPU, memory, or disk) to a single machine to increase processing capacity.

In this example, the Kafka Streams application is moved from a server with four CPU threads to one with ten CPU threads, increasing the available compute resources without increasing the number of application instances. Kafka Streams can take advantage of additional CPU threads through its num.stream.threads configuration, allowing more partitions to be processed in parallel within the same application instance.

Horizontal scaling (also known as scaling out) would involve adding more application instances across multiple servers, not upgrading a single server. Option D is therefore incorrect, and Option C is not a recognized scaling term.

Thus, this example clearly illustrates vertical scaling, where performance is improved by increasing the capacity of a single node.

To change the replication factor of an existing topic, you must perform a partition reassignment, because the replication factor is not directly modifiable via kafka-topics.sh. The kafka-reassign-partitions.sh tool is the official command-line utility to handle partition reassignments, including changing the replication factor.

This tool allows you to generate a reassignment JSON specifying new replica assignments per partition and then apply it.

From official documentation:

“To change the replication factor, use the kafka-reassign-partitions tool to assign the partitions to more or fewer brokers.”

Neither kafka-topics.sh nor kafka-reassign-topics.sh exists for this purpose, and kafka-recreate-topic.sh is not an official Kafka tool.

To increase consumer throughput, one effective strategy is to increase the amount of data fetched in each poll by raising fetch.max.bytes or max.partition.fetch.bytes. This allows each poll to retrieve more records per request, improving processing efficiency.

From Kafka Consumer Config Docs:

“Increasing fetch size allows consumers to retrieve larger batches of messages, improving throughput and reducing request overhead.”

Removing consumers (A) may reduce parallelism.

Manual commit (C) adds complexity, not throughput.

Decreasing session timeout (D) risks unnecessary rebalances.

According to the official Apache Kafka Connect documentation, RegexRouter is the SMT specifically designed to dynamically change the destination topic name of records produced by a connector. It works by applying a regular expression to the original topic name and rewriting it to a new topic name.

This makes RegexRouter the correct choice when some records must be routed to a different topic, typically in combination with connector-level logic, predicates, or multiple connectors. It is commonly used for topic renaming, topic versioning, or routing records to alternate topics.

InsertField (Option B) only adds metadata fields (such as topic, partition, or timestamp) to the record payload and does not affect routing. TombstoneHandler (Option C) is used to manage null-value records, especially with compacted topics. HeaderFrom (Option D) copies fields into headers but does not change the target topic.

Therefore, RegexRouter is the only SMT that directly supports changing the output topic, as documented in the Kafka Connect SMT reference.

=========================

According to the official Apache Kafka documentation, log compaction is the mechanism used to retain only the latest record for each key in a topic. By setting cleanup.policy=compact, Kafka ensures that older records with the same key are eventually removed, leaving only the most recent value for each key.

This behavior is exactly suited for use cases such as tracking the current state of an entity, including GPS tracker locations, user profiles, or configuration data. In this scenario, the key (trackerId) uniquely identifies a tracker, and the value represents its most recent latitude and longitude.

Option B (retention.ms=infinite) retains all historical data, which contradicts the requirement. Option C (min.cleanable.dirty.ratio) controls when compaction runs, not what data is retained. Option D (retention.ms=0) would immediately delete all data and is unsafe.

Therefore, setting cleanup.policy=compact is the correct and officially documented solution to always retain the latest location per tracker.

=========================

The offset in the RecordMetadata object returned by the producer represents the position of the record in the partition — i.e., the sequential ID assigned by Kafka once the message is committed.

From Kafka Producer API Documentation:

“The offset is the position of the record in the partition. This is a unique, sequential number assigned by the broker.”

D refers to metadata.partition(), not offset().

B and C are unrelated to how Kafka handles committed offsets.

This syntax is a clear match to Protocol Buffers (Protobuf). It defines a schema with fields, types, and tags, which is a format supported by Kafka when using Protobuf-based schema registry serialization.

From Confluent Schema Registry Docs:

“Kafka supports Protobuf serialization, where schemas are written in .proto files and include fields with tags.”

int32, double, and string are standard Protobuf types.

Avro uses JSON-style schema.

JSON Schema uses JSON object structure, not .proto.

YAML is unrelated.

According to the official Apache Kafka consumer documentation, the auto.offset.reset configuration is only applied when a consumer has no valid committed offset for a partition. In this scenario, the consumer last committed offset 217, but the topic’s earliest available offset is now 318. This means the previously committed offset is out of range, most likely due to topic retention cleanup.

When a committed offset is out of range, Kafka treats it as invalid and applies the auto.offset.reset policy. Since the consumer is configured with auto.offset.reset=latest, it will start reading from the end of the log, not from the earliest available offset.

Kafka defines the “latest” offset as the log end offset, which is the offset of the next message to be written. Given that the topic’s offsets end at 588, the next offset is 589.

Therefore, the consumer will start reading from offset 589, skipping all existing records in the topic, which aligns with the documented Kafka consumer behavior.

=========================

Log compaction ensures that Kafka retains at least the latest value per key in a topic. Compaction happens in the background and removes older records with the same key in inactive log segments, not immediately.

From the Kafka Documentation > Log Compaction:

“Kafka guarantees that the last message for each key will be retained in the log after compaction, even if earlier messages with the same key are deleted.”

So, D is correct. A is incorrect because compaction does not overwrite; it's a background process. B is incorrect—deletion is not immediate. C incorrectly suggests client-side hashmap behavior.

The official Apache Kafka Streams developer documentation clearly states that TopologyTestDriver is the recommended and supported tool for unit testing Kafka Streams applications without requiring a running Kafka cluster. It is specifically designed to test complex topologies consisting of multiple sources, processors, state stores, sinks, and even sub-topologies.

TopologyTestDriver allows developers to instantiate the Kafka Streams topology directly in memory and feed test records into input topics while reading results from output topics. This makes it ideal for fast, deterministic, and isolated unit tests. As documented, it supports testing stateful and stateless operations, windowed aggregations, joins, and exactly-once behavior at the topology level.

Option A is incorrect because TestProducer and TestConsumer are not official Kafka Streams testing utilities. Option B is invalid because KafkaUnitTestDriver does not exist in Apache Kafka. Option D (MockProducer and MockConsumer) belongs to the Kafka Clients API and is suitable for producer/consumer testing, but it does not support Kafka Streams topologies or state stores.

Therefore, TopologyTestDriver is the only fully supported and officially documented solution for unit testing Kafka Streams applications without a real Kafka cluster.

If the consumer fails to send heartbeats in time (due to being slow under load), it is considered dead, causing a rebalance. Increasing session.timeout.ms gives the consumer more time to process messages and send heartbeats, preventing premature removal.

From Kafka Docs > Heartbeats and Failures:

“If the consumer does not send a heartbeat in session.timeout.ms, it is considered dead and the group coordinator will trigger a rebalance.”

Reducing max.poll.records might slow consumption further. More consumers won’t help since each partition already has a dedicated consumer.

Using assign() gives manual control over partition assignment. In this mode:

B is true: The application chooses partitions directly without involving the group coordinator.

C is true: Since no group coordination occurs, no rebalancing happens if a consumer dies.

From Kafka Consumer API Docs:

“assign() manually assigns a list of partitions to this consumer. No coordination takes place and no rebalancing is triggered.”

A is false: subscribe() and assign() are mutually exclusive.

D is incorrect: Partitions don't need to be equal across topics.

The official Apache Kafka producer documentation states that setting enable.idempotence=true guarantees that messages are written to a partition exactly once, even in the presence of retries caused by network failures or broker timeouts. This feature prevents duplicate records by assigning producer sequence numbers and validating them on the broker side.

For idempotent producers to work correctly, retries must be enabled, which is why Kafka recommends a very large value such as retries=2147483647. Additionally, limiting max.in.flight.requests.per.connection to 1 ensures strict ordering during retries, preventing message reordering in older Kafka versions and providing the safest configuration.

Option A is irrelevant to producers. Option B explicitly disables idempotence, which causes duplicates. Option D disables retries, which increases the risk of message loss.

Therefore, the correct and fully documented solution to eliminate duplicate messages is enabling idempotence with retries and a safe in-flight request limit, as shown in Option C.

If the application is CPU-bound and lagging, adding more consumers to the group will allow better parallel processing, especially since the topic has 4 partitions, allowing up to 4 active consumers.

From Kafka Consumer Group Docs:

“Kafka achieves parallelism by distributing partitions across consumers in a group. Adding consumers helps reduce lag if partitions are underutilized.”

B may help but requires repartitioning and coordination.

C or D affects how much data is polled, not how fast it’s processed.