ISTQB CT-AI ISTQB Certified Tester AI Testing Exam Exam Practice Test

The syllabus describes automation bias as:

"A type of bias caused by a person favoring the recommendations of an automated decision-making system over other sources."

This is also known as complacency bias, where testers accept automated system outputs without questioning them.

(Reference: ISTQB CT-AI Syllabus v1.0, Glossary, Page 89 of 99)

Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.

Testing the distribution shift in the training data for inappropriate bias (A): This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.

Test the model during model evaluation for data bias (B): This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.

Testing the data pipeline for any sources for algorithmic bias (C): This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.

Check the input test data for potential sample bias (D): While this is an important step, it focuses more on the input data and less on the overall data pipeline.

Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline isB. Test the model during model evaluation for data bias.

The ISTQB CT-AI syllabus (Section5.3 – AI Support for Defect Analysis) explains that AI can categorize defect reports using natural language processing or classification models. Categorization helps route defects efficiently and determine which areas of the system are affected. Thus, OptionCis correct: AI canidentify defect categories, supporting assignment and triage.

Option A is incorrect because ML cannot infer severity or consequence without domain-specific risk modeling. Option B inaccurately ties duplicate detection to criticality; the syllabus separatesduplicate detectionfromseverity analysis. Option D is incorrect because AI may assist in routing defects using categories, but the syllabus does not claim AI identifies specific developers.

Thus,Option Cis the syllabus-accurate statement.

Decision trees are a foundational algorithm used in supervised machine learning. The syllabus describes:

"A decision tree is a tree-like ML model whose nodes represent decisions and whose branches represent possible outcomes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.4)

The syllabus highlights that on-device training and processing require considerable computational power, which may exceed the capabilities of some mobile devices:

"Self-learning and continuous learning systems require large amounts of computational power, which can impact system performance and stability if the hardware is not powerful enough."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.3, page 22 of 99)

The syllabus explains that EDA can be used to analyze data to identify outliers and unusual patterns, which can indicate adversarial attacks like data poisoning:

"Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.1.2, page 67 of 99)

The syllabus states:

"Concept drift occurs when the operational environment changes without the trained model changing correspondingly. The outputs of the model become less accurate and less useful. Therefore, the operational model should be regularly evaluated against its acceptance criteria."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 7.6, Page 54 of 99)

The ISTQB CT-AI syllabus indicates inSection 5.2 – AI for Testingthat AI-generated test cases may not come withpredefined expected results. This is because test-case generation methods—such as evolutionary algorithms, reinforcement learning, or clustering-based sampling—produceinputs, but the tester must still determine the correct outputs. Therefore, OptionCis correct: expected results may not be available, especially when AI produces novel or previously unseen input combinations.

Option A is incorrect: AI-based test generation can improve coverage by exploring large input spaces. Option B is incorrect because the need for oracles does not depend on whether the test case is AI-generated. Option D is incorrect because the syllabus allows using a model as an oracle inback-to-back testing, provided limitations are understood.

Therefore,Option Cis the correct syllabus-aligned statement.

 Testing Autonomy:Testing for human handover when it should not be relinquishing control is the least appropriate because it contradicts the very definition of autonomous systems. The other tests are relevant to ensuring smooth operation and transitions between human and AI control.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Testing Autonomous AI-Based Systems and Testing for Human-AI Interaction​​​​. ​

The syllabus clearly explains the structure of a deep neural network (DNN):

"A deep neural network comprises three types of layers. The input layer receives inputs... Between the input and output layers are hidden layers made up of artificial neurons, which are also known as nodes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 6.1, page 45 of 99)

The syllabus explains that input changes that arein the same domainas what was used for training are expected to be handled with adaptability:

"Adaptability refers to the ability of a system to adjust its behavior in response to changes in its environment or inputs. This includes changes to the inputs which are still within the expected operational range of the system, such as resolution changes in images or sensor data."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 7.6 and 8.2)

The syllabus explains that supervised learning requires correctly labeled data so the algorithm can learn the relationship between input features and output labels:

"In supervised learning, the algorithm creates the ML model from labeled data during the training phase. The labeled data is used to infer the relationship between the input data and output labels."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.1)

Flexibility in AI systems is crucial for various reasons, particularly because it allows for easier modification and adaptation of the system as a whole.

AI systems are inherently flexible (A): This statement is not correct. While some AI systems may be designed to be flexible, they are not inherently flexible by nature. Flexibility depends on the system's design and implementation.

AI systems require changing operational environments; therefore, flexibility is required (B): While it's true that AI systems may need to operate in changing environments, this statement does not directly address the importance of flexibility for the modification of the system.

Flexible AI systems allow for easier modification of the system as a whole (C): This statement correctly describes the importance of flexibility. Being able to modify AI systems easily is critical for their maintenance, adaptation to new requirements, and improvement.

Self-learning systems are expected to deal with new situations without explicitly having to program for it (D): This statement relates to the adaptability of self-learning systems rather than their overall flexibility for modification.

Hence, the correct answer isC. Flexible AI systems allow for easier modification of the system as a whole.

A. Regression

Regression is used to predict a continuous value and is not suitable for grouping people based on similar tastes.

B. Association

Association is used to find relationships between variables in large datasets, often in the form of rules (e.g., market basket analysis). It does not directly group individuals but identifies patterns of co-occurrence.

C. Clustering

Clustering is an unsupervised learning method used to group similar data points based on their features. It is ideal for identifying groups of people with similar tastes without prior knowledge of the group labels. This technique will help the company segment its customer base effectively.

D. Classification

Classification is a supervised learning method used to categorize data points into predefined classes. It requires labeled data for training, which is not the case here as we want to identify groups without predefined labels.

Therefore, the correct answer isCbecause clustering is the most suitable method for grouping people with similar tastes for targeted product marketing​​.

The ISTQB CT-AI syllabus describes theML data preparation workflowin Section2.2 – Data Preparation. Data preparation consists ofdata gathering,cleaning,transformation, andsampling. The syllabus emphasizes that one significant challenge duringdata gatheringis combining data frommultiple heterogeneous sources, which often differ in structure, quality, and format. Ensuring the resulting dataset is accurate, complete, and representative can be complex, making this a critical challenge in the ML workflow. This aligns directly with OptionC.

Option A is incorrect because erroneous data correction is part ofcleaning, not transformation. Option B contradicts the syllabus: while automation can help,not all steps should be automateddue to the need for expert oversight, especially in detecting subtle data quality issues. Option D is incorrect because sampling continues to involve risk—particularly around representativeness—and the syllabus emphasizes caution, not complacency.

Thus, OptionCis the only statement that accurately reflects the syllabus.

The question asks which combination of tests would be most appropriate to include in the strategy for optimal detection in a workflow system using multiple ML models.

Pairwise testing of combinations (I): This method is useful for testing interactions between different components in the workflow to ensure they work well together, identifying potential issues in the integration.

Testing each individual model for accuracy (II): Ensuring that each model in the workflow performs accurately on its own is crucial before integrating them into a combined workflow.

A/B testing of different sequences of models (III): This involves comparing different sequences to determine which configuration yields the best results. While useful, it might not be as fundamental as pairwise and individual accuracy testing in the initial stages.

The question refers to a problem where data used for an object detection ML system was labelled incorrectly. This issue is most closely related to "accuracy issues." Here's a detailed explanation:

Accuracy Issues: The primary goal of labeling data in machine learning is to ensure that the model can accurately learn and make predictions based on the given labels. Incorrectly labeled data directly impacts the model's accuracy, leading to poor performance because the model learns incorrect patterns.

Why Not Other Options:

Security Issues: This pertains to data breaches or unauthorized access, which is not relevant to the problem of incorrect data labeling.

Privacy Issues: This concerns the protection of personal data and is not related to the accuracy of data labeling.

Bias Issues: While bias in data can affect model performance, it specifically refers to systematic errors or prejudices in the data rather than outright incorrect labeling.

According to Section4.6 – AI Behaviour Testing (AIB Testing)of the ISTQB CT-AI syllabus, AIB testing is used to evaluate changes in the functional behavior of self-learning systems. The core principle iscomparing pre-change and post-change model behavior using the same test inputs, so that any difference in outputs can be attributed to the model’s learning and not to differences in input data. This directly corresponds to OptionC.

Option A is incorrect because the absence of a test oracle does not justify generating new test cases; AIB relies onreusing identical inputsto detect behavioral drift. Option B is invalid because using different inputs prevents meaningful comparison. Option D is incorrect because comparing with an unrelated non-self-learning system does not allow evaluation of the same model’s behavioral evolution.

Thus, OptionCaccurately represents the correct application of AIB testing: assessing model behavior changes by running identical test inputs before and after learning updates.

 Technology Most Typically Used to Implement AI:Genetic algorithms are a well-known technique used in AI. They are inspired by the process of natural selection and are used to find approximate solutions to optimization and search problems. Unlike search engines, procedural programming, or case control structures, genetic algorithms are specifically designed for evolving solutions and are commonly employed in AI implementations.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 1.4 AI Technologies, which identifies different technologies used to implement AI​​.

The syllabus defines reward hacking as:

"Reward hacking can result from an AI-based system achieving a specified goal by using a 'clever' or 'easy' solution that perverts the spirit of the designer’s intent."

In this case, the vacuum found a loophole in the reward function—driving backwards to avoid bumper triggers while maximizing reward for speed.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.6, page 24 of 99)

The syllabus defines clustering as:

“Clustering: This is when the problem requires the identification of similarities in input data points that allows them to be grouped based on common characteristics or attributes. For example, clustering is used to categorize different types of customers for the purpose of marketing.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.2, page 26 of 99)

The ISTQB CT-AI syllabus explains in Section1.6 – Machine Learning Approachesthatsupervised learningis appropriate when labeled data exists and the goal is to predict an output based on known historical examples. Predicting a customer’spurchase probabilityis aclassificationtask when the output corresponds to discrete categories such as“likely to purchase”vs.“not likely to purchase.”The syllabus gives similar examples in describing classification as the process of assigning instances to predefined classes based on learned patterns in labeled data. Because the retail company wants to determine whether a customer will make a purchase based on marketing actions, classification is the most appropriate choice .

Option A (regression) predicts continuous numeric values and is less suitable because the task centers oncategorical likelihood, not estimating exact monetary values. Option C (unsupervised learning) is used when labels are not available—here, the company has labeled purchase histories. Option D (reinforcement learning) requires an interactive environment with reward-driven behavior, which is not applicable to this scenario.

Thus,supervised learning (classification)is the most suitable approach according to the syllabus.

The ISTQB CT-AI syllabus emphasizes that testing AI-based systems requirescross-functional collaborationandexperience-based testingwhen parts of the team lack domain knowledge. In this scenario, the ML expert understands ML and dataset preparation but lacks knowledge ofcamera system behavior, the device’s operational data pipeline, and end-user workflows. The remainder of the team understands the domain and system testing but not ML. Section4.4 – Human Factors and AI Testingand4.3 – System Testing of AI Componentshighlight that when domain understanding is unevenly distributed,experience-based testing conducted by the full team(testers, developers, domain experts) is the most effective approach. This ensures that AI outputs align with actual user expectations and system behavior. OptionCaligns exactly with this principle.

Option A is too limited and does not address the need to validate ML integration. Option B is incorrect because reusing old test cases overlooks AI-specific risks in the operating data pipeline. Option D is useful but focuses only on data representativeness, not system-level user validation. Therefore,Option Cis the best, syllabus-aligned test approach.

TheISTQB Certified Tester AI Testing Syllabus v1.0explicitly categorizes different AI implementation technologies in Section1.4 – AI Technologies. Within this section, AI methods are grouped into categories, one of which is“Reasoning techniques.”These reasoning techniques includerule engines, deductive classifiers, case-based reasoning, and procedural reasoning. Because deductive classifiers are directly listed under this set of reasoning approaches, they are recognized as a reasoning-based AI technology.

Reasoning techniques differ from machine learning approaches because they rely onstructured, predefined rules or logicto reach conclusions. Deductive classifiers use logical inference and symbolic reasoning to classify inputs by applying encoded knowledge. This makes them fundamentally different from statistical or data-driven ML algorithms.

The other options—Linear regression,Random Forest, andGenetic algorithms—are listed by the syllabus asmachine learning techniques, not reasoning methods. Linear regression performs numerical prediction, Random Forest is an ensemble decision-tree ML model, and genetic algorithms are optimization-based ML approaches inspired by evolutionary processes. None of these involve symbolic logical deduction.

Thus, based on the authoritative definitions in the syllabus,Deductive classifiers (Option A)is the only technology classified as a reasoning technique.

Section2.4 – Data Labeling Approachesof the ISTQB CT-AI syllabus explicitly definesAI-assisted data labelingas a hybrid process in which an automated tool performs the initial labeling and human annotators subsequently verify, correct, or complete the labels. This two-step process improves efficiency while retaining human oversight to ensure data quality. The syllabus describes this method as an effective compromise when manual labeling alone would be too slow or costly, and when initial automation can identify obvious patterns before a human provides the final authoritative labels .

Option A (internal labeling) refers to labeling conducted by the organization’s own staff but does not imply automation. Option B (crowdsourced labeling) leverages a distributed workforce, typically without automation. Option C (outsourced labeling) transfers labeling tasks to external vendors but similarly does not involve an AI-first step. Only OptionDreflects the two-stage automated-then-human workflow described in the syllabus.

Therefore,AI-assisted data labeling(Option D) is unequivocally correct.

The syllabus defines overfitting as:

“Overfitting is when the ML model learns the training data so well that it is unable to generalize to accommodate new data.”

This occurs when the model memorizes the training data, including noise, instead of learning the general patterns.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.5.1, page 31 of 99)

 Factors Affecting ML Functional Performance:The data pipeline, quality of the labeling, and biased data are all factors that significantly affect the performance of machine learning models. The number of classes, while relevant for the model structure, is not a direct factor affecting the performance metrics such as accuracy or bias.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Quality and its Effect on the ML Model and ML Functional Performance Metrics​​​​.

Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options isleast likelyto be a reason for the explosion in the number of parameters.

Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters.

Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered.

ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle.

Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters.

Hence, theleast likelyreason for the incredible growth in the number of parameters isC. ML model metrics to evaluate the functional performance.

The syllabus mentions that AI can help optimize regression test suites:

"An AI-based tool can perform optimization of the regression test suite by analyzing… the information from previous test results, associated defects, and the latest changes that have been made, such as features which are broken more frequently and which tests exercise code impacted by recent changes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.4, page 79 of 99)

The ISTQB CT-AI syllabus defines these two concepts clearly inSection 2.1 – Flexibility and Adaptability. Flexibility is described as the ability of a system to operate in situationsnot explicitly covered in its original requirements, while adaptability refers to how easily the system can bemodifiedto handle new environments or conditions. The syllabus stresses that both flexibility and adaptability are crucial, particularly inself-learning AI systemsthat may need to respond to changes in their environment and adjust their behavior accordingly. It states that systems must be capable of determiningwhenandhowto adjust behavior in evolving situations, especially when the operational environment is not fully known at deployment time . This directly aligns with OptionA.

Option B reverses definitions—the syllabus states flexibility (not adaptability) relates to unspecified situations. Option C is incorrect: self-learning systems requirebothflexibility and adaptability; they are not categorized as one or the other. Option D incorrectly defines flexibility; the syllabus defines adaptability—not flexibility—as ease of modification.

Thus,Option Acorrectly reflects the syllabus.

The ISTQB CT-AI syllabus explains ML performance metrics in Section3.2 – Evaluating ML Models. Forclustering, which is an unsupervised learning method, the syllabus lists metrics such asintra-cluster distance,inter-cluster distance, and coherence measures. Intra-cluster metrics evaluate how close data points are within a cluster, which directly corresponds to Option A.

Option B is incorrect becauseR-squaredis a regression metric measuring goodness-of-fit, not classification performance, and has no connection to ROC curves. Option C is wrong because thesilhouette coefficientis also a clustering metric, measuring cohesion vs. separation—not regression accuracy. Option D is incorrect because ROC curves evaluatebinary or multiclass classification, not clustering.

Thus, OptionAis the only accurate statement based on the syllabus.

A. 64-bit CPUs.

While 64-bit CPUs are essential for handling large amounts of memory and performing complex computations, they are not specifically optimized for the types of operations commonly used in machine learning.

B. Hardware supporting fast matrix multiplication.

Matrix multiplication is a fundamental operation in many machine learning algorithms, especially in neural networks and deep learning. Hardware optimized for fast matrix multiplication, such as GPUs (Graphics Processing Units), is most suitable for implementing AI and ML because it can handle the parallel processing required for these operations efficiently.

C. High powered CPUs.

High powered CPUs are beneficial for general-purpose computing tasks and some aspects of ML, but they are not as efficient as specialized hardware like GPUs for matrix multiplication and other ML-specific tasks.

D. Hardware supporting high precision floating point operations.

High precision floating point operations are important for scientific computing and some specific AI tasks, but for many ML applications, fast matrix multiplication is more critical than high precision alone.

Therefore, the correct answer isBbecause hardware supporting fast matrix multiplication, such as GPUs, is most suitable for the parallel processing requirements of machine learning​​.

The syllabus explains:

"Where the internal structure of an AI-based system is too complex for humans to understand, the system can only be tested as a black box. Even when the internal structure is visible, this provides no additional useful information to help with testing."

This confirms that black-box testing is chosen because the tester does not need to understand the system’s internal structure.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.5, page 61 of 99)

 Labelling Approaches:Among the options provided, pre-labeled datasets require the least time and effort because the data has already been labeled, eliminating the need for further manual or automated labeling efforts.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 4.5 Data Labelling for Supervised Learning, which discusses various approaches to data labeling, including pre-labeled datasets, and their associated time and effort requirements​​.

The syllabus states:

“When testing probabilistic and non-deterministic systems, the same input may produce different outputs. Tests need to be run several times to produce statistically valid test results, ensuring that an appropriate number of answers are accurate.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.4, page 58 of 99)

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.