iSQI CT-GenAI ISTQB Certified Tester Testing with Generative AI (CT-GenAI) v1.0 Exam Practice Test

In structured prompt engineering, theRolecomponent (also known as a Persona) is used to set the perspective, expertise, and tone of the LLM’s response. By assigning the role of a "senior test manager," the tester instructs the model to adopt the specific domain knowledge, vocabulary, and professional standards associated with that position. This technique is highly effective because LLMs are trained on vast datasets containing diverse professional documents; invoking a specific persona helps the model narrow its "latent space" to retrieve information relevant to that specific field. For instance, a senior test manager persona will prioritize risk management, resource allocation, and high-level strategy, whereas a "junior developer" persona might focus more on syntax and local unit tests. WhileContext(Option B) provides the background of the project andInstruction(Option A) defines the specific task to be performed, theRoleserves as the foundation for how those instructions are interpreted. This ensures the generated testware aligns with the expected professional seniority and organizational maturity required for high-stakes environments like a payments platform.

As Generative AI is integrated into the testing lifecycle, the role of the human tester undergoes a significant shift from "author" to "orchestrator and reviewer." In traditional testing, a significant portion of a tester's time is spent manually drafting test cases, scripts, and documentation. With GenAI, these artifacts can be generated in seconds. Consequently, the tester’s responsibility shifts towardreviewing, refining, and validatingthe AI-generated testware to ensure accuracy, relevance, and compliance with project goals. This "Human-in-the-Loop" (HITL) approach is critical because LLMs are prone to hallucinations and may lack the deep domain context of a human expert. Testers must apply their critical thinking to verify that the AI-generated scripts actually cover the necessary edge cases and do not contain logical errors. This evolution does not mean the end of human oversight (Option B) or a move exclusively to white-box testing (Option C). Instead, it elevates the tester to a higher-level analytical role, focusing on quality strategy and the final verification of AI outputs rather than the repetitive task of initial content creation.

Data security is a paramount concern when using GenAI in testing, as test environments often contain sensitive business logic or PII (Personally Identifiable Information). To protect this data "at rest" (stored in databases or vector stores) and "in transit" (being sent to the LLM), a combination of technical controls is required.Role-Based Access Control (RBAC)is a fundamental security pillar that ensures only authorized individuals or services can access specific datasets or trigger GenAI workflows. This prevents unauthorized users from feeding sensitive enterprise data into public AI models. While encryption (omitted in Option A as an alternative to obfuscation) and TLS (falsely suggested to be disabled in Option C) are essential technical layers for protecting data in transit, RBAC provides the organizational "gatekeeping" necessary to manage who can interact with the AI system. In a professional GenAI strategy, testers must ensure that the tools they use adhere to strict access policies, ensuring that the "Input Data" used for prompting remains within the secured organizational boundary and is not leaked to unauthorized entities or public training sets.

This scenario describes a failure in the model's ability to apply logical weight to specific domain concepts, specifically in the context of Risk-Based Testing (RBT). When an LLM ranks a low-impact UI element (a tooltip) higher than a critical functional failure (payment processing), it demonstrates a "Reasoning error in risk calculation logic." While LLMs can follow formulas like $Risk = Likelihood \times Impact$, they may lack the deep semantic understanding of "Impact" within a specific business domain unless explicitly guided. This is not necessarily a hallucination (Option C), as the model isn't necessarily inventing facts, but rather misapplying the logic of prioritization. It is also distinct from dataset bias (Option D), which would involve a systematic skewing across all outputs. In professional testing, this type of error highlights the necessity of "human-in-the-loop" verification. Testers must review AI-generated prioritizations to ensure that the logical deductions align with the actual business risk and technical criticality of the features being tested.

A hallucination refers to a phenomenon where a Large Language Model generates text that is grammatically correct and seemingly plausible but is factually incorrect or unsupported by the provided context or real-world data. In the context of software testing, this is a critical limitation. For example, an LLM might generate a test case for a software feature that does not exist or cite a non-existent API parameter. These errors occur because LLMs are probabilistic engines designed to predict the "most likely" next token rather than "reasoning" from a set of verified facts. They do not have a built-in "truth" mechanism. While a logical mistake (Option B) is a failure in reasoning and a systematic preference (Option D) describes bias, a hallucination is specifically about the fabrication of information. Testers must be particularly vigilant regarding hallucinations, as they can lead to "false confidence" in test coverage or the creation of invalid bug reports. Mitigations include grounding the model with Retrieval-Augmented Generation (RAG) and implementing rigorous "human-in-the-loop" verification of all AI-generated test artifacts.

Vision-Language Models (VLMs)represent a specialized subset of multimodal Large Language Models. Their defining characteristic is the ability to process, understand, and reason across both textual and visual modalities simultaneously. In the field of software testing, VLMs are revolutionary because they allow the AI to "see" a User Interface (UI). A tester can provide a screenshot of a web page alongside a natural language prompt, and the VLM can identify UI elements, detect visual regressions, or even validate that the visual layout matches a design specification. They are not a "superset" (Option C) of multimodal AI, but rather a specific implementation of it focused on the intersection of sight and language. Unlike traditional OCR or pixel-comparison tools used in legacy UI automation (Option B), VLMs understand thecontextof what they see—for instance, identifying a "broken" button icon that a human would recognize but a rule-based script might miss. This integration of visual and textual data is what makes them a vital component of modern, AI-augmented Quality Assurance strategies.

In a structured prompt, "Input Data" (or Reference Data) provides the specific subject matter that the model must process or analyze. The statement provided consists of factual identifiers and specific entities related to the System Under Test (SUT), such as the version number, the specific module name, reference IDs for existing tests, and a specific defect record. These elements serve as the raw material for the LLM's task. This differs from "Instructions" (Option C), which would be the command (e.g., "Analyze the following..."), or "Constraints" (Option B), which would define the boundaries of the task (e.g., "Do not include T-115"). "Output Format" (Option D) would define how the result should look (e.g., "Provide a JSON list"). By clearly labeling this section as Input Data, the tester helps the model distinguish between the "what" (the data) and the "how" (the instructions), which is a key principle of structured prompt engineering aimed at improving the accuracy of AI-generated analysis.

Understanding the hierarchy of LLM types is vital for selecting the right tool for specific testing tasks.Foundation LLMsare trained on massive datasets to predict the next token; they excel at broad, creative "ideation" (Statement i) but often struggle with following specific instructions or constraints (making Statement iv incorrect).Instruction-tuned LLMshave undergone additional training (Fine-tuning) to follow explicit commands and templates. They are highly effective at structured tasks like converting requirements into Gherkin feature files (Statement ii).Reasoning LLMs(or those utilizing specialized prompting like Chain-of-Thought) are designed to handle complex, multi-stage logic. This makes them the superior choice for diagnostic tasks like root-cause analysis, where the model must synthesize information across logs and requirements to find a defect's origin (Statement iii). Statement v is incorrect because while instruction-tuned models are capable, complex "stepwise reasoning" usually requires specific prompting techniques or the inherent logic of specialized reasoning models. Therefore, the combination of i, ii, and iii represents the correct alignment of model capability to testing functionality.

Differentiating between prompting techniques is essential for a tester to select the right tool for the task.Few-shot promptingis characterized by providing the model with a few examples of inputs and desired outputs, allowing it to learn the pattern and format.Prompt Chaininginvolves breaking a complex task into a sequence of smaller, interconnected prompts, where the output of one step becomes the input for the next (e.g., first extract requirements, then generate test cases from those requirements).Meta-promptingis a more advanced technique where the user asks the LLM to help design, write, or refine the prompt itself—essentially using the AI as a "prompt engineer" to optimize the instructions. Option D correctly identifies these core characteristics. Options A, B, and C contain fundamental mischaracterizations: for instance, Few-shotrequiresexamples (contradicting A), and Chaining is theoppositeof a single prompt (contradicting A). Mastering these distinctions allows testers to move from simple "chatting" to sophisticated AI orchestration that can handle complex, multi-stage testing workflows with high reliability.

In the context of LLM inference,Temperatureis a hyperparameter that controls the randomness or "creativity" of the model's output. When the temperature is set high, the model's probability distribution is "flattened," meaning it is more likely to select less-probable tokens, leading to more diverse and sometimes unpredictable text. For software testing, where precision and repeatability are paramount,lowering the temperature(Option C) is the standard practice. A temperature of 0.0 makes the model "deterministic," meaning it will consistently choose the token with the highest probability. This narrows the sampling distribution and significantly reduces variability between runs. While a larger context window (Option D) allows the model to process more information, it does not directly control the randomness of token selection. Similarly, the "learning rate" (Option B) is a parameter used during thetrainingorfine-tuningphase, not during inference. For generating test cases or scripts that must follow strict logic, a lower temperature ensures that the model remains focused and produces consistent results.

This scenario describes a form ofRequest Manipulation, specifically a type of "Prompt Injection" or "Adversarial Prompting." In this attack vector, the user (or an external attacker) provides malicious or deceptive input—in this case, via an image in a multimodal LLM—to bypass the model's intended constraints or to steer its logic toward an unintended outcome. By crafting an image that tricks the LLM into seeing "acceptance criteria" that aren't actually there, the attacker manipulates the model’s request processing to generate false validation results. This is different fromData Poisoning(Option A), which involves corrupting the training data before the model is even built. It is also distinct fromData Exfiltration(Option B), which aims to steal data from the model. In a testing environment, request manipulation is a significant risk because it can lead to "Silent Failures," where the AI reports that tests have passed or requirements are met based on deceptive input, thereby compromising the integrity of the entire Quality Assurance process.

Few-shot promptingis the technique of providing a few examples (exemplars) within the prompt to demonstrate the desired task and output format to the LLM. In this scenario, providing 10 existing, high-quality test cases acts as a "pattern" for the model to follow. This is significantly more effective than "Zero-shot prompting" (Option D), where the model is given a task without examples and may deviate from the specific organizational format required (e.g., specific JSON structures or assertion styles). While "Prompt chaining" (Option A) is useful for breaking down complex tasks into sub-tasks, the primary need here is pattern recognition and replication, which is the core strength of Few-shot learning. "Meta prompting" (Option C) involves having the AI write the prompt itself, which is unnecessary when the team already has clear examples. By using Few-shot prompting, the tester "conditions" the model's latent space to prioritize the provided format, ensuring that all 500 generated test cases maintain consistency with the HTTP methods, headers, and assertion logic defined in the exemplars.