After successful completion of this module, students are able to elicit and formulate requirements for technical systems, including AI-supported requirements elicitation. They can design, implement and statistically evaluate systematic test, in particular for systems that are represented via AI/LLM-based behavioral surrogates, spanning unit to end-to-end tests. They can construct behavioral and adversarial tests to uncover weaknesses in such AI/LLM-based behavioral surrogates, detect hallucinations, factuality and groundedness issues, and compare different LLMs or prompt variants used to represent the technical system under test. They can integrate automated tests into CI/CD pipelines with automated evaluation gates. Students are able to independently plan, implement and document an end-to-end, CI/CD-integrated test suite for a technical system.
Modern technical systems are increasingly represented, simulated, or operated through LLM-based components. Ensuring their trustworthiness requires both classical V&V and AI-specific evaluation techniques; this module focuses on
Fundamentals of requirements elicitation, including AI-supported techniques
Fundamentals of testing (unit, integration, system and end-to-end tests) and statistics, including AI-based techniques
Comparing different LLMs and prompt variants used as a behavioral surrogate for the system under test
Adversarial testing and robustness testing with prompt injection, jailbreaks and edge cases
Detecting hallucinations, factuality and groundedness issues
End-to-end pipeline testing, test automation and CI/CD for test pipelines, automated evaluation gates