Dr. Stefan Haas

After completing this module, students will be able to:

- Contextualize uncertainty awareness within the EU AI Act, including risk tiers and requirements for high-risk systems

- Distinguish between aleatoric (data) and epistemic (model) uncertainty in machine learning

- Quantify uncertainty in classification using predictive probability distributions and measures such as entropy and margin

- Quantify uncertainty in regression using prediction intervals (e.g., quantile regression)

- Assess and improve the calibration of model outputs using standard diagnostic tools (e.g., ECE, reliability diagrams) and correction methods (e.g., temperature scaling)

- Construct statistically calibrated prediction sets and intervals with coverage guarantees using conformal prediction

- Quantify epistemic uncertainty through approximate Bayesian inference, including (Deep) Ensembles and Monte Carlo (MC) Dropout

- Detect out-of-distribution (OOD) inputs in computer vision, e.g., previously unseen medical conditions in images

- Quantify uncertainty in LLM-generated outputs using white-box, black-box, and LLM-as-a-judge approaches

- Apply these methods in practice using libraries such as PyTorch/skorch, skrub, scikit-learn, LangChain, MAPIE and UQLM

- Design uncertainty-aware AI systems incorporating selective classification and human-in-the-loop (HITL) workflows