# Donghwan Shin

> Senior Lecturer (Associate Professor) in Software Engineering, School of Computer Science, University of Sheffield, UK. Research on software testing for AI-enabled autonomous systems, mutation testing, and log analysis.

I am a Senior Lecturer (Associate Professor in the American system) in the School of Computer Science at the University of Sheffield, a member of the AI x Software Engineering & Testing (ASET) group, and a PI of the Intelligent Testing Lab. I received my PhD, MS and BS in Computer Science from KAIST, advised by Prof. Doo-Hwan Bae, and then spent four years as a research associate and research scientist in Prof. Lionel Briand’s SVV group at the SnT Centre, University of Luxembourg.

Contact: d.shin@sheffield.ac.uk

## Research themes

- Testing for AI: Making AI-enabled autonomous systems, such as self-driving cars, safe and reliable through simulation-based, search-based and multi-fidelity testing, failure simplification, and debugging.
- Mutation Testing: Understanding whether artificial faults can stand in for real ones, and making mutation analysis effective and scalable, from theory to efficient tools for Rust and safety-aware mutation for ADS.
- Log Analysis: Using software execution logs to solve software engineering problems, including model inference, log parsing, anomaly detection, and failure prediction.

## Key resources

- [Full page as Markdown](https://dshin.info/index.md): bio, research, all publications with summaries and BibTeX, projects, people, service.
- [Homepage](https://dshin.info/)

## Featured publications

- [Multi-Fidelity Bayesian Optimization for Simulation Based Autonomous Driving Systems Testing](https://dshin.info/#iv26-mfbo) (IV'26): Realistic driving simulations take minutes to hours each, which limits simulation-based ADS testing. MFBO-Drive treats critical scenario generation as multi-fidelity optimisation, choosing both which scenario to run and at what fidelity. Key implication: across more than 100,000 MetaDrive scenarios it improved cost-effectiveness by 16.8% over state-of-the-art Bayesian optimisation with comparable test quality.
- [Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems](https://dshin.info/#morlot) (ICSE'23): Online testing of DNN-enabled systems must find scenarios that break several safety requirements, while the environment changes during a run. MORLOT combines reinforcement learning, which changes the environment step by step, with many-objective search that targets requirements not yet violated. Key implication: on Transfuser in CARLA it was significantly more effective and efficient than alternatives, with a large effect size.
- [Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and Many-Objective Optimization](https://dshin.info/#samota) (ICSE'22, ACM SIGSOFT Distinguished Paper Award): Testing a DNN-enabled driving system in a high-fidelity simulator is slow, so few scenarios can be tried. SAMOTA trains cheap surrogate models of the simulator and uses them inside many-objective search to find safety requirement violations. Key implication: on Pylot in CARLA it was significantly more effective and efficient than state-of-the-art search and random testing.
- [Are Mutation Scores Correlated with Real Fault Detection? A Large Scale Empirical Study on the Relationship Between Mutants and Real Faults](https://dshin.info/#icse18-mutants) (ICSE'18): Studies use mutants as stand-ins for real faults because mutation scores seem to correlate with fault detection. This large study on CoREBench and Defects4J shows the correlation becomes weak once test suite size is controlled, yet suites with high mutation scores still detect more real faults than random suites of the same size. Key implication: mutants are useful guidance, but researchers must control for suite size.
- [A Theoretical and Empirical Study of Diversity-aware Mutation Adequacy Criterion](https://dshin.info/#d-mutation) (TSE'17): The classic mutation criterion only asks whether mutants are killed, so one test killing every mutant counts as adequate. The distinguishing criterion requires tests to tell all mutants apart by their kill patterns, and is proven to subsume the classic one. Key implication: on 352 Defects4J faults, adequate suites detected 1.33 times more faults with 1.56 times more tests.

## Projects

- MF-Drive: Scalable and Sustainable Simulation-based Testing of Autonomous Vehicles using Multi-Fidelity Optimisation (PI, NVIDIA Academic Grant Program, Mar 2026 – Feb 2027): A multi-fidelity optimisation framework combining Software-in-the-Loop and Hardware-in-the-Loop validation to find safety violations and hardware-induced failures in autonomous driving systems.
- Simulation-Based Testing for Mobility Cyber-Physical Systems of Systems (Co-PI, IITP (South Korea), with KAIST Web Engineering Lab, Apr 2025 – Dec 2026): Testing and debugging approaches for systems of multiple independent autonomous vehicles sharing the same roads.
- SimpliFaiS: Simplification of Failure Scenarios for Machine Learning-enabled Autonomous Systems (PI, EPSRC New Investigator Award, Jan 2024 – Jun 2026): Automated methods to identify the minimal set of entities that cause failures in critical driving scenarios.