About DEFault
A research tool for automated DNN fault diagnosis, developed at Dalhousie University and published at ICSE 2025.
The Paper
DEFault: Detect and Explain Faults in DNNs using Hierarchical Classification
ICSE 2025 — 47th IEEE/ACM International Conference on Software Engineering
Author: Sigma Jahan
Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada
DEFault is a hierarchical approach to detect, categorize, and explain faults in deep neural network programs. It combines static code analysis with dynamic training metrics in a 3-stage pipeline: binary fault detection, 5-category classification, and SHAP-based root cause analysis. The tool processes Keras/TensorFlow model code and training history to provide actionable debugging recommendations.
Why DEFault?
DNN debugging is hard. Traditional software testing techniques do not transfer well to deep learning because faults manifest as degraded accuracy rather than crashes. A model can train without errors yet produce poor predictions, leaving developers with little guidance on what went wrong.
Existing approaches either focus only on detection (is the model buggy?) without explaining why, or they require manual expertise to interpret training dynamics. Practitioners need tools that go beyond a binary verdict and tell them where to look in their code.
DEFault bridges this gap with a hierarchical approach: first detect whether a fault exists, then categorize it into one of five actionable fault types, and finally explain the root cause at the code level using SHAP feature attribution. Each stage builds on the previous one to narrow down the problem.
Key Contributions
Hierarchical 3-stage diagnosis
Detection, categorization, and root cause analysis in one unified flow.
5 DNN fault categories
Activation, layer, loss function, optimizer, and hyperparameter faults based on a systematic taxonomy.
SHAP-based explanations
Feature-level attribution that pinpoints which code-level properties drive fault predictions.
Interactive web tool
Real-time training visualization, code editor, and session management for practical DNN debugging.
Sigma Jahan
PhD Student, Faculty of Computer Science
Dalhousie University, Halifax, NS, Canada
Research interests: Software engineering for AI/ML systems, DNN testing and debugging, explainable AI.
github.com/sigmaJahan/DEFault-Tool