Documentation

DEFault is a hierarchical DNN fault diagnosis tool built for Keras/TensorFlow practitioners. It uses a 3-stage process to detect faults in deep neural networks, categorize them into 5 types, and explain root causes using SHAP values.

The tool is based on research presented at ICSE 2025 and provides both a web-based platform for interactive use and a REST API for programmatic access.

Getting Started

Follow these steps to diagnose faults in your Keras model.

1
Navigate to the Platform
Open the DEFault platform from the main navigation or the home page.
2
Paste your Keras model code
Enter your model definition in the code editor, or upload a .py file. The code should include model architecture, compilation, and training calls.
3
Configure training parameters
Set the number of epochs, batch size, validation split, and other training options in the configuration panel.
4
Click "Train & Diagnose"
The system will execute your code in a sandboxed environment, train the model, and run the 3-stage fault analysis.
5
View results
Examine the detection verdict (fault or no-fault), fault category breakdown, SHAP-based feature explanations, and training performance curves.

API Reference

DEFault exposes three REST endpoints for programmatic fault analysis. All endpoints accept and return JSON unless noted otherwise.

POST/api/analyze-code

Static structural analysis of model code. Extracts features from the source code and runs the Stage 1 binary classifier to predict whether the model is faulty.

Request Body

application/json

{
  "code": "string",
  "model_name": "string"
}

Response

Returns an AnalyzeCodeResponse with fields: predicted_buggy, probability, top_features, and warnings.

POST/api/analyze-history

Full 3-stage analysis with training dynamics. Executes the model code in a sandboxed environment, trains the model, and runs all three diagnostic stages.

Request Body

Multipart form data with code, training configuration, and an optional dataset upload.

Response

Returns a FullAnalysisResponse with fields: stage1_detection, stage2_categories, and stage3_static.

Streaming

This endpoint streams Server-Sent Events (SSE) during training. Events include epoch-level metrics, stage transition markers, and a completion event with the full analysis result.

GET/api/fault-taxonomy

Returns the DNN fault taxonomy tree used by the system. The tree structure follows the taxonomy defined in the ICSE 2025 paper (Figure 3).

Response

application/json

{
  "label": "DNN Program Faults",
  "children": [
    {
      "label": "Activation Fault",
      "children": [...]
    },
    {
      "label": "Layer Fault",
      "children": [...]
    },
    ...
  ]
}

Fault Taxonomy

DEFault classifies detected faults into 5 categories. Each category corresponds to a specific aspect of the DNN program that may contain a bug.

Activation Fault

Wrong or missing activation function. For example, using sigmoid on the output layer of a multi-class classifier instead of softmax.

Layer Fault

Incorrect layer configuration, such as wrong kernel size, missing pooling layers, or an output layer with the wrong number of units.

Loss Function Fault

An inappropriate loss function for the task. For example, using binary crossentropy for a multi-class problem instead of categorical crossentropy.

Optimizer Fault

Suboptimal optimizer configuration. This includes choosing the wrong optimizer entirely or setting the learning rate too high or too low.

Hyperparameter Fault

Issues with training hyperparameters such as batch size, number of epochs, or validation split that prevent the model from converging properly.

Architecture

DEFault is built as a full-stack web application with a clear separation between the frontend interface and the backend analysis engine.

FrontendNext.js + TypeScript + Tailwind CSS

The user-facing interface with a code editor, real-time training visualization, and interactive result displays.

BackendFastAPI + Python

REST API server that handles code sandboxing, model training, and fault analysis. Streams training progress via SSE.

ML EngineTensorFlow/Keras, scikit-learn classifiers, SHAP

Trains user-submitted models, extracts static and dynamic features, runs the hierarchical classifier, and computes SHAP explanations.

Diagnosis Model3-stage hierarchical classifier

Stage 1 detects whether a fault exists (binary). Stage 2 categorizes the fault into one of 5 types (multi-label). Stage 3 performs static code analysis for additional structural checks. All classifiers were trained on curated DNN fault datasets from the ICSE 2025 study.