Adversarial Training
Expanded definition
Adversarial training adds adversarial examples to the learning process so a model must perform well on both ordinary inputs and difficult perturbed inputs. In image classification, a common threat model limits how much an attacker can change each pixel while still searching for a perturbation that causes a wrong prediction. The training loop generates these examples against the current model and updates the model on them. The method is usually framed as a robust optimization problem: minimize loss on the worst-case perturbation allowed by the threat model. Fast Gradient Sign Method (FGSM) training uses a one-step perturbation. Projected Gradient Descent (PGD) adversarial training searches with multiple steps inside the allowed perturbation set. TRADES separates the natural-accuracy objective from a robustness regularizer and exposes an explicit trade-off parameter. Adversarial training is a practical defense, not a blanket guarantee. Robustness must be measured against the attacks, perturbation bounds, norms, data distribution, and evaluation procedure that matter for the deployment. A model trained against one threat model can still fail outside it.
Common adversarial training patterns
The training pattern should match the deployment threat model and evaluation budget. These methods are empirical defenses: compare them with clean and attacked validation sets under the same perturbation rules.
| Pattern | Training path | Representative method | Operational profile |
|---|---|---|---|
| FGSM-based training | Generate a one-step sign-gradient perturbation and include the perturbed input in training. | Fast Gradient Sign Method from Goodfellow et al. | Lower-cost robustness baseline; evaluate against stronger multi-step attacks. |
| PGD adversarial training | Run multiple projected gradient steps inside the allowed perturbation set before each model update. | Robust optimization baseline from Madry et al. | Stronger empirical defense with higher training cost. |
| TRADES | Optimize natural prediction loss alongside a robustness regularizer controlled by a trade-off parameter. | TRADES objective from Zhang et al. | Explicit tuning between clean accuracy and adversarial robustness. |
| Domain-specific adversarial augmentation | Generate difficult examples that reflect the deployment surface, then train and evaluate against those cases. | Application-specific perturbation and red-team suites | Targeted coverage for a documented threat model; not a general robustness guarantee. |
Standard training vs adversarial training
| Dimension | Standard training | Adversarial training |
|---|---|---|
| Training inputs | Use the available training samples and ordinary augmentation. | Add deliberately perturbed samples generated against the current model or a defined attack suite. |
| Primary objective | Optimize predictive performance on the observed data distribution. | Optimize predictive performance while reducing failures inside a defined perturbation set. |
| Evaluation | Measure clean task metrics on held-out data. | Measure clean metrics and robust metrics under documented attacks, bounds, and norms. |
| Compute profile | Use the ordinary training budget. | Spend additional training compute to generate adversarial examples, especially for multi-step attacks. |
| Guarantee boundary | No claim of robustness to adversarial perturbations. | Empirical robustness is limited to the trained and evaluated threat models unless a separate certification method is used. |
Use adversarial training when manipulated inputs are a credible deployment risk and the team can define a threat model, pay the training cost, and maintain a robust evaluation suite. Keep standard training as the clean-accuracy baseline.
Related terms
Explore adjacent ideas in the knowledge graph.
Related
Comparisons, tools, and models that connect to this idea.