TsallisPGD: Adaptive Gradient Weighting for Adversarial Attacks on Semantic Segmentation
Abstract
Attacking semantic segmentation is harder than classification: an attacker must flip thousands of pixels at once, and standard cross-entropy (CE) overemphasizes already-misclassified pixels, overstating robustness. We introduce TsallisPGD, an attack built on the Tsallis cross-entropy — a generalization of CE parameterized by $q$ that controls gradient concentration across pixels. Since no single $q$ is universally optimal, we use a dynamic $q$-schedule. With one validation-selected schedule, TsallisPGD achieves the best average attack rank on Cityscapes, Pascal VOC, and ADE20K, improving over CEPGD, SegPGD, CosPGD, JSPGD, and MaskedPGD on both standard and robust models.
TL;DR
Attack Objective
For ground-truth probability $p_y$ and $q \ne 1$, Tsallis cross-entropy is
$$\mathcal{L}_q(p_y)=\frac{1-p_y^{1-q}}{1-q}.$$
It recovers standard cross-entropy as $q \to 1$. Its gradient is a confidence-weighted cross-entropy gradient:
$$\frac{\partial \mathcal{L}_q}{\partial p_y} =p_y^{1-q}\frac{\partial \mathcal{L}_{CE}}{\partial p_y}.$$
Dynamic $q$-Schedule
No single $q$ is best, so the attack sweeps $q$ linearly over iterations, from high-confidence targeting ($q=-2$) to CE-like refinement ($q=1$):
$$q_t=q_{\text{start}}+(q_{\text{end}}-q_{\text{start}})\frac{t}{T-1}.$$
The $[-2,1]$ schedule is selected once on a validation split, then frozen across all datasets, architectures, and budgets.
Core Idea: Gradient Allocation
A segmentation attack turns one perturbation into thousands of competing pixel losses, so attack strength depends on where gradient mass is spent. Cross-entropy's gradient grows as $p_y$ drops, wasting budget on already-fooled pixels while still-correct pixels stay under-attacked — which overstates robustness. Tsallis cross-entropy applies the confidence factor $p_y^{1-q}$: smaller $q$ redirects gradient toward higher-confidence pixels (peak at $p_y^\star=\tfrac{1-q}{2-q}$), larger $q$ recovers CE. TsallisPGD turns the loss into a control knob for gradient allocation.
Two-Pixel Toy Example
With one still-correct and one already-fooled pixel, CE spends almost all its gradient on the broken pixel:
| Pixel | $p_y$ | CE bound | Share |
|---|---|---|---|
| A — still correct | 0.80 | 0.24 | 11.5% |
| B — already fooled | 0.05 | 1.85 | 88.5% |
CE puts 88.5% of the weight on pixel B. At $q=-1$, Tsallis scales gradients by $p_y^2$ (squared-norm weight by $p_y^4$), shifting mass back to pixel A.
Evaluation
Protocol
- Threat model: $\ell_\infty$-bounded perturbations.
- Optimizer: APGD step-size selection.
- Budget: 300 iterations, single random start, and the multi-$\varepsilon$ trick.
- Metrics: pixel accuracy and mIoU, where lower is a stronger attack.
Coverage
- Datasets: Cityscapes, Pascal VOC, ADE20K.
- Models: PSPNet+ResNet-50, UPerNet+ConvNeXt-T, and Segmenter+ViT-S.
- Training regimes: clean, DDCAT, and PIR-AT robust checkpoints.
- Baselines: CEPGD, SegPGD, CosPGD, JSPGD, and MaskedPGD.
Results
Across all 21 evaluation settings, TsallisPGD has the best average rank — 1.38 in ACC (vs. MaskedPGD 2.43) and 1.57 in mIoU (vs. CosPGD 2.57) — and is best in 16/21 settings for ACC and 13/21 for mIoU (ties counted as best).
ACC Rank
- TsallisPGD: 1.38
- MaskedPGD: 2.43
- CosPGD: 2.81
mIoU Rank
- TsallisPGD: 1.57
- CosPGD: 2.57
- MaskedPGD: 2.76
The largest gains come on adversarially trained (DDCAT, PIR-AT) models, where gradient allocation matters most and a weak attack would otherwise overstate robustness. The one exception is MaskedPGD edging TsallisPGD on Cityscapes DDCAT at $\varepsilon=1/255$.
Citation
@inproceedings{matyasko2026tsallispgd,
title = {{TsallisPGD}: Adaptive Gradient Weighting for Adversarial Attacks on Semantic Segmentation},
author = {Matyasko, Alexander and Lou, Xin and Atmosukarto, Indriyati and Zhang, Wei},
booktitle = {International Joint Conference on Neural Networks (IJCNN)},
year = {2026}
}