In this chapter, we focus on before/during training backdoor defense, where the defender is also the training authority, with control of the training process and responsibility for providing an accurate, backdoor-free DNN classifier. Deployment of a backdoor defense during training is supported by the fact that the training authority is usually more resourceful in both computation and storage than a downstream user of the trained classifier. Moreover, before/during training detection could be easier than post-training detection because the defender has access to the (possibly poisoned) training set and, thus, to samples that contain the backdoor pattern. However, before/during training detection is still highly challenging because it is unknown whether there is poisoning and, if so, which subset of samples (among many possible subsets) is poisoned. A detailed review of backdoor attacks (Trojans) is given, and optimization-based reverse-engineering defense for training set cleansing deployed before/during classifier training is described. The defense is designed to detect backdoor attacks on samples with a human-imperceptible backdoor pattern, as widely considered in existing attacks and defenses. Detection of training set poisoning is achieved by reverse engineering (estimating) the pattern of a putative backdoor attack, considering each class as the possible target class of an attack.

In this chapter we describe unsupervised post-training defenses that do not make explicit assumptions regarding the backdoor pattern or how it was incorporated into clean samples. These backdoor defenses aim to be “universal.” They do not produce an estimate of the backdoor pattern (which may be valuable information as the basis for detecting backdoor triggers at test time, the subject of Chapter 10). We start by describing a universal backdoor detector that does not require any clean labeled data. This approach optimizes over the input image to the DNN, seeking the input that yields the maximum margin (for each putative target class of an attack). The premise here, under a winner-take-all decision rule, is that backdoors produce much larger classifier margins than those of un-attacked examples. Then a universal backdoor mitigation strategy is described that does leverage a small clean dataset. This optimizes a threshold (tamping down unusually large ReLU activations) for each neuron in the network. In each backdoor attack scenario described, different detection and mitigation strategies are compared, where some mitigation strategies are also known as “unlearning” defenses. Some universal backdoor defenses modify or augment the DNN itself, while others do not.

In this chapter we focus on post-training defense against backdoor data poisoning (Trojans). The defender has access to the trained DNN but not to the training set. The following are examples. (i) Proprietary: a customized DNN model purchased by government or a company without data rights and without training set access. (ii) Legacy: the data is long forgotten or not maintained. (iii) Cell phone apps: the user has no access to the training set for the app classifier. It is also assumed that a clean labeled dataset (no backdoor poisoning) is available with a small number of examples from each of the classes from the domain. This clean labeled dataset is insufficient for retraining and its small size makes its availability a reasonable assumption. Reverse-engineering defenses (REDs) are described including one that estimates putative backdoor patterns for each candidate (source class, target class) backdoor pair and then assesses an order statistic p-value on the sizes of these perturbations. This is successful at detecting subtle backdoor patterns, including sparse patterns involving few pixels, and global patterns where many pixels are modified subtly. A computationally efficient variant is presented. The method addresses additive backdoor embeddings and other embedding functions.

In this chapter we focus on post-training detection of backdoor attacks which replace a patch of pixels by a common backdoor pattern. We focus on scene-plausible perceptible backdoor patterns. Scene-plausibility is important for a perceptible attack to be evasive to human and machine-based detection, whether the attack is physical or digital. Though the focus is on image classification, the methodology could be applied to audio, where for “scene-plausibility” the backdoor pattern does not sound artificial or incongruous, amongst other sounds in the audio clip. For the Neural Cleanse method, the common backdoor pattern may be scene plausible or incongruous. In the latter case, backdoor trigger images (at test time) might be noticed by a human, thus thwarting the attack. The focus here is on defending against patch attacks that are scene plausible, meaning that the backdoor pattern cannot in general be embedded into the same location in every (poisoned) image. For example, a rainbow (one of the attack patterns) must be embedded in the sky (and this location may vary). The main method described builds on RED. It exploits the need for scene-plausibility, and attack “durability,” that the backdoor trigger will be effective in the presence of noise and occlusion.

In this chapter, we focus on post-training backdoor defense for classification problems involving only a few classes, particularly just two classes (K = 2), and involving arbitrary numbers of backdoor attacks, including different backdoor patterns with the same source and/or target classes. In Chapter 6, null models were estimated using (K – 1)2 statistics. For K = 2, only one such statistic is available, which is insufficient for estimating a null density model. Thus, the detection inference approach cannot be directly applied in the two-class case. Other detection statistics, such as the median absolute deviation (MAD) statistic used by Neural Cleanse, are also unsuitable for the two-class case. The developed method relies on high transferability of putative backdoor patterns that are estimated sample-wise, that is, a perturbation specifically designed to cause one sample to be misclassified also induces other (neighboring) samples to be misclassified. Intriguingly, the proposed method works effectively with a common (theoretically derived) detection threshold, irrespective of the classification domain and the particular attack. This is significant, as it may be difficult to set the detection threshold for any method in practice. The proposed method can be applied for various attack embedding functions (additive, patch, multiplicative, etc.).

Previous chapters considered detection of backdoors before/during training and post-training. Here, our objective is to detect use of a backdoor trigger operationally, that is, at test time. Such detection may prevent potentially catastrophic decisions, as well as potentially catching culprits in the act of exploiting a learned backdoor mapping. We also refer to such detection as “in-flight.” A likelihood based backdoor trigger detector is developed and compared against other detectors.