Research Brief

Automated Background Swapping (AutoBackSwap) is a novel data augmentation method that prevents deep neural networks from relying on spurious background correlations by disentangling foregrounds and synthesizing new backgrounds.

Deep Neural Networks (DNNs) can become unreliable if they learn to associate an object with its background rather than the object itself, a phenomenon known as spurious correlation. This paper introduces AutoBackSwap, a technique designed to make AI classifiers more robust by reducing their dependence on these misleading background features. AutoBackSwap works by using a specialized secondary network to separate the main subject (foreground) from its surroundings (background). It then generates new, diverse backgrounds using an infilling process and combines these with different foregrounds to create augmented training data. This approach requires only a small amount of human-labeled data for the secondary network, yet it can automatically expand the training dataset significantly. A key strength of AutoBackSwap is its effectiveness even when the original training data contains no examples that break the spurious correlation, allowing it to consistently outperform previous methods on challenging image classification tasks.

• Autonomous driving: Ensuring vehicles recognize pedestrians or traffic signs based on their features, regardless of varying urban or rural backgrounds, preventing dangerous misclassifications.
• Medical image diagnosis: Improving the reliability of AI in detecting diseases from scans by ensuring it focuses on pathological features rather than irrelevant background elements (e.g., scanner artifacts or patient position).
• Industrial quality control: Enabling vision systems to accurately identify defects in products even when the manufacturing environment or lighting conditions change, leading to more robust inspection.
• Security and surveillance: Enhancing facial recognition or object detection systems to perform reliably across diverse environments, mitigating the impact of scene variations on identification accuracy.