Salient Concept-Aware Generative Data Augmentation

Abstract

Recent generative data augmentation methods conditioned on both image and text prompts struggle to balance between fidelity and diversity, as it is challenging to preserve essential image details while aligning with varied text prompts. This challenge arises because representations in the synthesis process often become entangled with non-essential input image attributes such as environmental contexts, creating conflicts with text prompts intended to modify these elements. To address this, we propose a personalized image generation framework that uses a salient concept-aware image embedding model to reduce the influence of irrelevant visual details during the synthesis process, thereby maintaining intuitive alignment between image and text inputs. By generating images that better preserve class-discriminative features with additional controlled variations, our framework effectively enhances the diversity of training datasets and thereby improves the robustness of downstream models. Our approach demonstrates superior performance across eight fine-grained vision datasets, outperforming state-of-the-art augmentation methods with averaged classification accuracy improvements by 0.73% and 6.5% under conventional and long-tail settings, respectively.

Summary

Proposes a salient concept-aware generative augmentation framework that disentangles class-discriminative features from irrelevant context, improving fine-grained recognition under both conventional and long-tail settings.

Key Contributions

FAQ

What problem does salient concept-aware augmentation solve?

Standard generative augmentation methods entangle class-discriminative features with irrelevant context (e.g., background), so text prompts that modify the background can corrupt the object identity. Salient concept-aware embedding isolates the class-relevant features, allowing diverse generation without identity loss.

How does this differ from standard image-conditioned generation?

Rather than using the full image embedding (which includes background, lighting, and other irrelevant attributes), this method uses a salient concept-aware embedding that focuses on class-discriminative features, reducing conflicts between image and text conditioning.

Why is this particularly effective for long-tail recognition?

Long-tail classes have very few training samples, so augmentation quality matters more. By preserving class identity while varying context, the method generates more useful synthetic samples for rare classes, yielding a 6.5% accuracy improvement under long-tail settings.

What datasets were used for evaluation?

The method was evaluated on eight fine-grained vision datasets under both conventional and long-tail settings, outperforming state-of-the-art augmentation methods in both regimes.

Can this framework be used with different generative models?

Yes. The salient concept-aware embedding is a general approach that can interface with various text-to-image diffusion models for personalized generation.

Keywords

Related Papers

• Diversity-Sensitive Conditional Generative Adversarial Networks (ICLR 2019)
• Adversarial Defense via Learning to Generate Diverse Attacks (ICCV 2019)
• Model Diagnosis and Correction via Linguistic and Implicit Attribute Editing (CVPR 2025)
• Decoupling Vision and Language: Codebook Anchored Visual Adaptation (arXiv 2026)

BibTeX

@article{zhao2025salient,
  title={Salient Concept-Aware Generative Data Augmentation},
  author={Zhao, Tianchen and Chen, Xuanbai and Li, Zhihua and Fang, Jun and An, Dongsheng and Xu, Xiang and Tu, Zhuowen and Xing, Yifan},
  journal={arXiv preprint arXiv:2510.15194},
  year={2025}
}

Recommended Citation Contexts

• Identifying salient visual concepts before generating augmentation data improves downstream classifier performance over random augmentation strategies (Zhao et al., 2025).