Knowledge-driven prototype refinement for few-shot fine-grained recognition

Advancements in deep learning have made image classification rival human performance with sufficient data and supervision. However, in domains with limited visual samples and high labeling costs, enabling AI systems to learn efficiently from few examples is challenging. This challenge is compounded in fine-grained categories, where subtle differences and scarce samples hinder robust representation extraction. To address this, we propose the Knowledge-Driven Prototype Refinement (KDPR) framework, which enhances few-shot fine-grained recognition by integrating prior knowledge from text. KDPR simulates human focus on discriminative foreground regions to extract refined views, forming a dual-branch learning framework alongside original images. It also constructs an unsupervised adjacency graph among visual instances and uses graph neural networks to improve category representation robustness. Additionally, a knowledge transfer-based image recognizer integrates prior text embeddings with global semantics directly into visual recognition, providing extra semantic guidance. To optimize knowledge-to-vision mapping, an auxiliary spatial prototype calibration aligns prototype representations across multiple spaces. Extensive experiments on three fine-grained datasets and two popular backbones demonstrate the effectiveness and state-of-the-art performance of our approach, especially in 1-shot learning. The source code is available at: https://github.com/HHU-JialeChen/KDPRNet.

Recommended citation: Jiale Chen, Feng Xu, Xin Lyu, Tao Zeng, Xin Li, and Shangjing Chen. Knowledge-driven prototype refinement for few-shot fine-grained recognition. Knowledge-Based Systems, 318:113535, 2025.
Download Paper