Two-stream joint matching based on mutual information for few-shot action recognition
CSTR:
Author:
Affiliation:

1.School of Big Data and Software Engineering, Chongqing University, Chongqing 400044, P. R. China;2.Southwest Computer Co., Ltd., Chongqing 400060, P. R. China

Clc Number:

TP181

Fund Project:

Supported by the Specialized Project for Technology Innovation and Application Development of Chongqing (CSTB2022TIAD-KPX0100), National Natural Science Foundation of China (62176031), and the Fundamental Research Funds for the Central Universities (2023CDJYGRHZD05).

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Although few-shot action recognition based on the metric learning paradigm has achieved significant success, it fails to address the following issues: 1) inadequate action relation modeling and underutilization of multi-modal information; 2) challenges in handling video matching problems with different lengths and speeds, and misaligned video sub-actions. To address these limitations, we propose a two-stream joint matching (TSJM) method based on mutual information, which consists of two modules: multi-modal contrastive learning module (MCL) and joint matching module (JMM). The MCL extensively explores inter-modal mutual information relationships, and thoroughly extracts modal information to enhance the modeling of action relationships. The JMM is primarily designed to simultaneously solve the aforementioned video matching problems. By integrating dynamic time warping (DTW) and bipartite graph matching, it optimizes the matching process to generate the final alignment results, thereby achieving high few-shot action recognition accuracy. We evaluate the proposed method on two widely used few-shot action recognition datasets (SSV2 and Kinetics), and conduct comprehensive ablation experiments to substantiate the efficacy of our approach.

    Reference
    Related
    Cited by
Get Citation

邓龙,冯波,葛永新.基于双流互信息联合匹配的小样本行为识别[J].重庆大学学报,2025,48(6):63~73

Copy
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:April 20,2024
  • Revised:
  • Adopted:
  • Online: July 11,2025
  • Published:
Article QR Code