Weakly supervised temporal action localization method based on modal bias compensation
CSTR:
Author:
Affiliation:

1.School of Big Data Software Engineering,Chongqing University;2.Department of Computer Science and Engineering, Shanghai Jiao Tong University

Clc Number:

TP389.1???????

Fund Project:

National Natural Science Foundation of China General Program(62176031); The Chongqing Special Key Project for Technological Innovation and Application Development(CSTB2022TIAD-KPX0100).

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Weakly supervised temporal action localization has become one of the research hotspots in the field of video understanding due to its application potential in intelligent monitoring, video retrieval and other fields, and its low cost of training data annotation. In response to the poor localization performance caused by existing multimodal learning-based localization methods ignoring the biases inherent in each modality, we constructed an RGB action subject information compensation module and designed a optical flow-based dominant influence suppression strategy aimed at eliminating the location bias caused by each modality on the training model. Experimental results on two benchmark datasets THUMOS14 and ActivityNet v1.2 show that under multi-scale temporal intersection over union, mean average precision reached 45.3% and 26.5% respectively, overall localization performance is better than some latest methods, which demonstrates effectiveness of our proposed method. This method improves the basic localization performance of temporal action location models by compensating for bias at a coarse-grained modal level. It is also compatible with fine-grained viewpoint localization methods.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:April 09,2024
  • Revised:May 22,2024
  • Adopted:July 31,2024
  • Online:
  • Published:
Article QR Code