精细拓扑结构表示与深度特征融合在多目标图像检索中的应用
作者:
中图分类号:

TP391

基金项目:

国家自然科学基金资助项目(61972333);湖南省教育厅优秀青年项目(18B504);湖南省自然科学基金项目(2018JJ3479);郴州市科技局科技发展计划项目(zdyf201906)。


On the application of detailed topological structure representation and deep feature fusion to multi-object image retrieval
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [25]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    空间信息表示是增强图像特征表达性能的重要手段,通过空间关系建模与深度学习方法融合可有效提升深度特征的语义特性,从而提升图像检索性能。首先,针对复杂图像的空间关系表示提出了一种新的精细拓扑结构表示模型,该模型不仅具有完备的拓扑描述性能,还提供了两种拓扑不变量的推理算法,使得拓扑不变量可以由表示模型直接推导而不需要繁复的几何计算;其次,基于精细拓扑结构表示模型,提出了有效的拓扑结构相似性度量方法,为空间关系特征表达奠定了基础;最后,进一步结合卷积神经网络,提出融合复杂空间关系特征与深度特征的多目标图像检索方法。实验结果表明,所提出的拓扑结构表示模型在空间查询中具有良好的性能;所提出的图像检索框架取得优于现有方法的精度,并能够有效地结合手工特征与深度特征各自的优势,为提升深度学习方法的可解释性创造了有利条件。

    Abstract:

    Spatial information representation is an important means to improve image visual feature representation performance. The integration of the spatial relation model with deep learning can effectively enhance semantic property of deep features, improving the image retrieval accuracy. In this paper, we proposed a novel detailed topological structure representation model to describe spatial relation of complex images. This model not only had complete topology description performance, but also provided two efficient reasoning algorithms, which made the topological invariants directly deduced from the model without any geometric calculations. Similarity matching approaches based on fine topological structure representation model was proposed for spatial relationship feature representation. Finally, in combination with convolution neural network, a multi-object image retrieval framework was developed by fusing the spatial relation features and deep features. Experimental results demonstrate that the proposed topological model has remarkable performance in spatial query. Moreover, the proposed image retrieval framework outperforms the current methods in terms of precision and with advantages of both the manual and deep features, it provides a superior means to improve interpretability of deep learning methods.

    参考文献
    [1] Cheng Q M, Zhang Q, Fu P, et al. a survey and analysis on automatic image annotation[J]. Pattern Recognition, 2018, 79:242-259.
    [2] Zheng L, Yang Y, Tian Q. SIFT meets CNN:a decade survey of instance retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5):1224-1244.
    [3] Pietikäinen M, Zhao G. Two decades of local binary patterns:A survey[M]. Advances in Independent Component Analysis and Learning Machines. Academic Press, 2015:175-210.
    [4] Jabeen S, Mehmood Z, Mahmood T, et al. An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model[J]. PloS One, 2018, 13(4):e0194526.
    [5] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
    [6] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. 2014:arXiv:1409.1556[cs.CV]. https://arxiv.org/abs/1409.1556
    [7] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA:IEEE, 2016:770-778.
    [8] Zhang M M, Li W, Du Q. Diverse region-based CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2018, 27(6):2623-2634.
    [9] 刘智,冯欣,张杰. 基于深度卷积神经网络和深度视频的人体行为识别[J]. 重庆大学学报, 2017, 40(11):99-106.Liu Z, Feng X, Zhang J. Action recognition based on deep convolution neural network and depth sequences[J]. Journal of Chongqing University, 2017, 4(11):99-106. (in Chinese)
    [10] Liu Y Y, Yuan X H, Gong X, et al. Conditional convolution neural network enhanced random forest for facial expression recognition[J]. Pattern Recognition, 2018, 84:251-261.
    [11] Gong C, Anders R A, Zhu Q, et al. Quantitative characterization of CD8+ T cell clustering and spatial heterogeneity in solid tumors[J]. Frontiers in Oncology, 2018, 8:649-690.
    [12] 于瑞云, 苏展, 谢青, 等. 基于空间分割的人体模型骨骼提取算法[J]. 计算机学报,2019,42(9):2049-2061.Yu R Y, Su Z, Xie Q, et al. Skeleton extraction of character model based on space segmentation[J]. Chinese Journal of Computers, 2019, 42(9):2049:2061. (in Chinese)
    [13] Chen J, Cohn A G, Liu D, et al. A survey of qualitative spatial representations[J]. The Knowledge Engineering Review, 2015, 30(1):106-136.
    [14] Li S J, Liu W M. Topological relations between convex regions[C]//The Twenty-Fourth AAAI Conference on Artificial Intelligence, Atlanta, USA:AAAI, 2010:321-326.
    [15] Wang S S, Liu D, Zhang C, et al. Representation, reasoning and similar matching for detailed topological relations with DTString[J]. Information Sciences, 2014, 276:255-277.
    [16] Alajlan N, Kamel M S, Freeman G H. Geometry-based image retrieval in binary image databases[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(6):1003-1013.
    [17] Li S J. A layered graph representation for complex regions[C]//The 12th International Conference on Principles of Knowledge Representation and Reasoning, Toronto, Canada, Morgan Kuafmann, 2010:581-583.
    [18] Aresta G, Araújo T, Kwok S, et al. Bach:Grand challenge on breast cancer histology images[J]. Medical Image Analysis, 2019:1-19.
    [19] Escalera S, Alicia Fornés, Pujol O, et al. Blurred shape model for binary and grey-level symbol recognition[J]. Pattern Recognition Letters, 2009, 30(15):1424-1433.
    [20] Lyzinski V, Fishkind D E, Fiori M, et al. Graph matching:relax at your own risk[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(1):60-73.
    [21] Abbas S, Farhan S, Fahiem M A, et al. Efficient shape classification using zernike moments and geometrical features on MPEG-7 dataset[J]. Advances in Electrical and Computer Engineering, 2019, 19(1):45-51.
    [22] Wei C H, Li Y, Chau W Y, et al. Trademark image retrieval using synthetic features for describing global shape and interior structure[J]. Pattern Recognition, 2010, 42(3):386-394.
    [23] Ni H, Guo Z H, Huang B Q. Binary patent image retrieval using the hierarchical oriented gradient histogram[C]//2015 International Conference on Service Science (ICSS), Weihai, China:IEEE, 2015:23-27.
    [24] Zhang Z, Tang P, Duan R. Dynamic time warping under pointwise shape context[J]. Information Sciences, 2015, 315:88-101.
    [25] Wang X G, Feng B, Bai X, et al. Bag of contour fragments for robust shape classification[J], Pattern Recognition. 2014, 47(6):2116-2125.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

刘东,王生生.精细拓扑结构表示与深度特征融合在多目标图像检索中的应用[J].重庆大学学报,2021,44(3):132-143.

复制
分享
文章指标
  • 点击次数:567
  • 下载次数: 818
  • HTML阅读次数: 863
  • 引用次数: 0
历史
  • 收稿日期:2019-11-21
  • 在线发布日期: 2021-03-31
文章二维码