On the application of detailed topological structure representation and deep feature fusion to multi-object image retrieval
CSTR:
Author:
Clc Number:

TP391

  • Article
  • | |
  • Metrics
  • |
  • Reference [25]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    Spatial information representation is an important means to improve image visual feature representation performance. The integration of the spatial relation model with deep learning can effectively enhance semantic property of deep features, improving the image retrieval accuracy. In this paper, we proposed a novel detailed topological structure representation model to describe spatial relation of complex images. This model not only had complete topology description performance, but also provided two efficient reasoning algorithms, which made the topological invariants directly deduced from the model without any geometric calculations. Similarity matching approaches based on fine topological structure representation model was proposed for spatial relationship feature representation. Finally, in combination with convolution neural network, a multi-object image retrieval framework was developed by fusing the spatial relation features and deep features. Experimental results demonstrate that the proposed topological model has remarkable performance in spatial query. Moreover, the proposed image retrieval framework outperforms the current methods in terms of precision and with advantages of both the manual and deep features, it provides a superior means to improve interpretability of deep learning methods.

    Reference
    [1] Cheng Q M, Zhang Q, Fu P, et al. a survey and analysis on automatic image annotation[J]. Pattern Recognition, 2018, 79:242-259.
    [2] Zheng L, Yang Y, Tian Q. SIFT meets CNN:a decade survey of instance retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5):1224-1244.
    [3] Pietikäinen M, Zhao G. Two decades of local binary patterns:A survey[M]. Advances in Independent Component Analysis and Learning Machines. Academic Press, 2015:175-210.
    [4] Jabeen S, Mehmood Z, Mahmood T, et al. An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model[J]. PloS One, 2018, 13(4):e0194526.
    [5] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
    [6] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. 2014:arXiv:1409.1556[cs.CV]. https://arxiv.org/abs/1409.1556
    [7] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA:IEEE, 2016:770-778.
    [8] Zhang M M, Li W, Du Q. Diverse region-based CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2018, 27(6):2623-2634.
    [9] 刘智,冯欣,张杰. 基于深度卷积神经网络和深度视频的人体行为识别[J]. 重庆大学学报, 2017, 40(11):99-106.Liu Z, Feng X, Zhang J. Action recognition based on deep convolution neural network and depth sequences[J]. Journal of Chongqing University, 2017, 4(11):99-106. (in Chinese)
    [10] Liu Y Y, Yuan X H, Gong X, et al. Conditional convolution neural network enhanced random forest for facial expression recognition[J]. Pattern Recognition, 2018, 84:251-261.
    [11] Gong C, Anders R A, Zhu Q, et al. Quantitative characterization of CD8+ T cell clustering and spatial heterogeneity in solid tumors[J]. Frontiers in Oncology, 2018, 8:649-690.
    [12] 于瑞云, 苏展, 谢青, 等. 基于空间分割的人体模型骨骼提取算法[J]. 计算机学报,2019,42(9):2049-2061.Yu R Y, Su Z, Xie Q, et al. Skeleton extraction of character model based on space segmentation[J]. Chinese Journal of Computers, 2019, 42(9):2049:2061. (in Chinese)
    [13] Chen J, Cohn A G, Liu D, et al. A survey of qualitative spatial representations[J]. The Knowledge Engineering Review, 2015, 30(1):106-136.
    [14] Li S J, Liu W M. Topological relations between convex regions[C]//The Twenty-Fourth AAAI Conference on Artificial Intelligence, Atlanta, USA:AAAI, 2010:321-326.
    [15] Wang S S, Liu D, Zhang C, et al. Representation, reasoning and similar matching for detailed topological relations with DTString[J]. Information Sciences, 2014, 276:255-277.
    [16] Alajlan N, Kamel M S, Freeman G H. Geometry-based image retrieval in binary image databases[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(6):1003-1013.
    [17] Li S J. A layered graph representation for complex regions[C]//The 12th International Conference on Principles of Knowledge Representation and Reasoning, Toronto, Canada, Morgan Kuafmann, 2010:581-583.
    [18] Aresta G, Araújo T, Kwok S, et al. Bach:Grand challenge on breast cancer histology images[J]. Medical Image Analysis, 2019:1-19.
    [19] Escalera S, Alicia Fornés, Pujol O, et al. Blurred shape model for binary and grey-level symbol recognition[J]. Pattern Recognition Letters, 2009, 30(15):1424-1433.
    [20] Lyzinski V, Fishkind D E, Fiori M, et al. Graph matching:relax at your own risk[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(1):60-73.
    [21] Abbas S, Farhan S, Fahiem M A, et al. Efficient shape classification using zernike moments and geometrical features on MPEG-7 dataset[J]. Advances in Electrical and Computer Engineering, 2019, 19(1):45-51.
    [22] Wei C H, Li Y, Chau W Y, et al. Trademark image retrieval using synthetic features for describing global shape and interior structure[J]. Pattern Recognition, 2010, 42(3):386-394.
    [23] Ni H, Guo Z H, Huang B Q. Binary patent image retrieval using the hierarchical oriented gradient histogram[C]//2015 International Conference on Service Science (ICSS), Weihai, China:IEEE, 2015:23-27.
    [24] Zhang Z, Tang P, Duan R. Dynamic time warping under pointwise shape context[J]. Information Sciences, 2015, 315:88-101.
    [25] Wang X G, Feng B, Bai X, et al. Bag of contour fragments for robust shape classification[J], Pattern Recognition. 2014, 47(6):2116-2125.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

刘东,王生生.精细拓扑结构表示与深度特征融合在多目标图像检索中的应用[J].重庆大学学报,2021,44(3):132~143

Copy
Share
Article Metrics
  • Abstract:530
  • PDF: 794
  • HTML: 812
  • Cited by: 0
History
  • Received:November 21,2019
  • Online: March 31,2021
Article QR Code