SiamMT: 基于自适应特征融合机制的可修正RGBT目标跟踪算法
作者:
中图分类号:

TP394.41

基金项目:

国家自然科学基金项目(62363029);内蒙古科技计划项目(2021GG0164);内蒙古自然科学基金项目(2022MS06018, 2021MS06018);高校院所协同创新项目(XTCX2023-16).


SiamMT: A modifiable RGBT target tracking algorithm based on adaptive feature fusion mechanism
Author:
  • QI Yong-sheng

    QI Yong-sheng

    College of Electric Power, Inner Mongolia University of Technology, Hohhot 010080, China;Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in the Inner Mongolia Autonomous Region, Hohhot 010080, China;Engineering Research Center of the Ministry of Education for Large Scale Energy Storage Technology, Hohhot 010080, China
    在期刊界中查找
    在百度中查找
    在本站中查找
  • JIANG Zheng-ting

    JIANG Zheng-ting

    College of Electric Power, Inner Mongolia University of Technology, Hohhot 010080, China
    在期刊界中查找
    在百度中查找
    在本站中查找
  • LIU Li-qiang

    LIU Li-qiang

    College of Electric Power, Inner Mongolia University of Technology, Hohhot 010080, China;Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in the Inner Mongolia Autonomous Region, Hohhot 010080, China;Engineering Research Center of the Ministry of Education for Large Scale Energy Storage Technology, Hohhot 010080, China
    在期刊界中查找
    在百度中查找
    在本站中查找
  • SU Jian-qiang

    SU Jian-qiang

    College of Electric Power, Inner Mongolia University of Technology, Hohhot 010080, China;Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in the Inner Mongolia Autonomous Region, Hohhot 010080, China
    在期刊界中查找
    在百度中查找
    在本站中查找
  • ZHANG Li-jie

    ZHANG Li-jie

    College of Electric Power, Inner Mongolia University of Technology, Hohhot 010080, China;Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in the Inner Mongolia Autonomous Region, Hohhot 010080, China;Engineering Research Center of the Ministry of Education for Large Scale Energy Storage Technology, Hohhot 010080, China
    在期刊界中查找
    在百度中查找
    在本站中查找
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [25]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    针对传统RGBT目标跟踪算法网络精确度低、鲁棒性差, 以及在目标尺度变化大和长时跟踪过程中存在目标丢失无法找回等问题, 提出一种新的基于自适应特征融合机制的可修正RGBT目标跟踪算法. 首先, 引入一种特征层与模态间双自适应融合机制, 充分利用两模态间的互补信息, 增强RGB与红外特征的跨模态融合; 然后, 设计一种后端时序约束回归模块, 利用上一帧信息对IOU计算以及边界框回归进行约束, 有效减少相似物干扰; 最后, 提出一种基于元学习的在线模板更新机制, 对回归阶段得分较高的模板图像进行更新存储, 解决长时跟踪中累计误差和目标难以找回问题. 采用权威的目标跟踪数据集GTOT、RGBT234和VOT-RGBT2019进行算法验证, 所提出方法均可取得极具竞争力的结果. 将算法移植到嵌入式设备Jetson Xavier NX上进行性能测试, 实验结果表明: 所提出算法运行速度可达到29帧/s, 相比于当前流行的多种RGBT算法, 具有更为全面的跟踪性能, 且能够有效解决相似物干扰、目标丢失难找回等问题.

    Abstract:

    In order to solve the problems of low network accuracy and poor robustness of traditional RGBT target tracking algorithms, as well as the problems of target loss and unretrieval in the process of large target scale change and long-term tracking, a new modifiable RGBT target tracking algorithm based on adaptive feature fusion mechanism is proposed. Firstly, an adaptive dual-modal fusion module is introduced to make full use of the complementary information between the two modalities to enhance the cross-modal fusion of RGB and infrared features. Then, a back-end timing constraints regression module is designed, using the information of the previous frame to constrain the IOU calculation and bounding box regression, which effectively reduces the interference of similarities. Finally, an online template update mechanism based on meta-learning is proposed to update and store the template images with high scores in the regression stage, so as to solve the problems of cumulative error and difficult target recovery in long-term tracking. Using the authoritative object tracking datasets GTOT, RGBT234 and VOT-RGBT2019 for algorithm verification, the proposed method can achieve very competitive results. The algorithm is transplanted to the embedded device Jetson Xavier NX for performance testing. The testing results show that the proposed algorithm runs at a speed of 29 frames/s, which has more comprehensive tracking performance than the current popular RGBT algorithms, and can effectively solve the problem of similar interference, problems such as the difficulty of retrieving the lost target.

    参考文献
    [1] Nam H, Han B. Learning multi-domain convolutional neural networks for visual tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, 2016: 4293-4302.
    [2] 陈志旺, 孙泽兵, 吕昌昊, 等. 基于并行多外观特征的孪生网络目标跟踪算法研究[J]. 控制与决策, 2024, 39(11): 3628-3636.
    Chen Z W, Sun Z B, Lv C H, et al. Tracking algorithm of Siamese network based on parallel multiple appearance features[J]. Control and Decision, 2024, 39(11): 3628-3636.
    [3] Zhang X C, Ye P, Peng S Y, et al. Corrections to “SiamFT: An RGB-infrared fusion tracking method via fully convolutional Siamese networks”[J]. IEEE Access, 2019, 7: 144799.
    [4] Zhang T L, Liu X R, Zhang Q, et al. SiamCDA: Complementarity-and distractor-aware RGB-T tracking based on Siamese network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(3): 1403-1417.
    [5] Guo C Y, Xiao L. High speed and robust RGB-thermal tracking via dual attentive stream Siamese network[C]. IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumpur, 2022: 803-806.
    [6] 曾子林, 张宏军, 张睿, 等. 基于元学习思想的算法选择问题综述[J]. 控制与决策, 2014, 29(6): 961-968.
    Zeng Z L, Zhang H J, Zhang R, et al. Summary of algorithm selection problem based on meta-learning[J]. Control and Decision, 2014, 29(6): 961-968.
    [7] Park E, Berg A C. Meta-tracker: Fast and robust online adaptation for visual object trackers[C]. European Conference on Computer Vision. Munich, 2018: 587-604.
    [8] Dong X P, Shen J B, Shao L, et al. CLNet: A compact latent network for fast adjusting Siamese trackers[C]. European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 378-395.
    [9] Li C L, Cheng H, Hu S Y, et al. Learning collaborative sparse representation for grayscale-thermal tracking[J]. IEEE Transactions on Image Processing, 2016, 25(12): 5743-5756.
    [10] Li C L, Liang X Y, Lu Y J, et al. RGB-T object tracking: Benchmark and baseline[J]. Pattern Recognition, 2019, 96: 106977.
    [11] Li C L, Xue W L, Jia Y Q, et al. LasHeR: A large-scale high-diversity benchmark for RGBT tracking[J]. IEEE Transactions on Image Processing, 2022, 31: 392-404.
    [12] Zhang Q J, Cong R M, Li C Y, et al. Dense attention fluid network for salient object detection in optical remote sensing images[J]. IEEE Transactions on Image Processing, 2021, 30: 1305-1317.
    [13] Zhang P Y, Zhao J, Bo C J, et al. Jointly modeling motion and appearance cues for robust RGB-T tracking[J]. IEEE Transactions on Image Processing, 2021, 30: 3335-3347.
    [14] Fu J, Liu J, Tian H J, et al. Dual attention network for scene segmentation[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, 2019: 3141-3149.
    [15] Danelljan M, Bhat G, Khan F S, et al. ECO: Efficient convolution operators for tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, 2017: 6931-6939.
    [16] Kim H U, Lee D Y, Sim J Y, et al. SOWP: Spatially ordered and weighted patch descriptor for visual tracking[C]. IEEE International Conference on Computer Vision. Santiago, 2015: 3011-3019.
    [17] Zhang Z P, Peng H W. Deeper and wider Siamese networks for real-time visual tracking[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, 2019: 4586-4595.
    [18] Xiao Y, Yang M M, Li C L, et al. Attribute-based progressive fusion network for RGBT tracking[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(3): 2831-2838.
    [19] Tu Z Z, Lin C, Zhao W, et al. M5L: Multi-modal multi-margin metric learning for RGBT tracking[J]. IEEE Transactions on Image Processing, 2021, 31: 85-98.
    [20] Tomar N K, Jha D, Riegler M A, et al. FANet: A feedback attention network for improved biomedical image segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(11): 9375-9388.
    [21] Cai Y J, Sui X B, Gu G H. Multi-modal multi-task feature fusion for RGBT tracking[J]. Information Fusion, 2023, 97: 101816.
    [22] Li H X, Hu T J, Xiong Z T, et al. ADRNet: A generalized collaborative filtering framework combining clinical and non-clinical data for adverse drug reaction prediction[C]. Proceedings of the 17th ACM Conference on Recommender Systems. Singapore, 2023: 682-687.
    [23] Zhang P Y, Zhao J, Wang D, et al. Visible-thermal UAV tracking: A large-scale benchmark and new baseline[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, 2022: 8876-8885.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

齐咏生,姜政廷,刘利强,等. SiamMT: 基于自适应特征融合机制的可修正RGBT目标跟踪算法[J].控制与决策,2025,40(4):1312-1320

复制
分享
文章指标
  • 点击次数:10
  • 下载次数: 18
  • HTML阅读次数: 13
  • 引用次数: 0
历史
  • 收稿日期:2024-02-29
  • 在线发布日期: 2025-03-21
  • 出版日期: 2025-04-20
文章二维码