特征引导的多模态聚合低光环境行为识别方法
CSTR:
作者:
作者单位:

1. 西安建筑科技大学 信息与控制工程学院,西安 710055;2. 西安市建筑制造智动化技术重点实验室,西安 710055

作者简介:

通讯作者:

E-mail: mengyuebo@163.com.

中图分类号:

TP391

基金项目:

国家自然科学基金面上项目(52278125).


Night behavior recognition based on multi-mode feature fusion
Author:
Affiliation:

1. College of Information and Control Engineering,Xián University of Architecture and Technology,Xián 710055,China;2. Xián Key Laboratory of Intelligent Technology for Building and Manufacturing,Xián 710055,China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    诸如夜间等低光场景下的行为识别对于安防、自动驾驶等领域具有重要意义,针对现有方法在低光环境下识别效果不佳、鲁棒性较差等问题,提出一种基于特征引导的多模态聚合低光环境行为识别方法(MALNFG). 首先,设计分层骨架特征融合网络(HSFIE),利用光照增强算法提升低光场景的骨架提取能力,采用层次化时空特征融合策略获取侧重于人体行为本身表达的动作特征,改善低光场景下因骨架缺失造成的精度下降问题;其次,设计高效表观特征提取模块(EAFEM),采用零参数时间位移模块在2D特征提取网络上高效捕捉包含丰富场景信息的时空特征;接着,设计特征引导多模态聚合网络(MNF),利用特征引导策略执行骨架特征与RGB表观特征的深层信息交互,实现行为特征的全面性表征;最后,采用全连接层进行特征分类,完成行为识别.实验结果表明,所提出方法可以较好地适用于低光环境下的人体行为识别任务.

    Abstract:

    Action recognition in low light scenes such as night is of great significance to the fields of security, automatic driving and so on. Aiming at the problems of poor recognition effect and poor robustness of existing methods in low-light environment, a multimodal aggregate low light environment action recognition method based on feature guidance is proposed. Firstly, the hierarchical skeleton fusion network for illumination enhancement is designed. The illumination enhancement algorithm and hierarchical spatiotemporal feature fusion strategy are used to obtain the action features that focus on the expression of human behavior itself, and improve the accuracy degradation caused by skeleton missing in low-light scenes. Secondly, the apparent feature extraction network based on a fusion spatiotemporal conversion module is designed. Spatiotemporal features containing rich scene information on a 2D feature extraction network are efficiently captured using a zero-parameter temporal displacement module. Then, a multi-modal aggregation network based on feature guidance is designed. The feature guidance strategy is used to perform the deep information interaction between skeleton features and RGB features, so as to realize the comprehensive characterization of behavior features. Finally, the full connection layer is used for feature classification to complete behavior recognition. The experimental results show that the proposed method can be well applied to human action recognition tasks in low light environment.

    参考文献
    相似文献
    引证文献
引用本文

刘光辉,王秦蒙,孟月波,等.特征引导的多模态聚合低光环境行为识别方法[J].控制与决策,2024,39(7):2305-2314

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2024-06-06
  • 出版日期: 2024-07-20
文章二维码