基于注意力的共享参数胶囊网络
CSTR:
作者:
作者单位:

上海理工大学 控制科学与工程系,上海 200093

作者简介:

通讯作者:

E-mail: sonya@usst.edu.cn.

中图分类号:

TP273

基金项目:

国家自然科学基金项目(62073223);上海市自然科学基金项目(22ZR1443400);航天飞行动力学技术国防科技重点实验室开放课题项目(6142210200304).


Attention-based capsule network with shared parameters
Author:
Affiliation:

Department of Control Science and Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对传统胶囊网络特征信息的传播冗余性和解构低效性问题,提出一种共享参数的注意力胶囊网络.该网络的优点主要体现于以下两方面:1)提出注意力机制的动态路由方法,通过计算低级胶囊的相关性,使得在保留特征空间信息的同时更加关注相关性高的特征信息,并完成前向传播;2)在动态路由层提出共享转换矩阵,基于低级胶囊投票一致性对高级胶囊激活,并通过共享转换矩阵减少模型的参数量,同时实现改进胶囊网络的稳健性.首先,通过5个公开数据集的分类对比实验,表明所提出胶囊网络在Fashion-MNIST、SVHN和CIFAR10数据集上分别取得了5.17%、3.67%和9.35%的最好分类结果,而且在复杂数据集上具有显著的白盒对抗攻击鲁棒性;然后,通过在基于smallNORB和affNISH公开数据集的仿射变换对比实验,表明所提出的胶囊网络具有显著的仿射变换鲁棒性;最后,通过计算效率分析对比实验结果,表明所提出共享参数胶囊网络在不增加浮点运算的情况下,参数量比传统的胶囊网络减少4.9%,具有突出的计算量优势.

    Abstract:

    Aiming to handle the problem of propagation redundancy and deconstruction inefficiency of features in traditional capsule networks, this paper proposes an attention-based capsule network with shared parameters. The merits of such a network lie mainly in the following two issues: 1) A dynamic routing method based on an attention mechanism is proposed. This method calculates the correlation between low-level capsules to maintain the space information of features and pay more attention to the feature information with a high correlation, thus fulfilling the forward propagation; 2) A shared transformation matrix is developed in the dynamic routing layer. The high-level capsules are activated based on the voting consistency of the low-level capsules. Then, the transformation matrix with shared parameters is used to reduce the parameters of the model and obtain the robustness of the capsule network. Experimental results of comparison classification on five public datasets show that the proposed capsule network achieves the best classification results of 5.17%, 3.67% and 9.35% on the Fashion-MNIST, SVHN and CIFAR10 datasets, respectively. Moreover, it has significant robustness against the white-box anti-attack. In addition, the transformation experimental results on smallNORB and affNISH public datasets show that the proposed capsule network has obvious robustness to the transformation. Finally, the experimental results of computational efficiency show that the proposed capsule network with shared parameters reduces the parameters of traditional capsule networks by 4.9% without adding floating-point operations and has an overwhelming advantage in computation.

    参考文献
    相似文献
    引证文献
引用本文

宋燕,覃俞璋,曾入.基于注意力的共享参数胶囊网络[J].控制与决策,2023,38(6):1577-1585

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2023-05-13
  • 出版日期: 2023-06-20
文章二维码