﻿ 基于变分贝叶斯推断的字典学习算法
 控制与决策  2020, Vol. 35 Issue (2): 469-473 0

### 引用本文 [复制中英文]

[复制中文]
LIU Lian, WANG Xiao-tong. Dictionary learning algorithm based on variable Bayes inference[J]. Control and Decision, 2020, 35(2): 469-473. DOI: 10.13195/j.kzyjc.2018.0609.
[复制英文]

### 文章历史

Dictionary learning algorithm based on variable Bayes inference
LIU Lian , WANG Xiao-tong
Abstract: The traditional dictionary learning algorithms have slow convergence rate when learning the training image. And the effect of dictionary learning becomes worse if the images are corrupted by noise. Therefore, a dictionary learning algorithm based on variational inference is proposed to solve this problem. The algorithm firstly sets the conjugate sparse prior distribution of the parameters in the model, and then the joint probability density function of all parameters is calculated based on the Bayesian network. Finally, the optimal edge distribution of the parameters is calculated by the variational Bayesian inference, and the adaptive dictionary training is completed. The image denoising experiment and the compressed sensing image reconstruction experiment are carried out by the adaptive dictionary. The simulation results show that the algorithm can significantly increase the efficiency of dictionary learning, and the visual effect of the denoising and the reconstruction of the test images are improved.
Keywords: Bayesian network    variational inference    dictionary learning    image denoising    compressed sensing
0 引言

1 模型描述

 (1)

 (2)

 (3)

 图 1 贝叶斯网络模型
2 变分贝叶斯推断

 (4)

 (5)

 (6)

3 变分字典学习算法

 (7)

1) 字典矩阵的各列原子dj.

 (8)

2) 为了简化运算, 将系数矩阵各行向量及其协方差参数合并求解, 即求参数组(sj, γε)的概率分布函数, 有

 (9)

3) 隐变量矩阵各元素zij.

zij = 0, 有

 (10)

4) 隐变量元素分布参数βj.

 (11)

4 算法步骤

Step 1:根据式(10)计算出隐变量矩阵的边缘分布函数.

Step 2:将隐变量期望代入式(8)、(9)、(11)中, 更新各参数或参数组的边缘概率分布.

Step 3:利用式(3)重构数据集矩阵, 计算其与原始数据集的均方误差ε.

Step 4:判定均方误差ε与阈值的大小, 若大于阈值则转Step 5, 否则转Step 6.

Step 5:更新循环次数并判定是否达到R, 若达到则转Step 6, 否则转Step 1.

Step 6:输出训练字典.

5 实验结果与分析

5.1 训练字典

5.2 图像去噪

 图 2 各算法去噪效果

5.3 压缩感知图像重构

 图 3 各算法重构效果

6 结论

 [1] Tosic I, Frossard P. Dictionary learning[J]. Signal Processing Magazine of IEEE, 2011, 28(2): 27-38. DOI:10.1109/MSP.2010.939537 [2] Sulam J, Ophir B, Zibulevsky M, et al. Trainlets: Dictionary learning in high dimensions[J]. IEEE Transactions on Signal Processing, 2016, 64(12): 3180-3193. DOI:10.1109/TSP.2016.2540599 [3] Lu J, Wang G, Zhou J. Simultaneous feature and dictionary learning for image set based face recognition[J]. IEEE Transactions on Image Process, 2017, 26(8): 4042-4054. DOI:10.1109/TIP.2017.2713940 [4] Mehrdad Y, Laurent D, Mike E D. Parametric dictionary design for sparse coding[J]. IEEE Transactions on Signal Processing, 2009, 57(12): 4800-4811. DOI:10.1109/TSP.2009.2026610 [5] Ron R, Alfred M B, Elad M. Dictionaries for sparse representation modeling[J]. Proceedings of the IEEE, 2010, 98(6): 1045-1057. [6] Moshe M, Yonina C E. Blind multiband signal reconstruction: Compressed sensing for analog signals[J]. IEEE Transactions on Signal Processing, 2007, 57(3): 993-1009. [7] Aharon M, Elad M, Bruckstein A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322. DOI:10.1109/TSP.2006.881199 [8] Mairal J, Bach F, Ponce J, et al. Online learning for matrix factorization and sparse coding[J]. Journal of Machine Learning Research, 2009, 11(1): 19-60. [9] Yashar N, Soosan B, Mohammad A T. Online learning for matrix factorization and sparse coding[J]. IEEE Transactions on Signal Processing, 2015, 64(3): 592-602. [10] Zhou M, Chen H, Paisley J. Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images[J]. IEEE Transactions on Image Processing, 2011, 21(1): 130-144. [11] Rubinstein R, Peleg T, Elad M. Analysis K-SVD:A dictionary learning algorithm for the analysis sparse model[J]. IEEE Transactions on Signal Processing, 2013, 61(3): 661-677. DOI:10.1109/TSP.2012.2226445 [12] Saiprasad R, Yoram B. Sparsifying transform learning for compressed sensing MRI[C]. IEEE International Symposium on Biomedical Imaging. San Francisco, 2013: 17-20. [13] Jenatton R, Mairal J, Obozinski G, et al. Proximal methods for hierarchical sparse coding[J]. Journal of Machine Learning Research, 2010, 12(7): 2297-2334. [14] Boaz O, Michael L, Elad M. Multi-scale dictionary learning using wavelets[J]. IEEE Journal of Selected Topics in Signal Processing, 2011, 5(5): 1014-1024. DOI:10.1109/JSTSP.2011.2155032 [15] Mohammad A, Hayder R. Compressive dictionary learning for image recovery[C]. IEEE International Conference on Image Processing. Orlando, 2012: 661-664. [16] Jian Z, Chen Z, Debin Z, et al. Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization[J]. Signal Processing, 2014, 103(10): 114-126. [17] Seeger M W, Wipf D P. Variational Bayesian inference Techniques[J]. IEEE Transactions on Information Theory, 2010, 27(6): 81-91. [18] Fox C W, Roberts S J. A toturial on variational Bayesian inference[J]. Artificial Intelligence Review, 2012, 38(2): 85-95. DOI:10.1007/s10462-011-9236-8 [19] Pravin K R, Jalil T, Markus F L. A variational Bayesian inference framework for multiview depth image enhancement[J]. IEEE International Symposium on Multimedia, 2013, 41(11): 183-190.