应用注意力机制的文档图像篡改与脱敏定位

郑铿涛; 李斌; 曾锦华

doi:10.19665/j.issn1001-2400.20230105

您当前的位置：

首页 >

文章列表页 >

应用注意力机制的文档图像篡改与脱敏定位

网络空间安全 | 更新时间：2024-01-22

- 应用注意力机制的文档图像篡改与脱敏定位
- 暂无标题
- 西安电子科技大学学报 2023年50卷第6期页码：207-218
- 作者机构：
  
  1. 深圳大学广东省智能信息处理重点实验室,深圳市媒体信息内容安全重点实验室,广东深圳 518060
  2. 深圳市人工智能与机器人研究院,广东深圳 518129
  3. 司法鉴定科学研究院,上海 200063
- 作者简介：
  
  [ "郑铿涛(1997—),男,深圳大学硕士研究生,E-mail:[email protected];" ]
  李斌(1982—),男,教授,E-mail:[email protected]
  [ "曾锦华(1985—),男,高级工程师,E-mail:[email protected]。" ]
- 基金信息：
  
  国家自然科学基金(U22B2047);国家自然科学基金(61872244);国家自然科学基金(62272314);广东省基础与应用基础研究基金杰出青年基金项目(2019B151502001);深圳市研究与发展计划项目(JCYJ2020010905008228);上海市科委技术标准项目(21DZ2200100)
- DOI：10.19665/j.issn1001-2400.20230105
  中图分类号：
扫描看全文
郑铿涛, 李斌, 曾锦华. 应用注意力机制的文档图像篡改与脱敏定位[J]. 西安电子科技大学学报, 2023,50(6):207-218.
郑铿涛, 李斌, 曾锦华. 应用注意力机制的文档图像篡改与脱敏定位[J]. 西安电子科技大学学报, 2023,50(6):207-218. DOI： 10.19665/j.issn1001-2400.20230105.

DOI：

摘要

诸如合同、证明文件和通知书等一些重要的文档材料,常常以电子图像格式被存储和传播。然而,由于包含关键的文字信息,此类图像往往容易被非法篡改利用,造成严重的社会影响和危害;与此同时,考虑到个人的隐私安全问题,人们往往也会对这类图像做脱除敏感信息处理。恶意篡改与脱敏均会给原始图像引入额外痕迹,但在动机上存在区别,且在操作方式上也存在一定差异。因此,有必要对二者进行区分,从而更准确地定位出篡改区域。针对这个问题,提出了一个卷积编解码网络,通过U形连接获取编码器多级特征,有效学习篡改和脱敏处理痕迹;同时,在解码网络引入多个挤压激励注意力机制模块,抑制图像内容,关注更微弱的处理痕迹,提高网络的检测能力。为了有效地辅助网络训练,构建了一个包含常见篡改操作和脱敏操作的文档图像取证数据集。实验结果表明,算法模型在此数据集上表现良好,在公开的篡改数据集上也有不错的性能,并优于对比算法。同时,所提的算法对几种常见的后处理操作具有较好的鲁棒性。

Abstract

Some important documents such as contracts,certificates and notifications are often stored and disseminated in a digital format.However,due to the inclusion of key text information,such images are often easily illegally tampered with and used,causing serious social impact and harm.Meanwhile,taking personal privacy and security into account,people also tend to remove sensitive information from these digital documents.Malicious tampering and desensitization can both introduce extra traces to the original images,but there are differences in motivation and operations.Therefore,it is necessary to differentiate them to locate the tamper areas more accurately.To address this issue,we propose a convolutional encoder-decoder network,which has multi-level features of the encoder through U-Net connection,effectively learning tampering and desensitization traces.At the same time,several Squeeze-and-Excitation attention mechanism modules are introduced in the decoder to suppress image content and focus on weaker operation traces,to improve the detection ability of the network.To effectively assist network training,we build a document image forensics dataset containing common tampering and desensitization operations.Experimental results show that our model performs effectively both on this dataset and on the public tamper datasets,and outperforms comparison algorithms.At the same time,the proposed method is robust to several common post-processing operations.

关键词

文档图像篡改定位脱敏定位U-Net挤压激励注意力机制

Keywords

document imageforgery localizationdesensitization localizationU-Netsqueeze-and-excitation attention mechanism

references

孔月萍, 刘楚, 朱旭东. 一种利用背景光流特征的虚假人脸检测方法[J]. 西安电子科技大学学报, 2021, 48(5):86-91.

KONG Yueping, LIU Chu, ZHU Xudong. Faceanti-Spoofing Method Using the Optical Flow Features of Back Ground[J]. Journal of Xidian University, 2021, 48(5):86-91.

KIM M, TARIQ S, WOO S S. Fretal:Generalizing Deepfake Detection Using Knowledge Distillation and Representation Learning[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2021:1001-1012.

李昊东, 庄培裕, 李斌. 基于深度学习的数字图像篡改定位方法综述[J]. 信号处理, 2021, 37(12):2278-2301.

LI Haodong, ZHUANG Peiyu, LI Bin. A Survey on Deep Learning Based Digital Image Tampering Localization Methods[J]. Journal of Signal Processing, 2021, 37(12):2278-2301.

朱叶, 余宜林, 郭迎春. HRDA-Net:面向真实场景的图像多篡改检测与定位算法[J]. 通信学报, 2022, 43(1):217-226. DOI:10.11959/j.issn.1000-436x.2022016http://doi.org/10.11959/j.issn.1000-436x.2022016

ZHU Ye, YU Yilin, GUO Yingchun. HRDA-Net:Image Multiple Manipulation Detection and Location Algorithm in Real Scene[J]. Journal on Communications, 2022, 43(1):217-226. DOI:10.11959/j.issn.1000-436x.2022016http://doi.org/10.11959/j.issn.1000-436x.2022016

MAHDIAN B, SAIC S. Using Noise Inconsistencies for Blind Image Forensics[J]. Image and Vision Computing, 2009, 27(10):1497-1503. DOI:10.1016/j.imavis.2009.02.001http://doi.org/10.1016/j.imavis.2009.02.001https://linkinghub.elsevier.com/retrieve/pii/S0262885609000146https://linkinghub.elsevier.com/retrieve/pii/S0262885609000146

张旭, 胡晰远, 陈晨, 等. 基于透视投影下空间光照一致性分析的图像拼接篡改检测[J]. 自动化学报, 2019, 45(10):1857-1869.

ZHANG Xu, HU Xiyuan, CHEN Chen, et al. Image Splicing Detection Based on Spatial Lighting Consistency Analysis Under Perspective Projection[J]. Acta Automatica Sinica, 2019, 45(10):1857-1869.

FERRARA P, BIANCHI T, DE ROSA A, et al. Image Forgery Localization via Fine-Grained Analysis of CFA Artifacts[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(5):1566-1577. DOI:10.1109/TIFS.2012.2202227http://doi.org/10.1109/TIFS.2012.2202227http://ieeexplore.ieee.org/document/6210378/http://ieeexplore.ieee.org/document/6210378/

LIN Z, HE J, TANG X, et al. Fast,Automatic and Fine-Grained Tampered JPEG Image Detection via DCT Coefficient Analysis[J]. Pattern Recognition, 2009, 42(11):2492-2501. DOI:10.1016/j.patcog.2009.03.019http://doi.org/10.1016/j.patcog.2009.03.019https://linkinghub.elsevier.com/retrieve/pii/S0031320309001198https://linkinghub.elsevier.com/retrieve/pii/S0031320309001198

BAYAR B, STAMM M C. Constrained Convolutional Neural Networks:A New Approach towards General Purpose Image Manipulation Detection[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(11):2691-2706. DOI:10.1109/TIFS.2018.2825953http://doi.org/10.1109/TIFS.2018.2825953https://ieeexplore.ieee.org/document/8335799/https://ieeexplore.ieee.org/document/8335799/

WU Y, ABDALMAGEED W, NATARAJAN P. Mantra-Net:Manipulation Tracing Network for Detection and Localization of Image Forgeries with Anomalous Features[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:9543-9552.

LONG J, SHELHAMER E, DARREL T. Fully Convolutional Networks for Semantic Segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2015:3431-3440.

吴韵清, 吴鹏, 陈北京, 等. 基于残差全卷积网络的图像拼接定位算法[J]. 应用科学学报, 2019, 37(5):651-662.

WU Yunqing, WU Peng, CHEN Beijing, et al. Image Splicing Localization Method Based on Fully Convolutional Residual Networks[J]. Journal of Applied Sciences, 2019, 37(5):651-662.

KWON M J, YU I J, NAM S H, et al. CAT-Net:Compression Artifact Tracing Network for Detection and Localization of Image Splicing[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.Piscataway:IEEE, 2021:375-384.

ZHUANG P, LI H, TAN S, et al. Image Tampering Localization Using a Dense Fully Convolutional Network[J]. IEEE Transactions on Information Forensics and Security, 2021, 16:2986-2999. DOI:10.1109/TIFS.2021.3070444http://doi.org/10.1109/TIFS.2021.3070444https://ieeexplore.ieee.org/document/9393396/https://ieeexplore.ieee.org/document/9393396/

RONNEBERGER O, FISCHER P, BROX T. U-Net:Convolutional Networks for Biomedical ImageSegmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer, 2015:234-241.

BI X, WEI Y, XIAO B, et al. RRU-Net:The Ringed Residual U-Net for Image Splicing Forgery Detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Piscataway:IEEE, 2019:30-39.

ZHANG R, NI J. A Dense U-Net with Cross-Layer Intersection for Detection and Localization of Image Forgery[C]// ICASSP 2020-2020 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway:IEEE, 2020:2982-2986.

WU H, ZHOU J, TIAN J, et al. Robust Image Forgery Detection against Transmission over Online Social Networks[J]. IEEE Transactions on Information Forensics and Security, 2022, 17:443-456. DOI:10.1109/TIFS.2022.3144878http://doi.org/10.1109/TIFS.2022.3144878https://ieeexplore.ieee.org/document/9686650/https://ieeexplore.ieee.org/document/9686650/

CRUZ F, SIDERE N, COUSTATY M, et al. Local Binary Patterns for Document Forgery Detection[C]// 2017 14th IAPR International Conference on Document Analysis and Recognition(ICDAR).Piscataway:IEEE, 2017, 1:1223-1228.

SHIVAKUMARA P, BASAVARAJA V, GOWDA H S, et al. A New RGB Based Fusion for Forged IMEI Number Detection in Mobile Images[C]// 2018 16th International Conference on Frontiers in Handwriting Recognition(ICFHR).Piscataway:IEEE, 2018:386-391.

VAN BEUSEKOM J, SHAFAIT F, BREUEL T M. Document Inspection Using Text-Line Alignment[C]// Proceedings of the 9th IAPR International Workshop on Document Analysis Systems. New York: ACM, 2010:263-270.

VAN BEUSEKOM J, SHAFAIT F, BREUEL T M. Text-Line Examination for Document Forgery Detection[J]. International Journal on Document Analysis and Recognition(IJDAR), 2013, 16(2):189-207.

BERTRAND R, GOMEZ-KRÄMER P, TERRADES O R, et al. A System Based on Intrinsic Features for Fraudulent Document Detection[C]// 2013 12th International Conference on Document Analysis and Recognition.Piscataway:IEEE, 2013:106-110.

BERTRAND R, TERRADES O R, GOMEZ-KRÄMER P, et al. A Conditional Random Field Model for Font Forgery Detection[C]// 2015 13th International Conference on Document Analysis and Recognition(ICDAR).Piscataway:IEEE, 2015:576-580.

SHANG S, KONG X, YOU X. Document Forgery Detection Using Distortion Mutation of Geometric Parameters in Characters[J]. Journal of Electronic Imaging, 2015, 24(2):023008. DOI:10.1117/1.JEI.24.2.023008http://doi.org/10.1117/1.JEI.24.2.023008http://electronicimaging.spiedigitallibrary.org/article.aspx?doi=10.1117/1.JEI.24.2.023008http://electronicimaging.spiedigitallibrary.org/article.aspx?doi=10.1117/1.JEI.24.2.023008

NANDANWAR L, SHIVAKUMARA P, PAL U, et al. A New Method for Detecting Altered Text in Document Images[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2021, 35(12):2160010. DOI:10.1142/S0218001421600107http://doi.org/10.1142/S0218001421600107https://www.worldscientific.com/doi/abs/10.1142/S0218001421600107https://www.worldscientific.com/doi/abs/10.1142/S0218001421600107

JAMES H, GUPTA O, RAVIV D. Learning Document Graphs with Attention for Image Manipulation Detection[C]// International Conference on Pattern Recognition and Artificial Intelligence.Heidelberg:Springer, 2022:263-274.

XU W, LUO J, ZHU C, et al. Document Images Forgery Localization Using a Two-Stream Network[J]. International Journal of Intelligent Systems, 2022, 37(8):5272-5289. DOI:10.1002/int.v37.8http://doi.org/10.1002/int.v37.8https://onlinelibrary.wiley.com/toc/1098111x/37/8https://onlinelibrary.wiley.com/toc/1098111x/37/8

LIU Z, MAO H, WU C Y, et al. A Convnet for the 2020s[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2022:11976-11986.

HU J, SHEN L, SUN G. Squeeze-and-Excitation Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:7132-7141.

LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for Dense Object Detection[C]// Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE, 2017:2980-2988.

BERMAN M, TRIKI A R, BLASCHKO M B. The Lovász-Softmax Loss:A Tractable Surrogate for the Optimization of the Intersection-over-Union Measure in Neural Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:4413-4421.

COZZOLINO D, VERDOLIVA L. Noiseprint:A CNN-Based Camera Model Fingerprint[J]. IEEE Transactions on Information Forensics and Security, 2019, 15:144-159. DOI:10.1109/TIFS.10206http://doi.org/10.1109/TIFS.10206https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=10206https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=10206

CHEN X, DONG C, JI J, et al. Image Manipulation Detection by Multi-View Multi-Scale Supervision[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision.Piscataway:IEEE, 2021:14185-14193.

ALIBABA SECURITY, CHINA SOCIETY OF IMAGE AND GRAPHICS(CSIG). Real-World Image Forgery Localization Challenge(2022) [DB/OL].[2022-10-20]. https://tianchi.aliyun.com/competition/entrance/531945/introductionhttps://tianchi.aliyun.com/competition/entrance/531945/introductionhttps://tianchi.aliyun.com/competition/entrance/531945/introduction.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

注意力驱动的细胞团簇细胞核分割算法