结合自注意力与卷积的真实场景图像篡改定位

钟浩; 边山; 王春桃

doi:10.19665/j.issn1001-2400.20230213

您当前的位置：

首页 >

文章列表页 >

结合自注意力与卷积的真实场景图像篡改定位

计算机科学与技术 | 更新时间：2024-04-03

- 结合自注意力与卷积的真实场景图像篡改定位
- Real world image tampering localization combining the self-attention mechanism and convolutional neural networks
- 西安电子科技大学学报 2024年51卷第1期页码：135-146
- 作者机构：
  
  1. 华南农业大学数学与信息学院,广东广州 510642
  2. 广州市智慧农业重点实验室,广东广州 510642
  3. 广东省信息安全技术重点实验室,广东广州 510006
- 作者简介：
  
  [ "钟浩(1995—),男,华南农业大学硕士研究生,E-mail:[email protected]；" ]
  边山(1986—),女,副教授,博士,E-mail:[email protected]
  [ "王春桃(1979—),男,教授,博士,E-mail:[email protected]" ]
- 基金信息：
  
  国家自然科学基金(61702199);国家自然科学基金(62172165);国家自然科学基金(61872152);广东省基础与应用基础研究重大项目(2019B030302008);广东省自然科学基金(2022A1515010325);广州市科技计划项目(202102020582);广州市科技计划项目(201902010081)
- DOI：10.19665/j.issn1001-2400.20230213
  中图分类号： TP391.41
- 纸质出版日期：2024-1-20，
  
  网络出版日期：2023-9-6，
  
  收稿日期：2022-12-7，
扫描看全文
钟浩, 边山, 王春桃. 结合自注意力与卷积的真实场景图像篡改定位[J]. 西安电子科技大学学报, 2024,51(1):135-146.

Hao ZHONG, Shan BIAN, Chuntao WANG. Real world image tampering localization combining the self-attention mechanism and convolutional neural networks[J]. Journal of Xidian University, 2024,51(1):135-146.
钟浩, 边山, 王春桃. 结合自注意力与卷积的真实场景图像篡改定位[J]. 西安电子科技大学学报, 2024,51(1):135-146. DOI： 10.19665/j.issn1001-2400.20230213.

Hao ZHONG, Shan BIAN, Chuntao WANG. Real world image tampering localization combining the self-attention mechanism and convolutional neural networks[J]. Journal of Xidian University, 2024,51(1):135-146. DOI： 10.19665/j.issn1001-2400.20230213.

摘要

图像是移动互联网时代传播信息的重要载体

恶意图像篡改是潜在的网络安全威胁之一。与自然场景中在物体尺度上的图像篡改不同

真实场景中的图像篡改存在于伪造的资质证书、文案、屏幕截图等

这些篡改图像通常会经过精心的手工篡改干预

因此其篡改特征与自然场景篡改特征存在差异

更具有多样性

对其篡改区域的定位更具有挑战性。针对该场景复杂且多样的篡改特征

丰富的关系信息是重要的

文中通过卷积神经网络进行自适应特征提取

并利用逆向连接的全自注意力模块进行多阶段特征关注

最后融合多阶段注意力关注结果进行篡改区域定位。所提方法在真实场景图像篡改定位任务中取得了优于对比方法的性能

其中

指标比主流方法MVSS-Net高出约8.98%

AUC指标高出约3.58%。此外

所提方法在自然场景图像篡改定位任务中也达到了主流方法的性能

并提供了自然场景篡改特征与真实场景篡改特征存在差异的佐证。在两种场景中的实验结果表明

所提方法能够有效地定位出篡改图像的篡改区域

且在复杂的真实场景中的定位效果更显著。

Abstract

Image is an important carrier of information dissemination in the era of the mobile Internet

making malicious image tampering one of the potential cybersecurity threats.Different from the image tampering on the object scale in the natural scene

image tampering in the real world exists in forged qualification certificates

forged documentation

forged screenshots

etc.The tampered images in the real world usually involve elaborate manual tampering interventions

so their tampering features are different from those in the natural scene and are more diverse

making the localization of tampered areas in the real world more challenging.Rich dependency information is important in considering the complex and diverse tampering features in the real world.Therefore

in this paper

the convolutional neural network is used for adaptive feature extraction and the reversely connected fully self-attention module is adopted for multi-stage feature attention.Finally

the tamper area is located by merging the multi-stage attentional results.The proposed method outperforms the comparison methods in the real world image tampering localization task with the F1 metric 8.98% higher than that of the mainstream method MVSS-Net and the AUC metric 3.58% higher.Besides

the proposed method also achieves the performance of mainstream methods in the natural scene image tampering localization task

and the evidence that the natural scene tampering features are inconsistent with the real world tampering features is provided.Experimental results in two scenes show that the proposed method can effectively locate the tampered area of the tampered images

and that it is more effective in complicated real world.

关键词

图像篡改定位伪造检测数字图像取证计算机视觉自注意力机制卷积神经网络

Keywords

image tampering localizationfake detectiondigital image forensicscomputer visionself-attention mechanismconvolutional neural networks

references

DONG J, WANG W, TAN T. CASIA Image Tampering Detection Evaluation Database[C]//Proceedings of the 2013 IEEE China Summit and International Conference on Signal and Information Processing. Piscataway:IEEE, 2013:422-426.

GUAN H, KOZAK M, ROBERTSON E, et al. MFC Datasets:Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation[C]//Proceedings of the 2019 IEEE Winter Applications of Computer Vision Workshops. Piscataway:IEEE, 2019:63-72.

HSU Y, CHANG S. Detecting Image Splicing Using Geometry Invariants and Camera Characteristics Consistency[C]//Proceedings of the 2006 IEEE International Conference on Multimedia and Expo. Piscataway:IEEE, 2006:549-552.

WEN B, ZHU Y, SUBRAMANIAN R, et al. COVERAGE-A Novel Data base for Copy-Move Forgery Detection[C]//Proceedings of the 2016 IEEE International Conference on Image Processing. Piscataway:IEEE, 2016:161-165.

FRIDRICH J, SOUKAL D, LUKÁŠ J. Detection of Copy-Move Forgery in Digital Images[C]//Proceedings of the Digital Forensic Research Workshop. Cleveland: Digital Forensic Research, 2003:1-10.

POPESCU A C, FARID H. Exposing Digital Forgeries by Detecting Duplicated Image Regions[R]. USA: Dartmouth College, 2004.

ZIMBA M, SUN X M. DWT-PCA(EVD) Based Copy-Move Image Forgery Detection[J]. International Journal of Digital Content Technology and its Applications, 2011, 5(1):251-258.

HUANG H, GUO W, ZHANG Y. Detection of Copy-Move Forgery in Digital Images Using SIFT Algorithm[C]//Proceedings of the 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application. Piscataway: IEEE, 2008:272-276.

PAN X, LYU S. Region Duplication Detection Using Image Feature Matching[J]. IEEE Transactions on Information Forensics and Security, 2010, 5(4):857-867.

FERRARA P, BIANCHI T, DE ROSA A, et al. Image Forgery Localization via Fine-Grained Analysis of CFA Artifacts[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(5):1566-1577.

FARID H. Exposing Digital Forgeries from JPEG Ghosts[J]. IEEE Transactions on Information Forensics and Security, 2009, 4(1):154-160.

LI H, LUO W, HUANG J. Localization of Diffusion-Based Inpainting in Digital Images[J]. IEEE Transactions on Information Forensics and Security, 2017, 12(12):3050-3064.

RAO Y, NI J. A Deep Learning Approach to Detection of Splicing and Copy-Move Forgeries in Images [C]//Proceedings of the 2016 IEEE International Workshop on Information Forensics and Security. Piscataway:IEEE, 2016:1-6.

LI H, HUANG J. Localization of Deep Inpainting Using High-Pass Fully Convolutional Network[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE, 2019:8300-8309.

BAYAR B, STAMM M C. Constrained Convolutional Neural Networks:A New Approach Towards General Purpose Image Manipulation Detection[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(11):2691-2706.

SALLOUM R, REN Y, KUO C C J. Image Splicing Localization Using A Multi-Task Fully Convolutional Network(MFCN)[J]. Journal of Visual Communication and Image Representation, 2018, 51:201-209.

ZHOU P, HAN X, MORARIU V I, et al. Learning Rich Features for Image Manipulation Detection[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:1053-1061.

XU W, LUO J, ZHU C, et al. Document Images Forgery Localization Using a Two-Stream Network[J]. International Journal of Intelligent Systems, 2022, 37:5272-5289.

WU Y, ABDALMAGEED W, NATARAJAN P. ManTra-Net:Manipulation Tracing Network for Detection and Localization of Image Forgeries with Anomalous Features[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:9535-9544.

DONG C, CHEN X, HU R, et al. MVSS-Net:Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(3):3539-3553.

HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.

BI X, WEI Y, XIAO B, et al. RRU-Net:The Ringed Residual U-Net for Image Splicing Forgery Detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE, 2019:30-39.

HAO J, ZHANG Z, YANG S, et al. TransForensics:Image Forgery Localization with Dense Self-Attention[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE, 2021:15035-15044.

ZHUANG P, LI H, TAN S, et al. Image Tampering Localization Using A Dense Fully Convolutional Network[J]. IEEE Transactions on Information Forensics and Security, 2021, 16:2986-2999.

HU X, ZHANG Z, JIANG Z, et al. SPAN:Spatial Pyramid Attention Network for Image Manipulation Localization[C]//Proceedings of the European Conference on Computer Vision. Piscataway:IEEE, 2020:312-328.

WU H, ZHOU J, TIAN J, et al. Robust Image Forgery Detection against Transmission over Online Social Networks[J]. IEEE Transactions on Information Forensics and Security, 2022, 17:443-456.

LIU Z, MAO H, WU C Y, et al. A Convnet for the 2020s[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2022:11976-11986.

KWON M J, YU I J, NAM S H, et al. CAT-Net:Compression Artifact Tracing Network for Detection and Localization of Image Splicing[C]//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway:IEEE, 2021:375-384.

BOROUMAND M, CHEN M, FRIDRICH J. Deep Residual Network for Steganalysis of Digital Images[J]. IEEE Transactions on Information Forensics and Security, 2019, 14(5):1181-1193.

DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image is Worth 16x16 Words:Transformers for Image Recognition at Scale(2020)[C/OL].[2020-10-22].https://arxiv.org/abs/2010.11929. https://arxiv.org/abs/2010.11929https://arxiv.org/abs/2010.11929

WANG W, XIE E, LI X, et al. Pyramid Vision Transformer:A Versatile Backbone for Dense Prediction without Convolutions[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE, 2021:548-558.

HOWARD A G, ZHU M, CHEN B, et al. MobileNets:Efficient Convolutional Neural Networks for Mobile Vision Applications(2017)[J/OL].[2017-04-17].https://arxiv.org/abs/1704.04861. https://arxiv.org/abs/1704.04861https://arxiv.org/abs/1704.04861

HU J, SHEN L, ALBANIE S, et al. Squeeze-and-Excitation Networks[J]. IEEE Transactions on Information Forensics and Security, 2020, 42(8):2011-2023.

刘迪, 郭继昌, 汪昱东, 等. 融合注意力机制的多尺度显著性目标检测网络[J]. 西安电子科技大学学报, 2022, 49(4):118-126.

LIU Di, GUO Jichang, WANG Yudong, et al. Multi-Scale Salient Object Detection Network Combining an Attention Mechanism[J]. Journal of Xidian University, 2022, 49(4):118-126.

XIE E, WANG W, YU Z, et al. SegFormer:Simple and Efficient Design for Semantic Segmentation with Transformers[J]. Advances in Neural Information Processing Systems, 2021, 34:12077-12090.

MILLETARI F, NAVAB N, AHMADI S A. V-Net:Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation[C]//Proceedings of the 2016 Fourth International Conference on 3D Vision. Piscataway:IEEE, 2016:565-571.

ALIBABA SECURITY, CHINA SOCIETY OF IMAGE AND GRAPHICS(CSIG).Real-World Image Forgery Localization Challenge(2022)[DB/OL].[2022-11-01].https://tianchi.aliyun.com/competition/entrance/531945/introduction. https://tianchi.aliyun.com/competition/entrance/531945/introductionhttps://tianchi.aliyun.com/competition/entrance/531945/introduction

LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature Pyramid Networks for Object Detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:936-944.

ZOPH B, GHIASI G, LIN T Y, et al. Rethinking Pre-Training and Self-Training[J]. Advances in Neural Information Processing Systems, 2020, 33:3833-3845.

ZHANG J, SANG J, YI Q, et al. ImageNet Pre-Training also Transfers Non-Robustness[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2023:3436-3444.

更多指标>

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合自注意力机制与CNN-BiGRU的事件检测

使用图负采样的图卷积神经网络推荐算法

融合全局和局部信息的实时烟雾分割算法

面向雷达行为识别的多尺度卷积注意力网络

基于生成对抗网络的雷达脉内信号去噪与识别