融合全局和局部信息的实时烟雾分割算法

张欣雨; 梁煜; 张为

doi:10.19665/j.issn1001-2400.20230405

您当前的位置：

首页 >

文章列表页 >

融合全局和局部信息的实时烟雾分割算法

计算机科学与技术 | 更新时间：2024-04-03

- 融合全局和局部信息的实时烟雾分割算法
- Real-time smoke segmentation algorithm combining global and local information
- 西安电子科技大学学报 2024年51卷第1期页码：147-156
- 作者机构：
  
  天津大学微电子学院,天津 300072
- 作者简介：
  
  [ "张欣雨(1999—),女,天津大学硕士研究生,E-mail:[email protected]；" ]
  [ "梁煜(1975—),男,副教授,E-mail:[email protected]" ]
  张为(1975—),男,教授,E-mail:[email protected]
- 基金信息：
  
  天津市新一代人工智能科技重大专项(19ZXZNGX00030)
- DOI：10.19665/j.issn1001-2400.20230405
  中图分类号： TP391.41
- 收稿日期：2023-01-13，
  
  网络出版日期：2023-09-06，
  
  纸质出版日期：2024-01-20
- 稿件说明：
移动端阅览
张欣雨, 梁煜, 张为. 融合全局和局部信息的实时烟雾分割算法[J]. 西安电子科技大学学报, 2024,51(1):147-156.

Xinyu ZHANG, Yu LIANG, Wei ZHANG. Real-time smoke segmentation algorithm combining global and local information[J]. Journal of xidian university, 2024, 51(1): 147-156.
张欣雨, 梁煜, 张为. 融合全局和局部信息的实时烟雾分割算法[J]. 西安电子科技大学学报, 2024,51(1):147-156. DOI： 10.19665/j.issn1001-2400.20230405.

Xinyu ZHANG, Yu LIANG, Wei ZHANG. Real-time smoke segmentation algorithm combining global and local information[J]. Journal of xidian university, 2024, 51(1): 147-156. DOI： 10.19665/j.issn1001-2400.20230405.

摘要

针对烟雾形状不规则、呈半透明状且边界模糊导致烟雾分割困难的问题

提出一种融合全局和局部信息的双分支实时烟雾分割算法。该算法设计了轻量级的Transformer分支和卷积神经网络分支分别提取烟雾的全局特征和局部特征

Transformer分支和卷积神经网络分支共同作用

可以在充分学习烟雾的长距离像素依赖关系的同时保留烟雾细节信息

从而准确区分烟雾和背景像素

改善烟雾分割效果。同时该结构可以满足实际烟雾检测任务的实时性要求;基于多层感知机的解码器充分利用不同尺度的烟雾特征图

并进一步建模烟雾全局上下文信息

增强模型对多尺度烟雾的感知能力

从而提升烟雾分割精度;而且解码器结构简单

可以降低解码器部分的计算量。该算法在自建烟雾分割数据集上的平均交并比为92.88%

模型参数量为2.96 M

推理速度为56.94帧/s。该算法在公开数据集上的综合性能优于其他烟雾检测算法。实验结果表明

该算法分割烟雾的准确率高

推理速度快

可以满足实际烟雾检测任务的准确性和实时性需求。

Abstract

The smoke segmentation is challenging because the smoke is irregular and translucent and the boundary is fuzzy.A dual-branch real-time smoke segmentation algorithm based on global and local information is proposed to solve this problem.In this algorithm

a lightweight Transformer branch and a convolutional neural networks branch are designed to extract the global and local features of smoke respectively

which can fully learn the long-distance pixel dependence of smoke and retain the details of smoke.It can distinguish smoke and background accurately and improve the accuracy of smoke segmentation.It can satisfy the real-time requirement of the actual smoke detection tasks.The multilayer perceptron decoder makes full use of multi-scale smoke features and further models the global context information of smoke.It can enhance the perception of multi-scale smoke

and thus improve the accuracy of smoke segmentation.The simple structure can reduce the computation of the decoder.The algorithm reaches 92.88% mean intersection over union on the self-built smoke segmentation dataset with 2.96M parameters and a speed of 56.94 frames per second.The comprehensive performance of the proposed algorithm is better than that of other smoke detection algorithms on public dataset.Experimental results show that the algorithm has a high accuracy and fast inference speed.The algorithm can meet the accuracy and real-time requirements of actual smoke detection tasks.

关键词

Keywords

references

秦瑞 , 张为 . 一种无锚框结构的多尺度火灾检测算法 [J]. 西安电子科技大学学报 , 2022 , 49 ( 6 ): 111 - 119 .

QIN Rui , ZHANG Wei . Multi-Scale Fire Detection Algorithm with an Anchor Free Structure [J]. Journal of Xidian University , 2022 , 49 ( 6 ): 111 - 119 .

宁阳 , 杜建超 , 韩硕 , 等 . 改进DeeplabV3+的火焰分割与火情分析方法 [J]. 西安电子科技大学学报 , 2021 , 48 ( 5 ): 38 - 46 .

NING Yang , DU Jianchao , HAN Shuo , et al. Fire Segmentation Based on the Improved DeeplabV3+ and the Analytical Method for Fire Development [J]. Journal of Xidian University , 2021 , 48 ( 5 ): 38 - 46 .

DENG X , YU Z , WANG L , et al. Smoke Image Segmentation Based on Color Model [J]. Journal on Innovation and Sustainability RISUS , 2015 , 6 ( 2 ): 130 - 138 .

MA Z , CAO Y , SONG L , et al. A New Smoke Segmentation Method Based on Improved Adaptive Density Peak Clustering [J]. Applied Sciences , 2023 , 13 ( 3 ): 1281 .

WANG Z , YANG P , LIANG H , et al. Semantic Segmentation and Analysis on Sensitive Parameters of Forest Fire Smoke Using Smoke-Unet and Landsat-8 Imagery [J]. Remote Sensing , 2022 , 14 ( 1 ): 45 .

RONNEBERGER O , FISCHER P , BROX T . U-Net:Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015 . Heidelberg : Springer , 2015 : 234 - 241 .

YUAN F , SHI Y , ZHANG L , et al. A Cross-Scale Mixed Attention Network for Smoke Segmentation [J]. Digital Signal Processing , 2023 , 134 : 103924 .

ZHENG S , LU J , ZHAO H , et al. Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 6881 - 6890 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al. An Image Is Worth 16x16 Words:Transformers for Image Recognition at Scale [J]. International Conference on Learning Representations , 2021 .

CHEN J , LU Y , YU Q , et al. TransUnet:Transformers Make Strong Encoders for Medical Image Segmentation(2021) [J/OL].[ 2021-02-08 ].https://arxiv.org/abs/2102.04306v1. https://arxiv.org/abs/2102.04306v1 https://arxiv.org/abs/2102.04306v1

ZHENG Y , WANG Z , XU B , et al. Multi-Scale Semantic Segmentation for Fire Smoke Image Based on Global Information and U-Net [J]. Electronics , 2022 , 11 ( 17 ): 2718 .

LIU Z , LIN Y , CAO Y , et al. Swin Transformer:Hierarchical Vision Transformer Using Shifted Windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2021 : 10012 - 10022 .

XIE E , WANG W , YU Z , et al. SegFormer:Simple and Efficient Design for Semantic Segmentation with Transformers [J]. Advances in Neural Information Processing Systems , 2021 , 34 : 12077 - 12090 .

VASWANI A , SHAZEER N , PARMAR N , et al. Attention Is All You Need[C]// Advances in Neural Information Processing Systems 30(NIPS 2017) . San Diego : NIPS , 2017 : 1 - 11 .

ISLAM M A , JIA S , BRUCE N D B . How Much Position Information Do Convolutional Neural Networks Encoder(2020) [J/OL].[ 2020-01-22 ].https://arxiv.org/abs/2001.08248. https://arxiv.org/abs/2001.08248 https://arxiv.org/abs/2001.08248

SANDLER M , HOWARD A , ZHU M , et al. Mobilenetv2:Inverted Residuals and Linear Bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 4510 - 4520 .

LIU Z , MAO H , WU C Y , et al. A Convnet for the 2020s[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2022 : 11976 - 11986 .

YU C , WANG J , PENG C , et al. Bisenet:Bilateral Segmentation Network for Real-Time Semantic Segmentation[C]//Proceedings of the European Conference on Computer Vision(ECCV) . Heidelberg : Springer , 2018 : 325 - 341 .

HU J , SHEN L , SUN G . Squeeze-and-Excitation Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7132 - 7141 .

GUO M H , LU C Z , HOU Q , et al. SegneXt:Rethinking Convolutional Attention Design for Semantic Segmentation [J]. Advances in Neural Information Processing Systems , 2022 , 35 : 1140 - 1156 .

GENG Z , GUO M H , CHEN H , et al. Is Attention Better Than Matrix Decomposition?(2021) [J/OL].[ 2021-09-09 ].https://arxiv.org/abs/2109.04553. https://arxiv.org/abs/2109.04553 https://arxiv.org/abs/2109.04553

ZHAO H , SHI J , QI X , et al. Pyramid Scene Parsing Network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 2881 - 2890 .

CHEN L C , ZHU Y , PAPANDREOU G , et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//Proceedings of the European Conference on Computer Vision(ECCV) . Heidelberg : Springer , 2018 : 801 - 818 .

YUAN Y , CHEN X , CHEN X , et al. Object-Contextual Representations for Semantic Segmentation[C]// Proceedings of the European Conference on Computer Vision(ECCV) . Heidelberg : Springer , 2020 : 173 - 190 .

XIAO T , LIU Y , ZHOU B , et al. Unified Perceptual Parsing for Scene Understanding[C]//Proceedings of the European Conference on Computer Vision(ECCV) . Heidelberg : Springer , 2018 : 418 - 434 .

BESBES O , BENAZZA-BENYAHIA A . A Novel Video-Based Smoke Detection Method Based on Color Invariants[C]//2016 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP) . Piscataway : IEEE , 2016 : 1911 - 1915 .

赵敏 , 张为 , 王鑫 , 等 . 时空背景模型下结合多种纹理特征的烟雾检测 [J]. 西安交通大学学报 , 2018 , 52 ( 8 ): 67 - 73 .

ZHAO Min , ZHANG Wei , WANG Xin , et al. A Smoke Detection Algorithm with Multi-Texture Feature Exploration Under a Spatio-Temporal Background Model [J]. Journal of Xi’an Jiaotong University , 2018 , 52 ( 8 ): 67 - 73 .

王浩远 , 梁煜 , 张为 . 融合多分辨率表征的实时烟雾分割算法 [J]. 浙江大学学报(工学版) , 2021 , 55 ( 12 ): 2334 - 2341 .

WHANG Haoyuan , LIANG Yu , ZHANG Wei . Real-Time Smoke Segmentation Algorithm Fused with Multi-Resolution Representation [J]. Journal of ZheJiang University(Engineering Science) , 2021 , 55 ( 12 ): 2334 - 2341 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

联合多尺度高低频信息融合的变化检测方法

面向带宽受限场景的高效语义通信方法

基于多尺度特征信息融合的时间序列异常检测

基于多边形特征池化与融合的复杂文本检测

基于多注意力机制的纹理感知视频修复方法