MAN

🌟🌟 Checkout our new work HSDT, a superior HSI denoising transformer.

Official PyTorch Implementation of Mixed Attention Network for Hyperspectral Image Denoising.

Zeqiang Lai, Ying Fu.

Overall Architecture

🌟 Hightlights

• We propose a "Mixed Attention Network" for hyperspectral image denoising.
• We introduce a "Multi-Head Spectral Recurrent Attention" block to aggregate the inter-spectral features.
• We introduce a "Progressive Channel Attention block" for integrating the intra-spectral features.
• We introduce an "Attentive Skip Connection" to strengthen the important features from low- and high-level.

Illustrations of each network components.

Progressive Spectral Channel Attention	Attentive Skip Connection

Multi-Head Recurrent Spectral Attention

Usage

Download the pretrained model at Github Release.

• Training, testing, and visualize results with HSIR.

python -m hsirun.test -a mans.man -r ckpt/man_gaussian.pth -t icvl_512_30 icvl_512_50
python -m hsirun.train -a mans.man -s schedule.gaussian
python -m hsirun.train -a mans.man -s schedule.complex -r checkpoints/mans.man/model_latest.pth
python -m hsiboard.app --logdir results

• Using our model.

import torch
from mans import man

net = man()
x = torch.randn(4,1,31,64,64)
y = net(x)

• Using our components.

import torch
from mans import (
    MAB, BiMAB,
    AdaptiveSkipConnection, SimplifiedChannelAttention,
)

x = torch.randn(4,16,31,64,64)
block = MAB(16) # or BiMAB(16)
out = block(x) # [4,16,31,64,64]

y = torch.randn(4,16,31,64,64)
block = AdaptiveSkipConnection(16)
out = block(x, y) # [4,16,31,64,64]

block = SimplifiedChannelAttention(16)
out = block(x) # [4,16,31,64,64]

Performance

MAN v2

Gaussian denoising on ICVL

Complex denoising on ICVL

MAN

Gaussian denoising on ICVL

Complex denoising on ICVL

Citations

@article{lai2023mixed,
  title={Mixed Attention Network for Hyperspectral Image Denoising},
  author={Lai, Zeqiang and Fu, Ying},
  journal={arXiv preprint arXiv:2301.11525},
  year={2023}
}
@inproceedings{lai2023hsdt,
  author = {Lai, Zeqiang and Chenggang, Yan and Fu, Ying},
  title = {Hybrid Spectral Denoising Transformer with Guided Attention},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year = {2023},
}