Showing 1–12 of 12 results for author: Shoeiby, M

Underwater Image Restoration via Contrastive Learning and a Real-world Dataset

Authors: Junlin Han, Mehrdad Shoeiby, Tim Malthus, Elizabeth Botha, Janet Anstee, Saeed Anwar, Ran Wei, Mohammad Ali Armin, Hongdong Li, Lars Petersson

Abstract: Underwater image restoration is of significant importance in unveiling the underwater world. Numerous techniques and algorithms have been developed in the past decades. However, due to fundamental difficulties associated with imaging/sensing, lighting, and refractive geometric distortions, in capturing clear underwater images, no comprehensive evaluations have been conducted of underwater image re… ▽ More Underwater image restoration is of significant importance in unveiling the underwater world. Numerous techniques and algorithms have been developed in the past decades. However, due to fundamental difficulties associated with imaging/sensing, lighting, and refractive geometric distortions, in capturing clear underwater images, no comprehensive evaluations have been conducted of underwater image restoration. To address this gap, we have constructed a large-scale real underwater image dataset, dubbed `HICRD' (Heron Island Coral Reef Dataset), for the purpose of benchmarking existing methods and supporting the development of new deep-learning based methods. We employ accurate water parameter (diffuse attenuation coefficient) in generating reference images. There are 2000 reference restored images and 6003 original underwater images in the unpaired training set. Further, we present a novel method for underwater image restoration based on unsupervised image-to-image translation framework. Our proposed method leveraged contrastive learning and generative adversarial networks to maximize the mutual information between raw and restored images. Extensive experiments with comparisons to recent approaches further demonstrate the superiority of our proposed method. Our code and dataset are publicly available at GitHub. △ Less

Submitted 20 June, 2021; originally announced June 2021.

Comments: In submission, code/dataset are at https://github.com/JunlinHan/CWR. arXiv admin note: text overlap with arXiv:2103.09697

Dual Contrastive Learning for Unsupervised Image-to-Image Translation

Authors: Junlin Han, Mehrdad Shoeiby, Lars Petersson, Mohammad Ali Armin

Abstract: Unsupervised image-to-image translation tasks aim to find a map** between a source domain X and a target domain Y from unpaired training data. Contrastive learning for Unpaired image-to-image Translation (CUT) yields state-of-the-art results in modeling unsupervised image-to-image translation by maximizing mutual information between input and output patches using only one encoder for both domain… ▽ More Unsupervised image-to-image translation tasks aim to find a map** between a source domain X and a target domain Y from unpaired training data. Contrastive learning for Unpaired image-to-image Translation (CUT) yields state-of-the-art results in modeling unsupervised image-to-image translation by maximizing mutual information between input and output patches using only one encoder for both domains. In this paper, we propose a novel method based on contrastive learning and a dual learning setting (exploiting two encoders) to infer an efficient map** between unpaired data. Additionally, while CUT suffers from mode collapse, a variant of our method efficiently addresses this issue. We further demonstrate the advantage of our approach through extensive ablation studies demonstrating superior performance comparing to recent approaches in multiple challenging image translation tasks. Lastly, we demonstrate that the gap between unsupervised methods and supervised methods can be efficiently closed. △ Less

Submitted 15 April, 2021; originally announced April 2021.

Comments: Accepted to NTIRE, CVPRW 2021. Code is available at https://github.com/JunlinHan/DCLGAN

Single Underwater Image Restoration by Contrastive Learning

Authors: Junlin Han, Mehrdad Shoeiby, Tim Malthus, Elizabeth Botha, Janet Anstee, Saeed Anwar, Ran Wei, Lars Petersson, Mohammad Ali Armin

Abstract: Underwater image restoration attracts significant attention due to its importance in unveiling the underwater world. This paper elaborates on a novel method that achieves state-of-the-art results for underwater image restoration based on the unsupervised image-to-image translation framework. We design our method by leveraging from contrastive learning and generative adversarial networks to maximiz… ▽ More Underwater image restoration attracts significant attention due to its importance in unveiling the underwater world. This paper elaborates on a novel method that achieves state-of-the-art results for underwater image restoration based on the unsupervised image-to-image translation framework. We design our method by leveraging from contrastive learning and generative adversarial networks to maximize mutual information between raw and restored images. Additionally, we release a large-scale real underwater image dataset to support both paired and unpaired training modules. Extensive experiments with comparisons to recent approaches further demonstrate the superiority of our proposed method. △ Less

Submitted 15 April, 2021; v1 submitted 17 March, 2021; originally announced March 2021.

Comments: Accepted to IGARSS 2021 as oral presentation. Code is available at https://github.com/JunlinHan/CWR

Mosaic Super-resolution via Sequential Feature Pyramid Networks

Authors: Mehrdad Shoeiby, Mohammad Ali Armin, Sadegh Aliakbarian, Saeed Anwar, Lars Petersson

Abstract: Advances in the design of multi-spectral cameras have led to great interests in a wide range of applications, from astronomy to autonomous driving. However, such cameras inherently suffer from a trade-off between the spatial and spectral resolution. In this paper, we propose to address this limitation by introducing a novel method to carry out super-resolution on raw mosaic images, multi-spectral… ▽ More Advances in the design of multi-spectral cameras have led to great interests in a wide range of applications, from astronomy to autonomous driving. However, such cameras inherently suffer from a trade-off between the spatial and spectral resolution. In this paper, we propose to address this limitation by introducing a novel method to carry out super-resolution on raw mosaic images, multi-spectral or RGB Bayer, captured by modern real-time single-shot mosaic sensors. To this end, we design a deep super-resolution architecture that benefits from a sequential feature pyramid along the depth of the network. This, in fact, is achieved by utilizing a convolutional LSTM (ConvLSTM) to learn the inter-dependencies between features at different receptive fields. Additionally, by investigating the effect of different attention mechanisms in our framework, we show that a ConvLSTM inspired module is able to provide superior attention in our context. Our extensive experiments and analyses evidence that our approach yields significant super-resolution quality, outperforming current state-of-the-art mosaic super-resolution methods on both Bayer and multi-spectral images. Additionally, to the best of our knowledge, our method is the first specialized method to super-resolve mosaic images, whether it be multi-spectral or Bayer. △ Less

Submitted 14 April, 2020; originally announced April 2020.

Comments: Accepted by IEEE CVPR Workshop

Multi-FAN: Multi-Spectral Mosaic Super-Resolution Via Multi-Scale Feature Aggregation Network

Authors: Mehrdad Shoeiby, Sadegh Aliakbarian, Saeed Anwar, Lars Petersson

Abstract: This paper introduces a novel method to super-resolve multi-spectral images captured by modern real-time single-shot mosaic image sensors, also known as multi-spectral cameras. Our contribution is two-fold. Firstly, we super-resolve multi-spectral images from mosaic images rather than image cubes, which helps to take into account the spatial offset of each wavelength. Secondly, we introduce an ext… ▽ More This paper introduces a novel method to super-resolve multi-spectral images captured by modern real-time single-shot mosaic image sensors, also known as multi-spectral cameras. Our contribution is two-fold. Firstly, we super-resolve multi-spectral images from mosaic images rather than image cubes, which helps to take into account the spatial offset of each wavelength. Secondly, we introduce an external multi-scale feature aggregation network (Multi-FAN) which concatenates the feature maps with different levels of semantic information throughout a super-resolution (SR) network. A cascade of convolutional layers then implicitly selects the most valuable feature maps to generate a mosaic image. This mosaic image is then merged with the mosaic image generated by the SR network to produce a quantitatively superior image. We apply our Multi-FAN to RCAN (Residual Channel Attention Network), which is the state-of-the-art SR algorithm. We show that Multi-FAN improves both quantitative results and well as inference time. △ Less

Submitted 6 November, 2019; v1 submitted 16 September, 2019; originally announced September 2019.

Super-resolved Chromatic Map** of Snapshot Mosaic Image Sensors via a Texture Sensitive Residual Network

Authors: Mehrdad Shoeiby, Lars Petersson, Mohammad Ali Armin, Sadegh Aliakbarian, Antonio Robles-Kelly

Abstract: This paper introduces a novel method to simultaneously super-resolve and colour-predict images acquired by snapshot mosaic sensors. These sensors allow for spectral images to be acquired using low-power, small form factor, solid-state CMOS sensors that can operate at video frame rates without the need for complex optical setups. Despite their desirable traits, their main drawback stems from the fa… ▽ More This paper introduces a novel method to simultaneously super-resolve and colour-predict images acquired by snapshot mosaic sensors. These sensors allow for spectral images to be acquired using low-power, small form factor, solid-state CMOS sensors that can operate at video frame rates without the need for complex optical setups. Despite their desirable traits, their main drawback stems from the fact that the spatial resolution of the imagery acquired by these sensors is low. Moreover, chromatic map** in snapshot mosaic sensors is not straightforward since the bands delivered by the sensor tend to be narrow and unevenly distributed across the range in which they operate. We tackle this drawback as applied to chromatic map** by using a residual channel attention network equipped with a texture sensitive block. Our method significantly outperforms the traditional approach of interpolating the image and, afterwards, applying a colour matching function. This work establishes state-of-the-art in this domain while also making available to the research community a dataset containing 296 registered stereo multi-spectral/RGB images pairs. △ Less

Submitted 5 September, 2019; originally announced September 2019.

PIRM2018 Challenge on Spectral Image Super-Resolution: Dataset and Study

Authors: Mehrdad Shoeiby, Antonio Robles-Kelly, Ran Wei, Radu Timofte

Abstract: This paper introduces a newly collected and novel dataset (StereoMSI) for example-based single and colour-guided spectral image super-resolution. The dataset was first released and promoted during the PIRM2018 spectral image super-resolution challenge. To the best of our knowledge, the dataset is the first of its kind, comprising 350 registered colour-spectral image pairs. The dataset has been use… ▽ More This paper introduces a newly collected and novel dataset (StereoMSI) for example-based single and colour-guided spectral image super-resolution. The dataset was first released and promoted during the PIRM2018 spectral image super-resolution challenge. To the best of our knowledge, the dataset is the first of its kind, comprising 350 registered colour-spectral image pairs. The dataset has been used for the two tracks of the challenge and, for each of these, we have provided a split into training, validation and testing. This arrangement is a result of the challenge structure and phases, with the first track focusing on example-based spectral image super-resolution and the second one aiming at exploiting the registered stereo colour imagery to improve the resolution of the spectral images. Each of the tracks and splits has been selected to be consistent across a number of image quality metrics. The dataset is quite general in nature and can be used for a wide variety of applications in addition to the development of spectral image super-resolution methods. △ Less

Submitted 1 May, 2019; v1 submitted 31 March, 2019; originally announced April 2019.

High-order Radio Frequency Differentiation via Photonic Signal Processing with an Integrated Micro-resonator Kerr Optical Frequency Comb Source

Authors: Xingyuan Xu, Jiayang Wu, Mehrdad Shoeiby, Sai T. Chu, Brent E. Little, Roberto Morandotti, Arnan Mitchell, David J. Moss

Abstract: We demonstrate the use of integrated micro-resonator based optical frequency comb sources as the basis for transversal filtering functions for microwave and radio frequency photonic filtering and advanced functions. We demonstrate the use of integrated micro-resonator based optical frequency comb sources as the basis for transversal filtering functions for microwave and radio frequency photonic filtering and advanced functions. △ Less

Submitted 7 April, 2018; originally announced April 2018.

Comments: 8 pages, 7 figures, 46 References. arXiv admin note: substantial text overlap with arXiv:1512.01741, arXiv:1512.06301

Journal ref: 2017 Progress In Electromagnetics Research Symposium Fall (PIERS, FALL), Singapore, 19 to 22 November, Pages 2232 to 2236

Photonic RF and microwave reconfigurable filters and true time delays based on an integrated optical Kerr frequency comb source

Authors: Xingyuan Xu, Jiayang Wu, Thach G. Nguyen, Mehrdad Shoeiby, Sai T. Chu, Brent E. Little, Roberto Morandotti, Arnan Mitchell, David J. Moss

Abstract: We demonstrate advanced transversal radio frequency (RF) and microwave functions based on a Kerr optical comb source generated by an integrated micro-ring resonator. We achieve extremely high performance for an optical true time delay aimed at tunable phased array antenna applications, as well as reconfigurable microwave photonic filters. Our results agree well with theory. We show that our true t… ▽ More We demonstrate advanced transversal radio frequency (RF) and microwave functions based on a Kerr optical comb source generated by an integrated micro-ring resonator. We achieve extremely high performance for an optical true time delay aimed at tunable phased array antenna applications, as well as reconfigurable microwave photonic filters. Our results agree well with theory. We show that our true time delay would yield a phased array antenna with features that include high angular resolution and a wide range of beam steering angles, while the microwave photonic filters feature high Q factors, wideband tunability, and highly reconfigurable filtering shapes. These results show that our approach is a competitive solution to implementing reconfigurable, high performance and potentially low cost RF and microwave △ Less

Submitted 31 January, 2018; originally announced February 2018.

Comments: 15 pages, 11 Figures, 60 References

Journal ref: Optics Express Vol. 26, Issue 3, pp. 2569-2583 (2018)

Photonic radio frequency and microwave intensity differentiator based on an optical frequency comb source in an integrated micro-ring resonator

Authors: Xingyuan Xu, Jiayang Wu, Mehrdad Shoeiby, Thach G. Nguyen, Sai T. Chu, Brent E. Little, Roberto Morandotti, Arnan Mitchell, David J. Moss

Abstract: We propose and experimentally demonstrate a microwave photonic intensity differentiator based on a Kerr optical comb generated by a compact integrated micro-ring resonator (MRR). The on-chip Kerr optical comb, containing a large number of comb lines, serves as a high-performance multi-wavelength source for implementing a transversal filter, which will greatly reduce the cost, size, and complexity… ▽ More We propose and experimentally demonstrate a microwave photonic intensity differentiator based on a Kerr optical comb generated by a compact integrated micro-ring resonator (MRR). The on-chip Kerr optical comb, containing a large number of comb lines, serves as a high-performance multi-wavelength source for implementing a transversal filter, which will greatly reduce the cost, size, and complexity of the system. Moreover, owing to the compactness of the integrated MRR, frequency spacings of up to 200-GHz can be achieved, enabling a potential operation bandwidth of over 100 GHz. By programming and sha** individual comb lines according to calculated tap weights, a reconfigurable intensity differentiator with variable differentiation orders can be realized. The operation principle is theoretically analyzed, and experimental demonstrations of first-, second-, and third-order differentiation functions based on this principle are presented. The radio frequency (RF) amplitude and phase responses of multi-order intensity differentiations are characterized, and system demonstrations of real-time differentiations for a Gaussian input signal are also performed. The experimental results show good agreement with theory, confirming the effectiveness of our approach. △ Less

Submitted 26 September, 2017; originally announced October 2017.

Comments: 12 pages, 6 figures, 49 references

Journal ref: Applied Physics Letters (APL) Photonics volume 2, 096104 (2017)

Microwave and RF Applications for Micro-resonator based Frequency Combs

Authors: Thach G. Nguyen, Mehrdad Shoeiby, Marcello Ferrera, Alessia Pasquazi, Marco Peccianti, Sai T. Chu, Brent E. Little, Roberto Morandotti, Arnan Mitchell, David J. Moss

Abstract: Photonic integrated circuits that exploit nonlinear optics in order to generate and process signals all-optically have achieved performance far superior to that possible electronically - particularly with respect to speed. We review the recent achievements based in new CMOS-compatible platforms that are better suited than SOI for nonlinear optics, focusing on radio frequency (RF) and microwave bas… ▽ More Photonic integrated circuits that exploit nonlinear optics in order to generate and process signals all-optically have achieved performance far superior to that possible electronically - particularly with respect to speed. We review the recent achievements based in new CMOS-compatible platforms that are better suited than SOI for nonlinear optics, focusing on radio frequency (RF) and microwave based applications that exploit micro-resonator based frequency combs. We highlight their potential as well as the challenges to achieving practical solutions for many key applications. These material systems have opened up many new capabilities such as on-chip optical frequency comb generation and ultrafast optical pulse generation and measurement. We review recent work on a photonic RF Hilbert transformer for broadband microwave in-phase and quadrature-phase generation based on an integrated frequency optical comb. The comb is generated using a nonlinear microring resonator based on a CMOS compatible, high-index contrast, doped-silica glass platform. The high quality and large frequency spacing of the comb enables filters with up to 20 taps, allowing us to demonstrate a quadrature filter with more than a 5-octave (3 dB) bandwidth and an almost uniform phase response. △ Less

Submitted 19 December, 2015; originally announced December 2015.

Comments: 10 pages, 6 figures, 68 references. arXiv admin note: substantial text overlap with arXiv:1512.01741

Report number: SPIE Photonics West, San Francisco, February 2016 paper 9750-15 to be published

Hilbert Transform based Quadrature Hybrid RF Photonic Coupler via a Micro-Resonator Optical Frequency Comb Source

Authors: Thach G. Nguyen, Mehrdad Shoeiby, Sai T. Chu, Brent E. Little, Roberto Morandotti, Arnan Mitchell, David J. Moss

Abstract: We demonstrate a photonic RF Hilbert transformer for broadband microwave in-phase and quadrature-phase generation based on an integrated frequency optical comb, generated using a nonlinear microring resonator based on a CMOS compatible, high-index contrast, doped-silica glass platform. The high quality and large frequency spacing of the comb enables filters with up to 20 taps, allowing us to demon… ▽ More We demonstrate a photonic RF Hilbert transformer for broadband microwave in-phase and quadrature-phase generation based on an integrated frequency optical comb, generated using a nonlinear microring resonator based on a CMOS compatible, high-index contrast, doped-silica glass platform. The high quality and large frequency spacing of the comb enables filters with up to 20 taps, allowing us to demonstrate a quadrature filter with more than a 5-octave (3 dB) bandwidth and an almost uniform phase response. △ Less

Submitted 6 December, 2015; originally announced December 2015.

Comments: 11 pages, 8 figures, 38 references

Journal ref: Optics Express Volume 23 Issue 17 Pages 22087-22097 (2015)