A New Cross-Space Total Variation Regularization Model for Color Image Restoration with
Quaternion Blur Operator

Abstract

The cross-channel deblurring problem in color image processing is difficult to solve due to the complex coupling and structural blurring of color pixels. Until now, there are few efficient algorithms that can reduce color infection in deblurring process. To solve this challenging problem, we present a novel cross-space total variation (CSTV) regularization model for color image deblurring by introducing a quaternion blur operator and a cross-color space regularization functional. The existence and uniqueness of the solution is proved and a new L-curve method is proposed to find a sweet balance of regularization functionals on different color spaces. The Euler-Lagrange equation is derived to show that CSTV has taken into account the coupling of all color channels and the local smoothing within each color channel. A quaternion operator splitting method is firstly proposed to enhance the ability of color infection reduction of the CSTV regularization model. This strategy also applies to the well-known color deblurring models. Numerical experiments on color image databases illustrate the efficiency and manoeuvrability of the new model and algorithms. The color images restored by them successfully maintain the color and spatial information and are of higher quality in terms of PSNR, SSIM, MSE and CIEde2000 than the restorations of the-state-of-the-art methods.

I Introduction

Blurring process of color image is typically applied to all color channels and is also cross-channel. The observed color pixels may have been affeced by neighboring pixels, and there possiblly are mutual contamination between their color channels. The classic and latest color image restoration models have made outstanding contributions in this regard. Their innovation focuses on the improvement of regularization terms, while the fidelity items basically follow the classic total variation (TV) model. A long-overlooked, but important, fact is that fidelity items containing blur operators require more accurate characterization and robust algorithms. In this paper, we present a learning regularization term and a new characterization of fidelity item for color image restoration, and also develop a new efficient and fast algorithm to enhance the performance of deblurring.

A color image contains multiple color channels, each of which can be thought of as a binary function. A general color image degradation model is

$$\mathbf{z}(x,y)=\mathbf{K}\star\mathbf{u}(x,y)+\mathbf{n}(x,y),$$ (1)

where $\mathbf{z}(x,y),\mathbf{K},\mathbf{u}(x,y),\mathbf{n}(x,y)$ denote degraded color image, blur operator, original color image and additive noise, respectively. Here, $\star$ denotes the convolution operator and $\mathbf{K}\star\mathbf{u}(x,y)$ characterizes the blurring process. One main task of color image processing is to reconstruct the original color image $\mathbf{u}(x,y)$ from the observed color image $\mathbf{z}(x,y)$. Since the noise $\mathbf{n}(x,y)$ is unknown, one has to inversely solve (1) with minimizing the noise indicator. This unfortunately leads to an ill-posed inverse problem. The way of transforming it into a well-posed problem is to introduce a regularization term into the initial problem. However, the development of color image regularization is blocked for a long time by the difficulty of characterizing complexly coupling and structured blur of color pixels and the hardness to eliminate color infection.

A groundbreaking color image regularization is the (global) channel coupling color TV regularization [1], followed by a local version [2]. These two regularization functionals are based on the RGB color space and couple red, green and blue channels by different norms of TVs on three channels and they can been seen as different norms of vectorial TV of color image. A general framework, called collaborative TV [6], incorporates various norms along different dimensions to provide a comprehensive approach to regularization and the considerd norms includes nuclear, Frobenius, and spectral norms. Recently, Duan et al. [5] proposed a new Beltrami regularization model for color image denoising and an efficient and robust operator splitting method, with regarding color images as manifolds embedded in a five dimensional spatial-chromatic space. And a new color elastica model is developed in [17, 18] by using the Polyakov action and a Laplace–Beltrami operator on color channels. These beautiful regularization functionals greatly enriche color image models based on vector representation in one color space.

To find more efficient manners of coupling, various studies explore color space transformation methods to generate regularization functionals on new color space. The chrominance, luminance, or R/B component was formed through a weighted linear combination of the R, G, and B channels of the color image in [3]. This process is also represented as $\sum_{k}~{}TV(u_{k}\circ\psi)$ in [6], where $\psi$ is an orthonormal transform to describe the color space transformation. Besides, the human eye is highly sensitive to changes in the opponent color channel. In [25], the opponent transform is employed to transform the RGB color space into the opponent space and the decomposition of the coupling of color channels is allowed. Jia, Ng and Wang [21] recently introduce a saturation-value TV (SVTV) model (given later in (4)) based on quaternion representation of color images. The SVTV regularization functional successfully reflects the physical principle of the human visual system and considers the coupling of red, green, and blue channels. So the SVTV model reconstructs color images of high quality and with slight color infection. The SVTV ragularization has been combined with other methods to handle other image processing tasks and the corresponding modified models achieve at a high level on numerical performance; see [13, 31, 32, 33] for instance.

Above models consider color image restoration in a single color space and thier regullarizations have been well known for their prior on color edges. Can their advantages be combined into one model? A natural way is to define a learnable regularization term that can chararize the color structures in different color spaces. (See the general model (8).) It is expected to perform bettter than those with single color space regularization. Color image restoration models with cross color space regularition terms breaks out the limitations in terms of preserving color and texture information. This is shown in Section III-A and also in Section V.

As is well known, the SVTV regularization model is a quaternion-based description [21]. In the actual alternating iteration process, color images are not directly represented and computed by quaternions but by their real counterparts. This conversion of quaternion operations into real operations makes the calculation easier. However, this could cause color infection in the deblurring process (see numerical experiments in Section V). In order not to weaken the advantages of the quaternion model, we need to solve the quaternion model using the quaternion algorithm. In this paper, we further embed quaternion operations in color image cross-channel restoration based on the quaternion operator splitting approach that considers color pixels as quaternions rather than a vectorised representation of individual color channels. Additionally, we directly solve the quaternion linear systems throughout the iterative process using the quaternion Krylov subspace method. This decision will contribute to color fidelity. One effective existing quaternion Krylov subspace method is the quaternion generalized minimal residual method (QGMRES) [22], which can provide more accurate numerical solutions. For details, please refer to [22]. Therefore, the quaternion linear systems in the deblurring process will be solved by the QGMRES method in this paper.

The contribution is in three aspects:

• We present a versatile framework for balanced regularization in multi-color space blending, enhancing color image restoration. This framework contains diverse regularization terms across different color spaces, maintaining color channel relationships and addressing limitations of single-space models. We detail the cross-channel SVTV-CTV balanced regularization model with RGB and HSV color space regularization and provide theoretical proof of its solution’s existence and uniqueness.

• A new algorithm is proposed for the cross-channel SVTV-CTV balance regularization model based on the framework of the alternating minimization method, based on quaternion operator splitting.

• The newly proposed method is applied to color image cross-channel restoration. Their efficiency and superiority are indicated by numerical results in terms of visual, PSNR, SSIM, MSE and CIEde2000 criteria. The numerical results also support the assertion that the proposed new methods can better preserve the color fidelity and texture.

The structure of this paper is as follows. We review several preliminary findings of the quaternion and SVTV regularization models in Section II of this paper. In Section III, we propose a new color image restoration model with an SVTV-CTV balance regularization term and give theoretical proof of the existence and uniqueness of the model solution. In Section IV, we propose an effective algorithm for the cross-channel SVTV-CTV balance regularization model. In Section V, we demonstrate numerical examples to illustrate the superiority of the proposed methods. In Section VI, we present concluding remarks.

II Preliminaries

In this section, we shortly recall quaternions and the existing total variation regularization functions for color images.

Let $\mathbb{Q}=\left\{{\bf a}=a_{0}+a_{1}\mathbf{i}+a_{2}\mathbf{j}+a_{3}\mathbf{k}|a_{0},a_{1},a_{2},a_{3}\in\mathbb{R}\right\}$ denote the quaternion skew-field, $\mathbb{Q}^{n}$ the set of $n$-dimensional quaternion vectors, and $\mathbb{Q}^{m\times n}$ the set of $m\times n$ quaternion matrices [15], where three imaginary units $\mathbf{i},\mathbf{j},\mathbf{k}$ satisfy

$$\mathbf{i}^{2}=\mathbf{j}^{2}=\mathbf{k}^{2}=\mathbf{i}\mathbf{j}\mathbf{k}=-1.$$

For a quaternion ${\bf a}=a_{0}+a_{1}\mathbf{i}+a_{2}\mathbf{j}+a_{3}\mathbf{k}\in\mathbb{Q}$, $a_{0}$ is called real part and $a_{1},a_{2},a_{3}$ are called three imagenary parts. Pure quaternion is nonzero quaternion with zero real part. The conjugate and modulus of ${\bf a}$ are defined by $\bar{{\bf a}}=a_{0}-a_{1}\mathbf{i}-a_{2}\mathbf{j}-a_{3}\mathbf{k}$ and $|{\bf a}|=\sqrt{\bar{{\bf a}}{\bf a}}=\sqrt{a_{0}^{2}+a_{1}^{2}+a_{2}^{2}+a_{3}^{2}}$, respectively. Every nonzero quaternion is invertible and its unique inverse is defined by ${\bf a}^{-1}=\bar{{\bf a}}/|{\bf a}|^{2}$. The conjugate transpose of quaternion vector ${\bf v}=v_{0}+v_{1}\mathbf{i}+v_{2}\mathbf{j}+v_{3}\mathbf{k}\in\mathbb{Q}^{n}$ is defined as ${\bf v}^{*}=v_{0}^{T}-v_{1}^{T}\mathbf{i}-v_{2}^{T}\mathbf{j}-v_{3}^{T}\mathbf{k}$, where $v_{0},v_{1},v_{2},v_{3}\in\mathbb{R}^{n}$ and $\cdot^{T}$ denotes the transpose operator. Similarly, the conjugate transpose of quaternion matrix ${\bf A}=A_{0}+A_{1}\mathbf{i}+A_{2}\mathbf{j}+A_{3}\mathbf{k}\in\mathbb{Q}^{m\times n}$ is defined as ${\bf A}^{*}=A_{0}^{T}-A_{1}^{T}\mathbf{i}-A_{2}^{T}\mathbf{j}-A_{3}^{T}\mathbf{k}$, where $A_{0},A_{1},A_{2},A_{3}\in\mathbb{R}^{m\times n}$. According to [22], we define a homeomorphic map** $\Re$ from quaternion matrices, vectors, scalars or functions to real structured real matrices:

$$\Re({\bf A})=\left[\begin{array}[]{rrrr}A_{0}&-A_{1}&-A_{2}&-A_{3}\\ A_{1}&A_{0}&-A_{3}&A_{2}\\ A_{2}&A_{3}&A_{0}&-A_{1}\\ A_{3}&-A_{2}&A_{1}&A_{0}\end{array}\right].$$ (2)

Let $\Re_{c}({\bf A})$ denote the first column of $\Re({\bf A})$.

Now we introduce the measurement of quaternion vectors and matrices. The absolute of quaternion vector ${\bf v}=[{\bf v}_{i}]\in\mathbb{Q}^{n}$ is $|{\bf v}|=[|{\bf v}_{i}|]\in\mathbb{R}^{n}.$ Similarly, the absolute of quaternion matrix ${\bf A}=[{\bf a}_{ij}]\in\mathbb{Q}^{m\times n}$ is $|{\bf A}|=[|{\bf a}_{ij}|]\in\mathbb{R}^{m\times n}.$ From [20], we know that quaternion vector (or matrix) norms are functions from quaternion vectors (or matrices) to nonnegative real numbers.

Definition 1.

Let $p\geq 1$. The $p$-norm of ${\bf v}\in\mathbb{Q}^{n}$ is $\|{\bf v}\|_{p}=\big{(}\sum_{i=1}^{n}|{\bf v}_{i}|^{p}\big{)}^{\frac{1}{p}}.$ The $p$-norm and $F$-norm of ${\bf A}\in\mathbb{Q}^{m\times n}$ are

$$\|{\bf A}\|_{p}=\max_{{\bf x}\in\mathbb{Q}^{n}/\{0\}}\frac{\|{\bf A}{\bf x}\|_{p}}{\|{\bf x}\|_{p}},~{}\|{\bf A}\|_{F}=\big{(}\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}|{\bf a}_{ij}|^{2}\big{)}^{\frac{1}{2}}.$$

From this definition and the definition (2), one can easily derive that $\|{\bf v}\|_{2}=\|\Re_{c}({\bf v})\|_{2}=\frac{1}{2}\|\Re({\bf v})\|_{2}$, $\|{\bf A}\|_{2}=\|\Re({\bf A})\|_{2}$ and $\|{\bf A}\|_{F}=\|\Re_{c}({\bf A})\|_{F}=\frac{1}{2}\|\Re({\bf A})\|_{F}$.

Next, we give a brief description of the SVTV regularization function for color image based on quaternion representation [21]. A quaternion function

$$\mathbf{u}(x,y)=u_{1}(x,y)\mathbf{i}+u_{2}(x,y)\mathbf{j}+u_{3}(x,y)\mathbf{k}$$

was used to represent a color image in the RGB color space in [21], where $(x,y)$ denotes the position of a color pixel in a given range $\Omega$ and three real binary functions $u_{i}(x,y)$$(i=1,~{}2,~{}3)$ denote pixel values of red, green and blue channels, respectively. After a strict derivation of quaternion operations, a real-valued version of SVTV was also given in [21], which make it possible to handle it by real calculations instead of quaternion calculations. For simplicity, let $\partial_{x/y}$ denote $\partial_{x}$ or $\partial_{y}$. Without causing misunderstandings, we still use the following vector function to represent a color image,

$$\mathbf{u}(x,y):=\left[\begin{array}[]{ccc}u_{1}(x,y)&u_{2}(x,y)&u_{3}(x,y)\end{array}\right]^{T}.$$ (3)

Define

$$\partial_{x/y}\mathbf{u}(x,y)^{T}\!=\!\left[\!\begin{array}[]{c}\partial_{x/y}u_{1}(x,y)\\ \partial_{x/y}u_{2}(x,y)\\ \partial_{x/y}u_{3}(x,y)\end{array}\!\right]\!,C=\left[\begin{array}[]{rrr}2&-1&-1\\ -1&2&-1\\ -1&-1&2\end{array}\right].$$

The saturation and value components of color image $\mathbf{u}(x,y)$ are defined as

$$|\mathbf{u}(x,y)|_{s}=\frac{1}{2}\|C\mathbf{u}(x,y)^{T} \|_{2},$$

$$|\mathbf{u}(x,y)|_{v}=\frac{1}{\sqrt{3}}|\sum\limits_{i=1}^{3}u_{i}(x,y)|.$$

The SVTV regularization is defined in [21] by

$${\rm{SVTV}}(\mathbf{u})=\int_{\Omega}\sqrt{(|\partial_{x}\mathbf{u}(x,y)|_{s}^{2})+(|\partial_{y}\mathbf{u}(x,y)|_{s}^{2})}dxdy$$ (4)

$$+\alpha\sqrt{(|\partial_{x}\mathbf{u}(x,y)|_{v}^{2})+(|\partial_{y}\mathbf{u}(x,y)|_{v}^{2})}dxdy,$$

where $\alpha$ is the weight of the value component. ${\rm{SVTV}}(\mathbf{u})$ is exactly a functional of real functions: $u_{1}(x,y),~{}u_{2}(x,y),~{}u_{3}(x,y)$.

Before SVTV regularization, there are several well-known TV regulariztions developed for color image in the RGB color space. For instance,

• Blomgren and Chan [1]:

  $${\rm CTV}_{1}(\mathbf{u}):=\sqrt{\sum_{i=1}^{3}\bigg{(}\int_{\Omega}\|\nabla u_{i}(x,y)\|_{2}~{}dxdy\bigg{)}^{2}}.$$ (5)

• Bresson and Chan [2]:

  $${\rm CTV}_{2}(\mathbf{u}):=\int_{\Omega}\sqrt{\sum_{k=1}^{3}\|\nabla u_{i}(x,y)\|_{2}^{2}}~{}dxdy.$$ (6)

• Sapiro [27]:

  $${\rm VTV}(\mathbf{u}):=\int_{\Omega}f(\lambda((\nabla\mathbf{u}(x,y))^{T}\nabla\mathbf{u}(x,y)))dxdy,$$ (7)

  where $\lambda(\cdot)$ refers to eigenvalues and $f(\cdot)$ is a penalty function on eigenvalues.

In above equations, $\nabla=[\partial_{x},\partial_{y}]^{T}$ represents the gradient operator. In [6], classic and advanced versions of vector TV regularizations are summarized into a classification called collaborative total variation (CTV). When applied to color images, they are all based on one color space. A new regularization on defferent color spaces will be introduced in next section.

III Cross-space color image restoration model

In this section, we present a novel cross color space regularization and a new color image restoration model for cross-channel deblurring.

With the degradation model (1) in hand, we are concerned with a general framework for color image restoration expressed as

$$\mathop{\min}\limits_{\bm{\lambda}\in\mathbb{R}^{m}\atop\mathbf{u}(x,y)\in\mathbb{BV}(\Omega)}~{}J(\mathbf{u},\bm{\lambda})+F(\mathbf{u},\mathbf{z},\mathbf{K}),$$ (8)

where $\mathbb{R}^{m}$ denotes the $m$-dimensional real-valued vector space, $\mathbb{BV}(\Omega)=\{\mathbf{u}(x,y)=u_{1}(x,y)\mathbf{i}+u_{2}(x,y)\mathbf{j}+u_{3}(x,y)\mathbf{k}~{}|~{}u_{i}(x,y)\in{\rm BV}(\Omega),~{}(x,y)\in\Omega\}$ and ${\rm BV}(\Omega)$ is bounded variation space on a given color image region $\Omega\subseteq\mathbb{R}^{2}$. The function $J(\mathbf{u},\bm{\lambda})$ punishes the gradients of the targeted color image $\mathbf{u}(x,y)$ in different color spaces and it is called a cross color space regularization term with balance parameters $\bm{\lambda}=(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\in\mathbb{R}^{m}$. The function $F(\mathbf{u},\mathbf{z},\mathbf{K})$ is a data fidelity term to control the distance between the targeted and observed color images under the blurring operator.

We now face two difficulties: how to construct the correct regularization term, and how to use quaternion matrices to approximate blurring operators. Next, let us do a detailed analysis.

III-A Cross color space regularization term

The regularization term $J(\mathbf{u},\bm{\lambda})$ is a fusion of regularizations on different color spaces. This fusion is tailored to extract information across various dimensions within color images, thereby enhancing the preservation of color and texture details. For instance, a regularization crossing HSV and RGB color spaces can be defined by

$$J(\mathbf{u},\bm{\lambda})=\lambda_{1}{\rm{SVTV}}(\mathbf{u})+\lambda_{2}{\rm{CTV}}(\mathbf{u}),$$ (9)

where regulariation function ${\rm{SVTV}}$ is defined in (4), ${\rm{CTV}}$ represents ${\rm{CTV}}_{1}$ or ${\rm{CTV}}_{2}$ defined in (5) and (6), and $\lambda_{1},\lambda_{2}\in\mathbb{R}$. The choice of different regularization fuctions is empirical and it relies on the requirement from practical applictions. A solution based on machine learning methods is to perform a linear combination of all possible regular functions with parameters, and then learn the best parameters. There are a number of well-known techniques for dynamically calculating optimal parameters, such as the generalized cross-validation [11, 16], the L-curve method [14] and the discrepancy principle [29]. Here, we generalize the L-curve method to a new L-surface method to determine optimal parameters simultaneously.

The L-surface method is to find a sweet spot on a surface with respect to a certain quality index of the parameter $\bm{\lambda}$. To be clear, we discribe the L-surface method by applying it to determine the optimal $\bm{\lambda}=(\lambda_{1},\lambda_{2})$ in (9). Suppose we have obtained an observed color image, say ’statues’ in Figure 5, under the setting of Example V-A. The upper bound of parameters is set as $b=2\sqrt{N\sigma^{2}}\times 10^{-3}$, where $N$ is image size and $\sigma$ is the noise level. The surface of PSNR values of restored color images are plotted in Figure 1 (a) according to $\lambda_{1},\lambda_{2}\in(0,b)$. High PSNR values are obtained with $\lambda_{1}\in(0.4\times 10^{-3},b)$ and $\lambda_{2}\in(0.2\times 10^{-3},0.5\times 10^{-3})$ and the highest PSNR value is $27.1903$ at $(\lambda_{1}=0.8\times 10^{-3},~{}\lambda_{2}=0.2\times 10^{-3})$. The surface of SSIM values of restored color images are plotted in Figure 1 (b) according to $\lambda_{1},\lambda_{2}\in(0,b)$. High SSIM values are obtained with $\lambda_{1}\in(0.5\times 10^{-3},b)$ and $\lambda_{2}\in(0.1\times 10^{-3},0.3\times 10^{-3})$, and the highest SSIM value is $0.7704$ at $(\lambda_{1}=0.8\times 10^{-3},~{}\lambda_{2}=0.1\times 10^{-3})$. By the weighted average surface of above PSNR and SSIM surfaces, we obtain the sweet spot $\bm{\lambda}=(0.8\times 10^{-3},~{}0.2\times 10^{-3})$ that leads to a good combination of SVTV and CTV regularizations.

Figure 1: The values of PSNR and SSIM for the restored image ‘statues’ in different $\lambda_{1},\lambda_{2}$ conditions.

Figure 2: The values of PSNR and SSIM for the restored image ‘statues’ in $\lambda_{1}=0$ or $\lambda_{2}=0$.

Obviously, the combined regularization term $J(\mathbf{u},\bm{\lambda})$ performs better than single regullariztion ${\rm{SVTV}}(\mathbf{u})$ or ${\rm{CTV}}(\mathbf{u})$. This implicits that the color image restoration model (8) with the combined regularization term can obtain a broader range of pre-existing knowledge about color image and thus it is enriched an ability to interpret color image. To demonstrate the superiority, we plot the PSNR and SSIM values in Figure 2 at boundary $\lambda_{1}=0$ or $\lambda_{2}=0$. We can see that the PSNR and SSIM values in Figure 2 are significantly smaller than $27.1903$ and $0.7704$ , respectively. So one can conclude that color image restoration models with cross color space regularition performs bettter than those with single color space regularization.

III-B Fidelity term with quaternion operator

The fidelity term $F(\mathbf{u},\mathbf{z},\mathbf{K})$ of color image restoration model (8) aims to minimize the distance between the observed color image $\mathbf{z}$ and the original color image $\mathbf{u}$ after a blurring operation. Next, we introduce a new discription of blur operator by quaternion operator. The new form can lead to a stable algorithm to compute a solution without color confusion and mutual infection.

At first, we recall the cross-channel blur mechanism. The color image cross-channel blurring process is mathematically described in [9, 10, 12] as

$$\hat{\mathbf{u}}(x,y) =\mathbf{K}\star\mathbf{u}(x,y),$$ (10)

where $\mathbf{K}=W\odot K$ with

$$K\!\!=\!\!\left[\!\begin{array}[]{ccc}K_{11}&K_{12}&K_{13}\\ K_{21}&K_{22}&K_{23}\\ K_{31}&K_{32}&K_{33}\end{array}\!\right]\!\!,~{}W\!\!=\!\!\left[\!\begin{array}[]{ccc}w_{11}&w_{12}&w_{13}\\ w_{21}&w_{22}&w_{23}\\ w_{31}&w_{32}&w_{33}\end{array}\!\right]\!\!.$$ (17)

In (17), the sub-matrices of $K$, denoted by $K_{ij}$’s, are real blur kernels, and the entries of $W$, $w_{ij}$’s, belong to the interval $[0,1]$ and $\sum_{j=1}^{3}w_{ij}=1$ for $i=1,2,3$.

The cross-channel blurring process (10) can be embedded into a higher dimentional space,

$$\left[\begin{array}[]{cccc}\hat{u}_{0}(x,y)\\ \hat{u}_{r}(x,y)\\ \hat{u}_{g}(x,y)\\ \hat{u}_{b}(x,y)\end{array}\right] =\left[\begin{array}[]{c:ccc}B_{11}&B_{12}&B_{13}&B_{14}\\ \hdashline B_{21}&&&\\ B_{31}&&\mathbf{K}&\\ B_{41}&&&\end{array}\right]\star\left[\begin{array}[]{cccc}u_{0}(x,y)\\ u_{1}(x,y)\\ u_{2}(x,y)\\ u_{3}(x,y)\end{array}\right],$$ (18)

where $\hat{u_{0}}(x,y)$ and $u_{0}(x,y)$ are zero functions and $B_{ij}$’s are arbitrary operators of the same size of $K_{ij}$’s in (17). Denote the extended blurring operator by $B$, that is,

$$B=\left[\begin{array}[]{c:ccc}B_{11}&B_{12}&B_{13}&B_{14}\\ \hdashline B_{21}&&&\\ B_{31}&&\mathbf{K}&\\ B_{41}&&&\end{array}\right].$$ (19)

It can be splitted into a sum of a JRS-symmetric matrix $Q$ and a $4\times 4$ real block matrix $R$, according to the following rules:

$$B=Q+R,$$ (20)

where

$$Q=\frac{1}{4}(B+J_{n}B{J_{n}}^{T}+R_{n}B{R_{n}}^{T}+S_{n}B{S_{n}}^{T}),$$ (21)

$$R=\frac{1}{4}(3B-J_{n}B{J_{n}}^{T}-R_{n}B{R_{n}}^{T}-S_{n}B{S_{n}}^{T}),$$

and $J_{n},R_{n},S_{n}$ are unitary operators defined in [24]. Since these unitary operators satisfy $J_{n}J_{n}=-I_{4n}$, $J_{n}R_{n}=S_{n}$ and $J_{n}S_{n}=-R_{n}$, we can easily derive

$$J_{n}QJ_{n}^{T}=Q,~{}~{}R_{n}QR_{n}^{T}=Q,~{}~{}S_{n}QS_{n}^{T}=Q.$$

From [24], there exists a quaternion matrix

$${\bf Q}=Q_{0}+Q_{1}\mathbf{i}+Q_{2}\mathbf{j}+Q_{3}\mathbf{k}$$ (22)

such that

$$Q=\Re({\bf Q}),$$ (23)

and

$$Q_{0} =\frac{1}{4}(B_{11}+w_{11}K_{11}+w_{22}K_{22}+w_{33}K_{33}),$$ (24)

$$Q_{1} =\frac{1}{4}(B_{21}-B_{12}+w_{32}K_{32}-w_{23}K_{23}),$$

$$Q_{2} =\frac{1}{4}(B_{31}-B_{13}+w_{13}K_{13}-w_{31}K_{31}),$$

$$Q_{3} =\frac{1}{4}(B_{41}-B_{14}+w_{21}K_{21}-w_{12}K_{12}).$$

Note that the blocks $B_{ij}$’s of matrix $B$ defined by (19) are solution of the minimization problem:

$$\mathop{\min}\limits_{B_{ij}}~{}\|B-Q\|_{F}$$ (25)

$$\text{s.t.}~{}~{}B_{12}\star u_{1}+B_{1,3}\star u_{2}+B_{14}\star u_{3}=0.$$

Now we introduce a quaternion representation of blurring process. Define

$$\hat{\mathbf{u}}(x,y) =\hat{u}_{0}(x,y)+\hat{u}_{r}(x,y)\mathbf{i}+\hat{u}_{g}(x,y)\mathbf{j}+\hat{u}_{b}(x,y)\mathbf{k},$$

$${\mathbf{u}}(x,y) ={u}_{0}(x,y)+{u}_{r}(x,y)\mathbf{i}+{u}_{g}(x,y)\mathbf{j}+{u}_{b}(x,y)\mathbf{k}.$$

Based on the above matrix splitting, the blurring process (18) can be rewritten as

$$\hat{\mathbf{u}}(x,y)={\bf Q}\star\mathbf{u}(x,y)+{\bf r}(x,y),$$ (26)

where

$${{\bf r}}(x,y)=r_{0}(x,y)+r_{1}(x,y)\mathbf{i}+r_{2}(x,y)\mathbf{j}+r_{3}(x,y)\mathbf{k},$$

$$\left[\begin{array}[]{rrrr}r_{0}(x,y)\\ r_{1}(x,y)\\ r_{2}(x,y)\\ r_{3}(x,y)\end{array}\right]=R\star\left[\begin{array}[]{rrrr}u_{0}(x,y)\\ u_{1}(x,y)\\ u_{2}(x,y)\\ u_{3}(x,y)\end{array}\right].$$

In the discrete form, the cross-channel blurring process is in fact a sum of a quaternion matrix-vector product ${\bf Q}\mathbf{u}$ and a quaternion vector ${\bf r}$. The quaternion operations can preserve the ratio of red, green and blue information of color pixels in the blurring process and thus, color confusion and interinfection can be reduced in the delburring process. As a result, the correlation of color channels are well preserved and the revovered color images can achieve a high quality. In theroy, there always exists an invertible quaternion matrix ${\bf Q}$ such that ${\bf Q}\mathbf{u}=\widehat{\mathbf{u}}$. However, it is impractical to compute such quaternion matrix, since a large number of pairs of original and observed color images under the same blurring process are in need. Especially, the original color image is definitely unknown. So we altermatively constuct ${\bf Q}$ according to (22), (24) and (25) to minimize $\|{\bf r}\|_{2}$ in this paper.

Let us turn back to construct the fidelity term $F(\mathbf{u},\mathbf{z},\mathbf{K})$ of color image restoration model (8). To treat different kinds of noise, we define the distance between the observed and original color images by $L_{p}$ norm as

$$F(\mathbf{u},\mathbf{z},\mathbf{K})=\frac{1}{p}\int_{\Omega}|\mathbf{K}\star\mathbf{u}(x,y)-\mathbf{z}(x,y)|^{p}dxdy,~{}~{}p\geq 1.$$ (27)

According to (10) and (26), we have a quaternion operator form

$$F(\mathbf{u},\mathbf{z},\mathbf{K})=\frac{1}{p}\int_{\Omega}|{\bf Q}\star\mathbf{u}(x,y)+{\bf r}(x,y)-\mathbf{z}(x,y)|^{p}dxdy.$$ (28)

This general framework offers a flexibility to adaptlivey choose suitable fidelity terms to rescove color images based on specific noise types and image texture characteristics. For instance, one can explore the utilization of the $L_{2}$ fidelity term to reduce Gaussian noise, the $L_{1}$ fidelity term to eliminate poisson and mixed noise, and the $L_{1}$-$L_{2}$ mixed fidelity term for more effectively mitigating losses induced by degradation factors. The forms in (27) and (28) are mathematically equivalent to each other. The former is used for theorectical analysis and the later is used to design stable and fast algorithm.

III-C A new color image restoration model

Now we propose a color image restoration model with a cross color space regularisation term and a fidelity with quatrnion operator. To make the discription clear, we concentrate into two color spaces: the widely used RGB color space and the more human eye-perceiving HSV color space.

Let the cross-space regularization be defined as in (9) and the fidelity term in (28). For an observed image $\mathbf{z}$, a new cross-space color image restoration model is

$$\widehat{\mathbf{u}}=\mathop{\arg\min}\limits_{\mathbf{u}(x,y)\in\mathbb{BV}(\Omega)}~{}\lambda_{1}{\rm{SVTV}}(\mathbf{u})+\lambda_{2}{\rm{CTV}}(\mathbf{u})+$$ (29)

$$\frac{1}{p}\int_{\Omega}|{\bf Q}\star\mathbf{u}+{\bf r}-\mathbf{z}|^{p}dxdy,$$

where $\lambda_{1}$ and $\lambda_{2}$ are two given nonnegative regularization parameters, and $p$ is a positive integer not less than $1$. Here, parameters $\lambda_{1}$ and $\lambda_{2}$ play in two roles: one is the regularization parameters to banlance regularization and fidelity terms, and another is the balance parameters between two regularization terms.

This model contains one fidelity term and two regularization terms: SVTV regularization and CTV regularization. The two regularizations complement each other with color and texture information. In this way, the loss of information caused by considering only one color space will be decreased. Parameters $\lambda_{1}$ and $\lambda_{2}$ control the balance of them. When $\lambda_{1}=0$, the model degenerates to the classical color total variation model; when $\lambda_{2}=0$, the model degenerates to the saturation value total variation model; and when both $\lambda_{1}$ and $\lambda_{2}$ are greater than 0, both regularization terms come into play and the information of saturation, value, red, green and blue is used for color image restoration.

Next, we analyze the solvability of the newly proposed model (29) with $p=2$.

Theorem 1.

If $p=2$, there is at least one solution for the minimization problem (29) and moreover, the solution is unique when $\mathbf{u}\mapsto{\bf Q}\star\mathbf{u}+{\bf r}$ is injective.

Proof:

If $\mathbf{u}(x,y)$ is a constant value function, then ${\rm{SVTV}}(\mathbf{u})=0$ and ${\rm{CTV}}(\mathbf{u})=0$. The energy in (7) thus becomes finite. This implies that the infimum of the energy must be finite. Suppose that $\left\{\mathbf{u}^{(n)}(x,y)\right\}$ is a minimizing sequence of problem (7). Consequently, a constant $M>0$ exists such that

$${\rm{SVTV}}(\mathbf{u}^{(n)})\leq M.$$

We get that $\left\{{\rm{SVTV}}(\mathbf{u}^{(n)})+\sum\limits_{i=1}^{3}\|u_{i}^{(n)}\|_{L^{1}}\right\}$ is uniformly bounded by combining the boundedness of $u_{1}^{(n)},u_{2}^{(n)},u_{3}^{(n)}$. Noting the compactness of [21], we know that there exist $u_{1}^{*},u_{2}^{*},u_{3}^{*}$ such that

$$u_{i}^{(n)}(x,y)\!\xrightarrow{{L^{1}}(\Omega)}\!u_{i}^{*}(x,y),~{}~{}~{}u_{i}^{(n)}(x,y)\!\rightarrow\!u_{i}^{*}(x,y)$$

$$a.e.~{}(x,y)\!\in\!\Omega,~{}~{}i=1,2,3.$$

Recall that ${\bf Q}\star\mathbf{u}(x,y)+{\bf r}(x,y)=\mathbf{K}\star\mathbf{u}(x,y)$. Subsequently, the convergence of the following results is valid:

$$(\mathbf{K}\star u_{i}^{(n)}(x,y)-z_{i}(x,y))^{2}\rightarrow(\mathbf{K}\star u_{i}^{*}(x,y)-z_{i}(x,y))^{2}$$

$$a.e.~{}~{}(x,y)\in\Omega,~{}~{}i=1,2,3.$$

Taking advantage of the Fatou’s lemma, one has

$$\sum\limits_{i=1}^{3}\lim\inf\int_{\Omega}(\mathbf{K}\star u_{i}^{(n)}(x,y)-z_{i}(x,y))^{2}dxdy$$

$$\geq\sum\limits_{i=1}^{3}\int_{\Omega}(\mathbf{K}\star u_{i}^{*}(x,y)-z_{i}(x,y))^{2}dxdy.$$

On account of the lower semicontinuity of ${\rm{CTV}}(\mathbf{u})$ and ${\rm{SVTV}}(\mathbf{u})$,

$$\lim\inf~{}{\rm{CTV}}(\mathbf{u}^{(n)})\geq{\rm{CTV}}(\mathbf{u}^{*}),$$

$$\lim\inf~{}{\rm{SVTV}}(\mathbf{u}^{(n)})\geq{\rm{SVTV}}(\mathbf{u}^{*}),$$

then we obtain

$$\lim\inf~{}(\lambda_{1}{\rm{SVTV}}(\mathbf{u}^{(n)})+\lambda_{2}{\rm{CTV}}(\mathbf{u}^{(n)})$$

$$~{}~{}~{}~{}~{}~{}~{}+\sum\limits_{i=1}^{3}\int_{\Omega}(\mathbf{K}\star u_{i}^{(n)}(x,y)-z_{i}(x,y))^{2}dxdy)$$

$$\geq\lambda_{1}{\rm{SVTV}}(\mathbf{u}^{*})+\lambda_{2}{\rm{CTV}}(\mathbf{u}^{*}$$

$$~{}~{}~{}~{}~{}~{}+\sum\limits_{i=1}^{3}\int_{\Omega}(\mathbf{K}\star u_{i}^{*}(x,y)-z_{i}(x,y))^{2}dxdy).$$

It leads to the existence of the solution of (7) in the main body. If $\mathbf{u}\mapsto{\bf Q}\star\mathbf{u}+{\bf r}$ $(=\mathbf{K}\star\mathbf{u})$ is injective, the uniqueness of the solution can be obtained quickly by combining the convexity of SVTV($\mathbf{u}$) and CTV($\mathbf{u}$). ∎

An important advantage of the newly proposed model (29) is that it takes into account the color channels coupling and the local smoothing within each channel. This implicitly leads to achieving better results in terms of both image texture and color fidelity. We can get the explanation from the corresponding Euler-Lagrange equation:

$$\frac{\lambda_{1}}{3}\nabla\cdot\Bigg{(}\frac{\nabla\left({\bf C}\mathbf{u}\right)}{\sqrt{(|\partial_{x}\mathbf{u}(x,y)|_{s}^{2})+(|\partial_{y}\mathbf{u}(x,y)|_{s}^{2})}}$$ (30)

$$+\alpha\frac{\nabla\mathbf{u}}{\sqrt{(|\partial_{x}\mathbf{u}(x,y)|_{v}^{2})+(|\partial_{y}\mathbf{u}(x,y)|_{v}^{2})}}\Bigg{)}$$

$$+\lambda_{2}\sum\limits_{i=1}^{3} \nabla\cdot\left(\frac{\nabla u_{i}(x,y)}{|\mathbf{u}(x,y)|}\right)-\left(\mathbf{K}^{*}\star\mathbf{K}\star\mathbf{u}-\mathbf{K}^{*}\star\mathbf{z}\right)=0.$$

In (30), the proposed model takes into account both the diffusion coefficients of the channel coupling over the saturation and value components and the diffusion coefficients of the summation of the individual channels over the RGB space. The quaternion representation also leads to many other advantages, such as the values of three color channels of a color pixel are flocked together and their physical meanings are preserved in the whole color image processing based on quaternion computation. Especially, color distortion will be hugely reduced in the recovered color image by the new model.

To end this section, we present the dual form of model (29). The dual forms of regularization functions SVTV and CTV are

$${\rm{SVTV}}(\mathbf{u}) =\max_{\|g\|\leq 1}~{}\left<\operatorname{diag}(1,1,\alpha){\bf P}\mathbf{u},\nabla g\right>,$$

$${\rm{CTV}}(\mathbf{u}) =\max_{\|h\|\leq 1}~{}\left<\mathbf{u},\nabla h\right>,$$

where ${\bf P}$ is an orthogonal transformation matrix that takes the form of

$${\bf P}=\left[\begin{array}[]{ccc}\frac{1}{\sqrt{2}}{I}&\frac{-1}{\sqrt{2}}{I}&\textbf{0}\\ \frac{1}{\sqrt{6}}{I}&\frac{1}{\sqrt{6}}{I}&\frac{-2}{\sqrt{6}}{I}\\ \frac{1}{\sqrt{3}}{I}&\frac{1}{\sqrt{3}}{I}&\frac{1}{\sqrt{3}}{I}\end{array}\right].$$

By converting the regularization terms into dual forms, we can rephrase the minimization problem (29) as the following dual form:

$$\min\limits_{\mathbf{u}}\!\max\limits_{\|g\|\leq 1\atop\|h\|\leq 1}\lambda_{1}\!\left<\operatorname{diag}(1,1,\alpha){\bf P}\mathbf{u},\nabla g\right>+\lambda_{2}\!\left<\mathbf{u},\nabla h\right>+\atop\frac{1}{2}\|{\bf Q}\star\mathbf{u}+{\bf r}-\mathbf{z}\|^{2}.$$ (31)

In the dual form, the solution $\mathbf{u}(x,y)$ can be differentiable or not, which expands the feasible set. Obviously, the min-max problem (31) is a new saddle-point problem.

There are various methods to solve the new model (29) or the dual form (31) with real variables. However, there no methods for solving them with quaternion variables. So we will develop fast algorithms for solving model (29) in next section.

IV Fast algorithms

In this section, we present a new stable and fast algorithm for model (29) based on quaternion operator splitting.

It is impractical to solve the Euler-Lagrange equation (30) directly, since a non-linear differential equation leads to a non-linear system after discretization. So we introduce two auxilary variables into (29) and obtain an mathematically equivalent model:

$$\displaystyle\widehat{\mathbf{u}}=\mathop{\arg\min}\limits_{\mathbf{u}}~{}\lambda_{1}{\rm{SVTV}}(\mathbf{w})+\lambda_{2}{\rm{CTV}}({\bf v})+\atop\frac{1}{2}\|{\bf Q}\star\mathbf{u}+{\bf r}-\mathbf{z}\|^{2},$$

$$\text{s.t.}~{}~{}\mathbf{w}=\mathbf{u},~{}~{}{\bf v}=\mathbf{u}.$$

The constraints mentioned above can be converted into an unconstrained problem:

$$\widehat{\mathbf{u}}=\mathop{\arg\min}\limits_{\mathbf{u},\mathbf{w},{\bf v}}~{}\lambda_{1}{\rm{SVTV}}(\mathbf{w})+\frac{\alpha_{1}}{2}\|\mathbf{w}-\mathbf{u}\|^{2}+\atop\frac{1}{2}\|{\bf Q}\star\mathbf{u}+{\bf r}-\mathbf{z}\|^{2}+\lambda_{2}{\rm{CTV}}({\bf v})+\frac{\alpha_{2}}{2}\|{\bf v}-\mathbf{u}\|^{2},$$ (32)

where $\alpha_{1}$ and $\alpha_{2}$ are two positive parameters. Three unknown variables $\mathbf{u}$, $\mathbf{w}$ and ${\bf v}$ can be splitted into two indepentent groups and under the framework of alternating minimization method, problem (32) can be solved by altermatively solving two subproblems:

• $\mathbf{u}$-subproblem

  With fixing $\mathbf{w}$ and ${\bf v}$, the minimization problem (32) is equivalently reduced to

  $$\mathop{\arg\min}\limits_{\mathbf{u}}~{}\frac{\alpha_{1}}{2}\|\mathbf{u}-\mathbf{w}\|^{2}+\frac{\alpha_{2}}{2}\|\mathbf{u}-{\bf v}\|^{2}+\atop\frac{1}{2}\|{\bf Q}\star\mathbf{u}+{\bf r}-\mathbf{z}\|^{2}.$$ (33)

• $(\mathbf{w},{\bf v})$-subproblem

  With fixing $\mathbf{u}$, the minimization problem (32) is equivalently reduced to

  $$\mathop{\arg\min}\limits_{\mathbf{w},{\bf v}}~{}\lambda_{1}{\rm{SVTV}}(\mathbf{w})+\frac{\alpha_{1}}{2}\|\mathbf{w}-\mathbf{u}\|^{2}+\atop\lambda_{2}{\rm{CTV}}({\bf v})+\frac{\alpha_{2}}{2}\|{\bf v}-\mathbf{u}\|^{2}.$$ (34)

IV-A Quaternion operator splitting method for $\mathbf{u}$-subproblem

Now we consider the $\mathbf{u}$-subproblem (33) and use its real presentation to introduce a quaternion operator splitting method.

From the analysis in Section III-B, the real representation of (33) is

$$\mathop{\arg\min}\limits_{\mathbf{u}}~{}\frac{\alpha_{1}}{2}\|\mathbf{u}-\mathbf{w}\|^{2}+\frac{\alpha_{2}}{2}\|\mathbf{u}-{\bf v}\|^{2}+\atop\frac{1}{2}\|B\mathbf{u}-\mathbf{z}\|^{2},$$ (35)

where $B=Q+R$ with $Q$ and $R$ being 4-by-4 block matrices defined by (21). Its nomral equation is

$$[B^{T}B+(\alpha_{1}+\alpha_{2}){I}]\mathbf{u}=B^{T}\mathbf{z}+\alpha_{1}\mathbf{w}+\alpha_{2}{\bf v}.$$ (36)

We split $B^{T}B$ into $Q^{T}Q+Q^{T}R+R^{T}R+R^{T}Q$ and obtain an iterative format

$$\mathbf{u}^{k+1}=-(Q^{T}R+R^{T}Q+R^{T}R)\mathbf{u}^{k}$$ (37)

$$+\lambda(Q+R)^{T}\mathbf{z}+\alpha_{1}\mathbf{w}+\alpha_{2}{\bf v}.$$

From the structure of $Q$ as in (23), we can construct a equivalent quaternion linear system

$${\bf A}\mathbf{u}^{k+1}={\bf b},$$ (38)

where ${\bf A}=A_{0}+A_{1}\mathbf{i}+A_{2}\mathbf{j}+A_{3}\mathbf{k}$ with $\Re({\bf A})=Q^{T}Q+\alpha_{1}{I}+\alpha_{2}{I}$ and ${\bf b}=b_{0}+b_{1}\mathbf{i}+b_{2}\mathbf{j}+b_{3}\mathbf{k}$ with $\Re_{c}({\bf b})$ denotes the right handside of (37). So the solution of the $\mathbf{u}$-subproblem (33) can be computed by iteratively solving (38).

In the above, we have proposed a quaternion operator splitting method for solving the $\mathbf{u}$-subproblem, where the coefficient matrix is splitted into a quaternion operator and a residual operator. This leads to an advantage that the color information of restored color image $\mathbf{u}$ is preserved by quaternion algebra operations in the solving process. This advantage will be verified numerically in Section V. Now the core work becomes solving (38). Obviousely, the QGMRES method proposed in [22] is feasible to solve this quaternion linear system. However, we can further apply the Hermitian and positive definte properties of ${\bf A}$ to develop a new quaternion conjugate gradient method. Inspired by [22], we present a structure-preserving version in Algorithm 1. The quaternion matrix-vector product is implemented by real operation, i.e., ${\bf A}{\bf x}_{k}=\Re^{-1}(\Re({\bf A})\Re({\bf x}_{k}))$.

Algorithm 1 Quaternion Conjugate Gradient Method

1:  Initialization Set the stop** criteria $tol>0$ and the initial solution ${\bf x}_{0}=0$.

2:  $k=0$, ${\bf r}_{k}={\bf b}-\Re^{-1}(\Re({\bf A})\Re({\bf x}_{k}))$ and let ${\bf p}_{k}={\bf r}_{k}$.

3:  while $\|\Re_{c}({\bf r}_{k})\|_{2}>tol$ do

4:     ${\bf q}_{k}=\Re^{-1}(\Re({\bf A})\Re({\bf p}_{k}))$,

5:     Compute $a_{k}=\frac{\left<{\bf r}_{k},{\bf r}_{k}\right>}{\left<{\bf p}_{k},{\bf q}_{k}\right>}$,

6:     Compute ${\bf x}_{k+1}={\bf x}_{k}+a_{k}{\bf p}_{k}$,

7:     Compute ${\bf r}_{k+1}={\bf r}_{k}+a_{k}{\bf q}_{k}$,

8:     Compute $b_{k}=\frac{\left<{\bf r}_{k+1},{\bf r}_{k+1}\right>}{\left<{\bf r}_{k},{\bf r}_{k}\right>}$,

9:     Compute ${\bf p}_{k+1}={\bf r}_{k+1}+b_{k}{\bf p}_{k}$,

10:     $k=k+1$.

11:  end while

IV-B Augmented Lagrangian method for $(\mathbf{u},{\bf v})$-subproblem

Now, we present a new augmented Lagrangian method for solving $(\mathbf{w},{\bf v})$-subproblem (34). Since the two variables are independent of each other and have no intersecting terms, we construct the augmented Lagrangian schemes of computing $\mathbf{w}$ and ${\bf v}$, respectively.

• $\mathbf{w}$-subproblem

  With $\mathbf{u}$ and ${\bf v}$ fixed, the minimization problem (34) is reduced to

  $$\mathop{\arg\min}\limits_{\mathbf{w}}~{}\lambda_{1}{\rm{SVTV}}(\mathbf{w})+\frac{\alpha_{1}}{2}\|\mathbf{w}-\mathbf{u}\|^{2}.$$ (39)

  We define two discrete differential operators $\mathbf{D}_{x}$, $\mathbf{D}_{y}$,

  $$(\mathbf{D}_{x}\mathbf{w})_{i,j}=\mathbf{w}(i,j)-\mathbf{w}(i-1,j),$$

  $$(\mathbf{D}_{y}\mathbf{w})_{i,j}=\mathbf{w}(i,j)-\mathbf{w}(i,j-1),$$

  under adequate boundary conditions for color image. Define

  $${\bf s}={\bf P}\mathbf{w}~{}\text{and}~{}{\bf q}={\bf P}\mathbf{u}.$$

  Then the objective color image restoration model (39) is reformulated as

  $$\mathop{\arg\min}\limits_{{\bf s}}~{}\frac{\lambda_{1}}{\alpha_{1}}\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{n}(\sqrt{\sum\limits_{k=1}^{2}((\mathbf{D}_{x}s_{k})_{ij})^{2}\!+\!((\mathbf{D}_{y}s_{k})_{ij})^{2}}$$ (40)

  $$+\alpha\sqrt{((\mathbf{D}_{x}s_{3})_{ij})^{2}+((\mathbf{D}_{y}s_{3})_{ij})^{2}})+\frac{1}{2}\|{\bf s}-{\bf q}\|^{2}.$$

  By introducing auxiliary variables $t_{i}^{x}$ and $t_{i}^{y}$, the minimization problem (40) is equivalently rewritten into

  $$\mathop{\arg\min}\limits_{{\bf s}}~{}\frac{\lambda_{1}}{\alpha_{1}}\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{n}(\sqrt{\sum\limits_{k=1}^{2}|(t_{k}^{x})_{ij}|^{2}+|(t_{k}^{y})_{ij}|^{2}}$$

  $$+\alpha\sqrt{|(t_{3}^{x})_{ij}|^{2}+|(t_{3}^{y})_{ij}^{2}|})+\frac{1}{2}\|{\bf s}-{\bf q}\|^{2},$$

  s.t. $$t_{1}^{x}=\mathbf{D}_{x}s_{1},t_{2}^{x}=\mathbf{D}_{x}s_{2},t_{3}^{x}=\mathbf{D}_{x}s_{3},$$

  $$t_{1}^{y}=\mathbf{D}_{y}s_{1},t_{2}^{y}=\mathbf{D}_{y}s_{2},t_{3}^{y}=\mathbf{D}_{y}s_{3}.$$

  The augmented Lagrangian of the aforementioned minimization problem is

  $$\frac{\lambda_{1}}{\alpha_{1}}\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{n}(\sqrt{\sum\limits_{k=1}^{2}|(t_{k}^{x})_{ij}|^{2}+|(t_{k}^{y})_{ij}|^{2}}$$ (41)

  $$+\alpha\sqrt{|(t_{3}^{x})_{ij}|^{2}+|(t_{3}^{y})_{ij}^{2}|})+\frac{1}{2}\|{\bf s}-{\bf q}\|^{2}$$

  $$+\sum\limits_{i=1}^{3}((\tau_{i}^{x},t_{i}^{x}-\mathbf{D}_{x}s_{i})+(\tau_{i}^{y},t_{i}^{y}-\mathbf{D}_{y}s_{i}))$$

  $$+\frac{\beta}{2}\sum\limits_{i=1}^{3}(\|t_{i}^{x}-\mathbf{D}_{x}s_{i}\|^{2}+\|t_{i}^{y}-\mathbf{D}_{y}s_{i}\|^{2}),$$

  where $\tau_{i}^{x}$ and $\tau_{i}^{y}$ are Lagrangian multipliers and $\beta$ is a positive penalty parameter.

  According to the classic framework of the ADMM [7, 8], we construct a method to sovle (41) with three steps as follows.

  Step one: When fixing ${\bf s}$, $t_{i}^{x}$ and $t_{i}^{y}$ are computed by using the soft shrinkage functions,

  $$t_{i}^{x,y}= \max{(0,n_{1}-\frac{\lambda_{1}}{\beta\alpha_{1}})}\cdot\frac{\mathbf{D}_{x}s_{i}-\frac{\tau_{i}^{x,y}}{\beta}}{n_{1}},~{}~{}~{}~{}i=1,2,$$ (42a)

  $$t_{3}^{x,y}= \max{(0,n_{2}-\frac{\alpha\lambda_{1}}{\beta\alpha_{1}})}\cdot\frac{\mathbf{D}_{x}s_{3}-\frac{\tau_{3}^{x,y}}{\beta}}{n_{2}},$$ (42b)

  where

  $$n_{1} =\sqrt{\sum\limits_{i=1}^{2}(\mathbf{D}_{x}s_{i}-\frac{\tau_{i}^{x}}{\beta})^{2}+(\mathbf{D}_{y}s_{i}-\frac{\tau_{i}^{y}}{\beta})^{2}},$$

  $$n_{2} =\sqrt{(\mathbf{D}_{x}s_{3}-\frac{\tau_{3}^{x}}{\beta})^{2}+(\mathbf{D}_{y}s_{3}-\frac{\tau_{3}^{y}}{\beta})^{2}}.$$

  Step two: When fixing $t_{i}^{x}$ and $t_{i}^{y}$, ${\bf s}$ is computed by solving the following minimization problem

  $$\frac{1}{2}\|{\bf s}-{\bf q}\|^{2}+\sum\limits_{i=1}^{3}((\tau_{i}^{x},t_{i}^{x}-\mathbf{D}_{x}s_{i})+(\tau_{i}^{y},t_{i}^{y}-\mathbf{D}_{y}s_{i}))$$

  $$+\frac{\beta}{2}\sum\limits_{i=1}^{3}(\|t_{i}^{x}-\mathbf{D}_{x}s_{i}\|^{2}+\|t_{i}^{y}-\mathbf{D}_{y}s_{i}\|^{2}).$$

  The above minimization problem can be equivalently described by the following real linear system

  $$({I}+\beta\mathbf{D}_{x}^{T} \mathbf{D}_{x}+\beta\mathbf{D}_{y}^{T}\mathbf{D}_{y}){\bf s}$$ (43)

  $$={\bf q}+\sum\limits_{i=1}^{3}(\mathbf{D}_{x}^{T}\tau_{i}^{x}+ \mathbf{D}_{y}^{T}\tau_{i}^{y}+\beta\mathbf{D}_{x}^{T}t_{i}^{x}+\beta\mathbf{D}_{y}^{T}t_{i}^{y}).$$

  Once ${\bf s}$ is computed, we obtain $\mathbf{w}={\bf P}^{-1}{\bf s}$.

  Step three: The Lagrangian multipliers are updated as follows

  $$\tau_{i}^{x}\!=\!\tau_{i}^{x}+\beta(t_{i}^{x}-\mathbf{D}_{x}s_{i}),~{}~{}\tau_{i}^{y}\!=\!\tau_{i}^{y}+\beta(t_{i}^{y}-\mathbf{D}_{y}s_{i}),~{}i=1,2,3.$$

• ${\bf v}$-subproblem

  With $\mathbf{u}$ an $\mathbf{w}$ fixed, the minimization problem (34) is reduced to

  $$\mathop{\arg\min}\limits_{{\bf v}}~{}\lambda_{2}{\rm{CTV}}({\bf v})+\frac{\alpha_{2}}{2}\|{\bf v}-\mathbf{u}\|^{2}.$$ (44)

  This minimization problem can be solved by using a three-step method similar to that of the $\mathbf{w}$-subproblem. The solution can also be computed by the method given in [2].

To sumup, the $(\mathbf{w},{\bf v})$-subproblem need to compute several soft shrinkage functions and solve two real linear systems.

IV-C New algorithm

Now we present a new algorithm for model (29) based on the methods in Sections IV-A and IV-B. The puedo code is given in Algorithm 2.

Algorithm 2 Quaternion Operator Splitting Method

1:  Initialization Choose parameters $\lambda_{1},\lambda_{2},\alpha_{1},\alpha_{2},\tau_{i}^{x},\tau_{i}^{y}$ $(i=1,2,3),\alpha,\beta$. Set the stop** criteria $tol>0$ and $\mathbf{u}^{0}=\bf{0}$, $\mathbf{w}^{0}=\mathbf{z},{\bf v}^{0}=\mathbf{z},err=1,err_{u}=1,l=0$.

2:  while $err>tol$ do

3:     $k=0,\mathbf{u}^{k}=\mathbf{u}^{l}$

4:     while $err_{u}>tol$ do

5:        Solve $\mathbf{u}^{k+1}$ by (38) using Algorithm 1,

6:        $k=k+1$,

7:        Compute $err_{u}=\frac{\|\mathbf{u}^{k-1}-\mathbf{u}^{k}\|_{2}^{2}}{\|\mathbf{u}^{k-1}\|_{2}^{2}}$,

8:     end while

9:     $\mathbf{u}^{l+1}=\mathbf{u}^{k}$,

10:     Update $\mathbf{w}^{l+1}$ by solving (39),

11:     Update ${\bf v}^{l+1}$ by solving (44),

12:     $l=l+1$,

13:     Compute $err=\frac{\|{\bf v}^{l-1}-{\bf v}^{l}\|_{2}^{2}}{\|{\bf v}^{l-1}\|_{2}^{2}}$.

14:  end while

The main idea of Aglorithm 2 is to alternately solve (33) and (34) with proper initial values. The only problem left for discussion is how to determine their augumented Larange parameters $\alpha_{i}$’s. This is one of the well-known parameter-selection problems that stand for a long history. Here, we apply the newly proposed L-surface method in Section III-A to determine the optimal $\alpha_{i}$’s. These parameters are determined using a few typical images before running before running Algorithm 2 and, once determined, are used to process all images of the same degradation type.

Now we demonstrate the parameter selection process of $\alpha_{i}^{\prime}s$ by the L-surface method in Section III-A. Suppose that we have obtained an observed color image, say ’statues’ in Figure 5, under the setting of Example V-A. In case of taking CIEde2000 value as a criterion, we plot the surface of CIEde2000 values according to $\alpha_{1},\alpha_{2}\in[0,0.4]$ of restored color images in Figure 3. The optimal parameter is $\alpha_{1}=0.28,~{}\alpha_{2}=0.06$ at which the minimum CIEde2000 value reaches $3.1410$.

Figure 3: The values of CIEde2000 for the restored image ‘statues’ in different $\alpha_{1}$ and $\alpha_{2}$ conditions.

V Numerical Experiments

In this section, we consider various types of cross-channel blur kernels and weight matrices to illustrate the validity of the proposed model and algorithms. We compare the proposed m Algorithm 2 (referred to as Alg. 2) with the ${\rm{CTV}}_{1}$ [1], ${\rm{CTV}}_{2}$ [2], the ${\rm{SVTV}}$ [21], the split-algorithm-based ${\rm{CTV}}$ (for simplicity, the following is referred to as $\rm{{\rm{CTV}}_{split}}$) [30], ${\rm{MTV}}$ [34] by applying them to solve cross-channel deblurring problems. All experiments were performed by MATLAB ( R2020a) on a computer with Intel(R) Xeon(R) CPU E5-2630 @ 2.40Ghz/32.00 GB. The stop** criterion for all these iterative methods is that the norm of the successive iterations is less than $1.0\times 10^{-5}$.

We use the ground truth images shown in Figure 4 to test the image restoration capability of the proposed methods. Let $\mathbf{U}$, $\hat{\mathbf{U}}$ and $\mathbf{U}_{m}$ denote the original, observed and restored color images of size $n\times n$, respectively. The quality of the restored color image is indicated by the four standard criteria: PSNR, SSIM, MSE, and CIEde2000. Here, PSNR means the peak signal-to-noise ratio value of $\mathbf{U}_{m}$, defined by

$$\texttt{PSNR}(\mathbf{U}_{m},\mathbf{U})=10*\log_{10}\left(\frac{255^{2}n^{2}}{\|\mathbf{U}_{m}-\mathbf{U}\|_{F}^{2}}\right).$$

SSIM denotes the structural similarity index [28] of $\mathbf{U}_{m}$ and $\mathbf{U}$, defined by

$$\texttt{SSIM}(\mathbf{U}_{m},\mathbf{U})=\frac{(4\mu_{x}\mu_{y}+c_{1})(2\sigma_{xy}+c_{2})}{(\mu_{x}^{2}+\mu_{y}^{2}+c_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2})},$$

where $x$ and $y$ respectively stand for the vector forms of $\mathbf{U}$ and $\hat{\mathbf{U}}$, $c_{1,2}$ are two constants, $\mu_{x,y}$ signify the averages of $x$ and $y$, and $\sigma_{x,y}^{2}$ stand for the variances of $x$ and $y$, and $\sigma_{xy}$ denotes the covariance between $x$ and $y$. MSE is the mean square error value of $\mathbf{U}_{m}$, defined by

$$\texttt{MSE}(\mathbf{U}_{m},\mathbf{U})=\frac{\|\mathbf{U}_{m}-\mathbf{U}\|_{F}^{2}}{n^{2}}.$$

The CIEde2000 color difference formula is described in [26], which is used to evaluate the color difference between the original and the restored image.

Figure 4: Ground truth images

V-A Symmetric weight matrix

In this example, we consider the case where all sub-blur kernels are identical and the weight matrix is symmetric. The degraded images used for testing are generated for all models by applying the following cross-channel blur to the clean images,

$$ones(3,3)\otimes(G,5,5)$$

adding Gaussian noise with standard deviation $\sigma=0.01$ . The weight matrix is chosen as follows

$$W=\left[\begin{array}[]{ccc}0.7&0.15&0.15\\ 0.15&0.7&0.15\\ 0.15&0.15&0.7\end{array}\right].$$

We manually tuned the parameters in the three models to make the results solved by each model numerically optimal. In Figures 5 and 6, we give the observed images after the above degradation process, and the restored images of the seven models: The corresponding evaluation criteria for each of the methods used to restore the images are shown in Table I. We observed that the newly proposed methods have the highest PSNR, SSIM, MSE, and CIEde2000 values.

From Figures 5 and 6 we can see that ${\rm{CTV}}_{1}$ and ${\rm{CTV}}_{2}$ have a large color shift. The reason for this is that the algorithms corresponding to these methods are designed for within-channel blurring. As a result, only the diagonal blocks of the blur matrix are extracted for processing when considering cross-channel blurring. The $\rm{{\rm{CTV}}_{split}}$ still has some artifacts; see the grass skirt detail in ’aborigine’ and the watercress in ’bear’. The ${\rm{MTV}}$ has obtained relatively good results visually and numerically when dealing with cross-channel blurring. However, the ${\rm{MTV}}$ method seems oversmoothing in some small details; see the flower texture in the upper right corner of the ’butterfly’. Alg. 2 is an algorithms for the $\rm{SVTV}$-$\rm{CTV}$ balance regularization model, which restores images of higher quality under the same stop** criterion. Alg. 2 is more efficient in terms of PSNR, SSIM and CIEde2000 values because it introduces quaternion and thus considers color pixel as a whole.

To further highlight the differences between the ${\rm{MTV}}$ and Alg. 2 visual restoration results, we enlarge the local features and display them in Figure 7. The corresponding zoomed portions are denoted with red boxes in Figures 5 and 6. Magnification of local features reveals that Alg. 2 preserves more texture details; see the tip of a tree branch in ’waterfall’, the texture of petals in ’butterfly’, and the details of the letters in ’garden’. Additionally, we can see color infection at the edges of the ${\rm{MTV}}$ restored images, including the part of the hair in ’girl’ and the left edge of the flower in ’butterfly’ and the lower half of the fence in ’aborigine’.

Figure 5: Restoration results at $\sigma=0.01$ when the weight matrix is symmetric. (Listed in order are the observed images and the restoration results obtained using ${\rm{CTV}}_{1}$, ${\rm{CTV}}_{2}$, $\rm{{\rm{CTV}}_{split}}$, ${\rm{MTV}}$, ${\rm{SVTV}}$, Alg. 2.)

Figure 6: Restoration results at $\sigma=0.01$ when the weight matrix is symmetric. (Listed in order are the observed images and the restoration results obtained using ${\rm{CTV}}_{1}$, ${\rm{CTV}}_{2}$, $\rm{{\rm{CTV}}_{split}}$, ${\rm{MTV}}$, ${\rm{SVTV}}$, Alg. 2.)

TABLE I: The values of PSNR, SSIM, MSE and CIEde2000 for the restored images at symmetric weight matrix.

Observed image (PSNR SSIM) Method PSNR SSIM MSE CIEde2000 Observed image (PSNR SSIM) Method PSNR SSIM MSE CIEde2000 statues (22.6920 0.5002) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 24.7890 24.7416 26.1593 26.9277 27.3174 $\mathbf{27.3190}$ 0.6817 0.6740 0.6983 0.7374 0.7666 $\mathbf{0.7667}$ 0.0033 0.0034 0.0024 0.0020 $\mathbf{0.0019}$ $\mathbf{0.0019}$ 6.3750 6.1065 3.9703 4.0883 3.1070 $\mathbf{3.1066}$ waterfall (21.3554 0.5654) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 22.9317 22.8933 25.2652 26.3011 26.6026 $\mathbf{26.6046}$ 0.7027 0.7097 0.7541 0.8008 0.8132 $\mathbf{0.8133}$ 0.0051 0.0051 0.0030 0.0023 $\mathbf{0.0022}$ $\mathbf{0.0022}$ 7.5840 7.4116 3.7826 3.7302 3.1930 $\mathbf{3.1925}$ aborigine (21.1179 0.5610) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 22.3146 22.29382 26.8057 27.7981 28.0820 $\mathbf{28.0841}$ 0.7152 0.7181 0.7673 0.8107 0.8241 $\mathbf{0.8242}$ 0.0059 0.0059 0.0021 0.0017 $\mathbf{0.0016}$ $\mathbf{0.0016}$ 7.2164 7.0597 3.3089 3.3228 2.7849 $\mathbf{2.7847}$ bear (20.2929 0.5507) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 21.9875 21.9315 23.3130 24.3554 24.5442 $\mathbf{24.5476}$ 0.7154 0.7134 0.7477 0.7907 $\mathbf{0.8063}$ $\mathbf{0.8063}$ 0.0063 0.0064 0.0047 0.0037 $\mathbf{0.0035}$ $\mathbf{0.0035}$ 8.1320 8.0203 4.3142 4.3680 3.7621 $\mathbf{3.7613}$ penguin (21.8428 0.6083) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 24.1892 24.1561 25.3151 26.5423 26.5423 $\mathbf{26.6801}$ 0.7349 0.7389 0.7760 0.8079 0.8166 $\mathbf{0.8167}$ 0.0038 0.0038 0.0029 0.0022 $\mathbf{0.0021}$ $\mathbf{0.0021}$ 5.7218 5.3536 4.5552 4.7076 3.5964 $\mathbf{3.5954}$ butterfly (20.9821 0.6702) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 22.6934 22.6994 26.8852 28.3685 28.4456 $\mathbf{28.4609}$ 0.7702 0.7789 0.8355 0.8672 0.8705 $\mathbf{0.8709}$ 0.0054 0.0054 0.0020 0.0015 $\mathbf{0.0014}$ $\mathbf{0.0014}$ 7.8815 7.7457 3.0992 3.0369 2.6925 $\mathbf{2.6896}$ garden (19.4933 0.5176) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 21.6959 21.6205 23.7840 25.2706 25.2987 $\mathbf{25.3032}$ 0.7188 0.7167 0.7741 0.8354 0.8421 $\mathbf{0.8423}$ 0.0068 0.0069 0.0042 0.0030 0.0030 $\mathbf{0.0029}$ 7.7463 7.5397 4.9869 4.9658 4.3127 $\mathbf{4.3115}$ girl (22.5950 0.6946) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 24.0889 24.0870 28.5861 29.5834 29.7722 $\mathbf{29.7738}$ 0.7844 0.7941 0.8430 0.8634 $\mathbf{0.8709}$ $\mathbf{0.8709}$ 0.0039 0.0039 0.0014 $\mathbf{0.0011}$ $\mathbf{0.0011}$ $\mathbf{0.0011}$ 6.9659 6.7890 2.8742 2.9632 2.4627 $\mathbf{2.4624}$

Figure 7: Corresponding zooming parts is in turn a detail of the original image, restored by the ${\rm{MTV}}$ and Alg. 2 models at symmetric weight matrix.

V-B Asymmetric weight matrix

In this example, we consider the restoration results of each method when the sub-blur kernels are different and the weight matrix is asymmetric. For each method, the degraded images are generated by introducing the following cross-channel blur to the clean images,

$$\left[\begin{array}[]{ccc}(A,5)&(M,11,45)&(M,21,90)\\ (M,11,45),&(A,5)&(G,5,5)\\ (M,21,90)&(G,5,5)&(A,5)\end{array}\right].$$

adding Gaussian noise with standard deviation $\sigma=0.01$. The weight matrix is chosen as follows

$$W=\left[\begin{array}[]{ccc}0.7&0.15&0.15\\ 0.1&0.8&0.1\\ 0.05&0.05&0.9\end{array}\right].$$

In Figures 8 and 9, we display the observed images in cross-channel blurring with asymmetric weight matrix, and the restored images of the seven methods. From the visual, ${\rm{CTV}}_{1}$ and ${\rm{CTV}}_{2}$ are ineffective at removing cross-channel blur and all show the color shift, for example, the texture of the grass skirt in ’aborigine’ is relatively smooth and darker overall. We can observe that in this example the color shift of the images restored by these three methods is smaller than the previous example. This is because a more diagonally dominant weight matrix is used in this example. $\rm{{\rm{CTV}}_{split}}$ shows a significant color shift in this design of asymmetric weight matrix and different sub-blur kernels. The ${\rm{MTV}}$ method seems to lose some of the detailed textures and has an oversmoothing effect in the restored images, though it gives relatively good results visually; see the detailed textures on petals of ’butterfly’ and rock of ’waterfall’. Later on, we will also zoom in locally to specifically compare the differences between the ${\rm{MTV}}$ method and the newly proposed model. As expected, the proposed Alg. 2 achieves better visual results, preserving the geometric textures and color information of the color images.

We then compare the PSNR, SSIM, MSE, and CIEde2000 values of the images restored by these seven methods and show these values in Table II. We evidently observe that the new approach Alg. 2 obtains much higher PSNR, SSIM, MSE, and CIEde2000 values compared to the other methods. Especially for the CIEde2000 criterion, our proposed methods have an average minimum reduction of around 0.64 compared to the other methods.

To further compare ${\rm{MTV}}$ with Alg. 2, we zoom in the local details of the restored images from both methods (the zoomed areas are indicated by red boxes in Figures 8 and 9), as shown in Figure 10. From the comparison of the skin detail in ’aborigine’ and the petal textures in ’butterfly’, we can see that the newly proposed method produces better results in terms of texture features preservation, while the ${\rm{MTV}}$ method tends to oversmooth. Also, Alg. 2 produces superior results in terms of color fidelity judging from the branch edges in ’waterfall’ and the petal edges in ’butterfly’, where the ${\rm{MTV}}$ tends to have color infection in the sharply changing areas of the images. Besides, from the comparison between Tables I and II, it is worth emphasizing that the restoration numerical results of the ${\rm{MTV}}$ method are worse than the corresponding results in Example V-A in terms of indicators such as CIEde2000, while the two newly proposed methods are more stable for the restoration of different blur kernels. This means that ${\rm{MTV}}$ is more sensitive to the design of cross-channel blur kernel, while the newly proposed methods are more general and stable due to the application of the Krylov subspace methods. The above results support the efficiency and superiority of the newly proposed methods.

Figure 8: Restoration results at $\sigma=0.01$ when the weight matrix is asymmetric. (Listed in order are the observed images and the restoration results obtained using ${\rm{CTV}}_{1}$, ${\rm{CTV}}_{2}$, ${\rm{SVTV}}$, $\rm{{\rm{CTV}}_{split}}$, ${\rm{MTV}}$, ${\rm{SVTV}}$, Alg. 2.)

Figure 9: Restoration results at $\sigma=0.01$ when the weight matrix is asymmetric. (Listed in order are the observed images and the restoration results obtained using ${\rm{CTV}}_{1}$, ${\rm{CTV}}_{2}$, ${\rm{SVTV}}$, ${\rm{CTV}}_{split}$, ${\rm{MTV}}$, ${\rm{SVTV}}$, Alg. 2.)

TABLE II: The values of PSNR, SSIM, MSE and CIEde2000 for the restored images at asymmetric weight matrix.

Observed image (PSNR SSIM) Method PSNR SSIM MSE CIEde2000 Observed image (PSNR SSIM) Method PSNR SSIM MSE CIEde2000 statues (22.8211 0.4890) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 24.8674 24.8129 18.6735 26.7807 27.3078 $\mathbf{27.3089}$ 0.6587 0.6510 0.6336 0.7275 $\mathbf{0.7641}$ $\mathbf{0.7641}$ 0.0033 0.0033 0.0136 0.0021 $\mathbf{0.0019}$ $\mathbf{0.0019}$ 5.7584 5.4356 14.4771 4.3106 3.1218 $\mathbf{3.1214}$ waterfall (21.87546 0.5683) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 23.5651 23.5176 16.2687 26.3129 26.8064 $\mathbf{26.8077}$ 0.6927 0.7005 0.6819 0.7962 $\mathbf{0.8161}$ $\mathbf{0.8161}$ 0.0044 0.0044 0.0236 0.0023 $\mathbf{0.0021}$ $\mathbf{0.0021}$ 6.0229 5.8441 15.3867 3.8289 3.1233 $\mathbf{3.1229}$ aborigine (22.3385 0.5643) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 23.8554 23.8243 15.1137 27.7450 28.2340 $\mathbf{28.2353}$ 0.7054 0.7091 0.6704 0.8013 0.8230 $\mathbf{0.8231}$ 0.0041 0.0041 0.0308 0.0017 $\mathbf{0.0015}$ $\mathbf{0.0015}$ 6.0587 5.8796 15.6282 3.4535 2.6851 $\mathbf{2.6849}$ bear (20.4969 0.5420) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 22.1822 22.1152 14.9329 24.5403 24.9258 $\mathbf{24.9279}$ 0.6965 0.6947 0.6858 0.7876 0.8103 $\mathbf{0.8104}$ 0.0060 0.0061 0.0321 0.0035 $\mathbf{0.0032}$ $\mathbf{0.0032}$ 6.6658 6.5229 18.1410 4.5132 3.6703 $\mathbf{3.6697}$ penguin (21.5152 0.5904) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 23.6758 23.6289 16.1198 26.3597 26.6109 $\mathbf{26.6178}$ 0.7131 0.7171 0.7181 0.7989 0.8115 $\mathbf{0.8117}$ 0.0043 0.0043 0.0244 0.0023 $\mathbf{0.0022}$ $\mathbf{0.0022}$ 6.1883 5.7965 20.1197 4.9455 3.6455 $\mathbf{3.6437}$ butterfly (21.5967 0.6596) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 23.6836 23.6712 15.4844 28.2452 28.3127 $\mathbf{28.3145}$ 0.7609 0.7694 0.7548 0.8625 0.8687 $\mathbf{0.8688}$ 0.0043 0.0043 0.0283 $\mathbf{0.0015}$ $\mathbf{0.0015}$ $\mathbf{0.0015}$ 6.1226 5.9530 14.9422 3.1680 2.6089 $\mathbf{2.6085}$ garden (19.4949 0.5044) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 21.5268 21.4425 16.6254 25.0593 25.3852 $\mathbf{25.3918}$ 0.6970 0.6941 0.6942 0.8270 0.8409 $\mathbf{0.8411}$ 0.0070 0.0072 0.0217 0.0031 $\mathbf{0.0029}$ $\mathbf{0.0029}$ 7.5611 7.3168 18.7589 5.2695 4.2331 $\mathbf{4.2308}$ girl (23.0120 0.6945) ${\rm{CTV}}_{1}$ ${\rm{CTV}}_{2}$ $\rm{{\rm{CTV}}_{split}}$ ${\rm{MTV}}$ ${\rm{SVTV}}$ Alg. 2 24.6124 24.6185 16.4629 29.3705 29.7134 $\mathbf{29.7147}$ 0.7778 0.7885 0.7647 0.8552 $\mathbf{0.8666}$ $\mathbf{0.8666}$ 0.0035 0.0035 0.0226 0.0012 $\mathbf{0.0011}$ $\mathbf{0.0011}$ 6.2335 6.0251 18.9103 3.1128 2.4739 $\mathbf{2.4734}$

Figure 10: Corresponding zooming parts is in turn a detail of the original image, restored by the ${\rm{MTV}}$ and Alg. 2 models at asymmetric weight matrix.

VI Conclusion

In this paper, we have proposed a novel total variation model for color image cross-channel deblurring using quaternion matrices to represent color images. The proposed model introduces an SVTV-CTV balance regularization term, jointly exploiting the complementary information of the RGB and HSV color spaces. We also present two different iterative algorithms for the proposed model based on the alternating minimization framework. These two iterative algorithms introduce real and quaternion Krylov subspace methods to solve the linear system in the deblurring process, respectively. The numerical results of color image restoration demonstrates the superior performance of the newly proposed method in terms of PSNR, SSIM, MSE and CIEde2000 criteria compared to the existing total variation methods. Furthermore, we demonstrate the practicality and effectiveness of our newly proposed models and algorithms in addressing medical image artifacts.