Minimum Volume Constraint with Perturbation for Non-negative Matrix Factorization

Nonnegative matrix factorization (NMF) has been applied in hyperspectral unmixing. The nonconvexity of the NMF’s cost function leads to solutions that are only locally optimal. Adding regularized terms to the NMF helps improve the solutions. In this study, we proposed a regularized NMF model, the regularized tern is the minimum volume constraint with perturbation. The NMF model is solved with multiplicative updated rules. Numerical results verified that adding a disturbance term to the minimum volume constraint effectively improves the spectral curve’s local accuracy while maintaining the original model’s advantages.


Introduction
In nonnegative matrix factorization (NMF), the product of two nonnegative matrices U ∈ R + m×n and V ∈ R + r×n is used to approximate the nonnegative matrix of the original data W ∈ R + m×n , W ≈ UV. ( where r ≪ min(m, n) .Due to the requirement of nonnegativity, the original data can be partially represented.That is, only an additive combination of the sample data is allowed, which makes the NMF results have considerable descriptive capability for the characteristics and structure of the original data, especially the local characteristics.In fact, NMF provides a new approach for hyperspectral unmixing [1].During hyperspectral unmixing using NMF, the nonconvexity of the NMF objective function leads to the enormous influence of the initial points on the algorithm's efficiency and thus the high risk of falling into a local optimum.
In order to improve the NMF results, two types of work have been carried out so far.The first focuses on the initialization design of NMF.The clustering algorithm determines the cluster center or the initial endmember matrix.The initial endmember matrix is the column vector in the original matrix that is most similar to the spectrum vector of the actual surface feature.In this manner, the objective function can rapidly decrease in the initial stage of iteration, and the factorization results of the algorithm are effectively improved [2,3].The second type of work is the NMF model with regular terms.The abundance matrix is sparse, and the sparse NMF model has been applied to hyperspectral unmixing.By adding an orthogonality constraint to the endmember matrix, the linear independence of the column vector of the endmember matrix is improved [4].Some studies have proposed the minimum volume constrained NMF (Minvol) model [5,6].The graph regularized NMF model utilizes the nearest neighbor graph and fully considers the geometric structure of the data space [8] .The above works effectively improved the NMF results.
When NMF is used for the decomposition of mixed pixels, the high similarity between different endmembers has been found to affect the stability of the decomposition results.For example, the spectral curves of trees and grass have a similar spectral curve.If the image contains both trees and grass, there would be a significant deviation between the spectral curves of the two types of surface features and the real data.With a perturbation to the minimum volume constraint, the local accuracy of the spectral curve is obtained.
In this paper, the multiplication update (MU) algorithm was used to obtain the matrix decomposition results.MU algorithm has been used as the reference in the verification of various algorithms [7] .Section 2 describes the two regularization models for the endmember matrix.NMF with the regularized term, which is the minimum volume constraint with perturbation (PMinvol) and its multiplication update formulas.Section 3 shows the numerical results.We found that the endmember spectral curve obtained by the PMinvol model with regularized terms was significantly better than the original Minvol model in a specific band range.

Basic NMF Model
Problem (1) can be expressed as the following optimization model with nonnegative bound constraints: where • 2 is the Frobenius norm.The multiplicative iterative formula is essentially a gradient descent algorithm that, by choosing a proper step size, converts the subtractive updates of the general gradient descent algorithm into multiplicative updates.

Constrained NMF Model
Based on the specific meanings of the factor matrices U and V, corresponding constraints were added to improve the effectiveness of the local solutions obtained by the NMF model.The NMF model with different constraints is usually expressed as follows: .. ≥0,≥0 , = where 1 and 2 are regularization parameters used to control the constraint strength.In this study, we focused on two regularization models for the base (endmember) matrix U, and thus, 2 = 0.

Orthogonal NMF Model.
The orthogonality constraint was added to the endmember matrix: = .The constraint improves the independence of the endmember spectral vector, i.e., the column vector of U. The cost function of the problem is given as follows: For a three-dimensional matrix A, its determinant () represents the volume of a hexahedron spanned by the three column vectors of .Schachtner proposed to use ( ) to describe the volume of the high-dimensional polyhedral spanned by the column vector of .The Minvol model is as follows: .. ≥0,≥0 , = The gradient of problem (4) with respect to U is where is the adjoint matrix of .
The iterative equation of is or Next, a perturbation term is added to the regularized minimum volume constraint, and the corresponding DMinvol model is as follows: where is a given positive number.
The gradient of problem ( 8) with respect to U and V is Because ∇ , ⋅ Φ = 0 in the Karush-Kuhn-Tucker (KKT) condition of problem (8), the multiplication iterative equation of U is obtained, where Φ is the multiplier corresponding to the nonnegative constraint U ≥ 0.
In Equation ( 8), the constraints of V remain unchanged, and its multiplication iteration equation is also unchanged, and the iteration equation of U is as follows: The algorithm for solving problem ( 8) is given below.

Numerical Test
The data were downloaded from http://lesun.weebly.com/hyper-spectral-data-set.html.The Urban, Samson, and Jasper Ridge hyperspectral datasets were used in the numerical experiment.After removing the bands with a low signal-to-noise ratio and the water vapor absorption, the original matrices W of the three images were 162×94249, 198×10000, and 198×9025, respectively.In the numerical experiment, the spectral angle distance (SAD), i.e., the angle between two spectral vectors, was used to calculate the effectiveness of the decomposed endmember spectra.The calculation formula is as follows: where is the decomposed spectral vector, and is the real spectral vector.A smaller angle calculated by Equation (10) indicates that the spectrum is closer to the reference spectrum.
In this section, the results of spectral unmixing using Minvol, PMinvol, the orthogonality constrained NMF (ONMF), and the NMF model without a regular term (MU) were compared.The +real spectral vectors when the number of endmembers was 4, 5, and 6 are given for the standard Urban area test dataset.Tables 1, 2 and 3 show that the SAD value of the decomposition result was effectively improved after the regularized terms was added to the NMF.In the 5-endmember data of the Urban area, the SAD values of the road and bare soil were small (0.1216).
Figures 1 and 2 show the estimated spectral curves obtained by DMinvol and Minvol, respectively.The parameter of the disturbance term was δ=1.In the 80-120 band in Figure 1 and the 90-140 band in Figure 2, the spectral curve obtained by DMinvol was more consistent with the real spectral curve.

Conclusions
In the present study, a DMinvol algorithm was proposed.The numerical results showed that the SAD values of the endmember spectral vectors obtained by the Minvol and DMinvol were not much different from that of the real vector, and the endmember spectral vector obtained by DMinvol was closer to the real data.

Figure 1 .
Figure 1.Comparison of the real road spectral curve with the estimated spectral curves obtained by Minvol and Pminvol.

Figure 2 .
Figure 2. Comparison of the real bare soil spectral curve with the spectral curves estimated by Minvol and Pminvol.

Table 1 .
Comparison of 4-endmember SAD k values in the Urban area.

Table 2 .
Comparison of 5-endmember SAD k values in the Urban area.

Table 3 .
Comparison of 4-endmember SAD k values in the Jasper Ridge area.