Notes on Matrix Decomposition and the Generalized Inverse

This article is also available in 简体中文, 繁體中文.

This paper systematically introduces the equivalent, similarity and contractual relationship between matrices, as well as several important matrix decomposition techniques, including core-power zero decomposition, Hartwig-Spindelböck decomposition and generalized inverse matrix construction methods. This article also discusses some applications of matrix decomposition in the field of engineering, such as solving linear equation systems, least squares problems, singular value decomposition, etc. This article uses some mathematical and natural languages ​​to express the concepts and properties of matrix theory.

Part 1 Classification and invariant of matrix

This section summarizes the equivalence, similarity and contractual relationships between matrices, as well as their invariants. These relationships and invariants play an important role in matrix decomposition and the construction of generalized inverse matrices.

1. $A$, $B$ is equivalent

$A$, $B$ is equivalent $\iff A$ can be transformed into $B$ through elementary row transformation or elementary column transformation.
If there are $m$ order inversion matrix $P$ and $n$ order inversion matrix $Q$, so
$$
A=P\begin{pmatrix}E_r & O \\ O & O\end{pmatrix}Q,
$$
where $r=\operatorname{rank}(A)=\operatorname{rank}(B)$, $E_r$ is the $r$ order unit matrix, and $O$ is the zero matrix. This decomposition formula is called the standard decomposition formula of $A$. Here, the rank is an important invariant of the matrix, which represents the maximum linearly independent row (or column number) of the matrix, that is, the rank of the linear map corresponding to the matrix. The equivalence relationship is reflexive, symmetry and transitive, so all $m \times n$ matrices can be divided into thousands of equivalent classes. The matrices in each equivalent class have the same rank and can all be transformed into the standard shape shown above.

The equivalence relationship has the following properties:

• The equivalence relationship is an equivalence relationship, that is, satisfying reflexivity, symmetry and transmission.
• The equivalence relationship does not change the rank of the matrix, that is, $\operatorname{rank}(A)=\operatorname{rank}(B)$.
• The equivalence relationship can be realized by primary transformation, that is, through finite row transformation or column transformation, one matrix can be turned into another equivalent matrix.
• The equivalence relationship can be used to determine whether two matrices are equal, that is, the sufficiently necessary condition for the two matrices to be equal is that they are equivalent to the same standard shape.

2. $A$, $B$ are similar

$A$ and $B$ are similar to $\iff$. There is a $n$-order invertible matrix $P$, such that $B=P^{-1}AP$.

Or, if they represent the same linearly mapped matrix under different basis. Similarity relationships also have reflexivity, symmetry and transitiveness. Therefore, all nxn matrices can be divided into thousands of similar classes. The matrices in each similar class have the same algebraic invariants such as eigenpolynomials, eigenvalues, traces and determinants. In addition, there is a simplest standard shape in each similar class, called the Jordan standard shape. It is a blocked diagonal matrix composed of several Jordan blocks. Each Jordan block is an upper triangle matrix, and its main diagonal line is the same eigenvalue, and secondly, it is 1 on the diagonal line. Jordan’s standard shape is uniquely determined without caring about the order of Jordan blocks. Additionally, if a matrix can be similarly diagonalized, i.e. similar to a diagonal matrix, it must satisfy its minimum polynomial without a duplicate root.

Similar relationships have the following properties:

• Similarity relationship is an equivalent relationship, that is, satisfying reflexivity, symmetry and transmission.
• The similarity relationship does not change the eigenvalues ​​and eigenpolynomials of the matrix, i.e. $\det(A-\lambda I)=\det(B-\lambda I)$.
• Similarity relationships can be used to determine whether two matrices are the same, i.e. the sufficiently necessary condition for the two matrices to be the same is that they are similar to the same diagonal.
• Similarity relationships can be used to solve linear system of equations or linear differential system of equations, that is, through similar transformation, a complex system of equations can be transformed into a simple system of equations.

3. $A$,$B$ contract

$A$, $B$ contract $\iff$ has a $n$ reversible matrix $P$, such that $B=P^TAP$.

Or, if they represent the matrix of the same quadratic type under different basis. Contract relationships are also reflexive, symmetry and transitive, so all nxn matrices can be divided into several contract categories. For real symmetric matrices, a sufficiently necessary condition for their contract is that they have the same positive inertia index and negative inertia index, which represent the dimensions of the quadratic positive and negative determinant parts corresponding to the real symmetric matrix. In addition, there is a simplest standard shape in each contract class, called the norm shape, which is a diagonal matrix composed of several 1 and -1. The normative form is uniquely determined without considering the order of 1 and -1. Additionally, if a real symmetric matrix is ​​positive or negative or semi-positive or semi-negative, it must satisfy that all its eigenvalues ​​are positive or negative or non-negative or non-positive.

A contractual relationship has the following properties:

• Contractual relationship is an equivalent relationship, that is, satisfying reflexivity, symmetry and transitiveness.
• The contractual relationship does not change the matrix’s rank sum positive inertia index (number of positive eigenvalues), that is, $\operatorname{rank}(A)=\operatorname{rank}(B)$, $\operatorname{p}(A)=\operatorname{p}(B)$.
• The contractual relationship can be used to determine whether two real symmetric matrices are equal, that is, the sufficient and necessary condition for the two real symmetric matrices to be equal is that they contract on the same diagonal shape, and the diagonal elements are arranged from large to small.
• Contractual relationships can be used to solve real quadratic or real quadratic differential equation systems, that is, through contract transformation, a complex quadratic type can be transformed into a standard shape or a normative shape.

Part 2 Matrix Decomposition

1. Core-power zero decomposition: Any $n\times n$ complex matrix $A$ can be uniquely decomposed into
   $$
   A=A_1+A_2,
   $$
   Where $r(\operatorname{rank}(A_1))=r(\operatorname{rank}(A_2))=r(\operatorname{rank}(A))$, $A_1A_2=A_2A_1=O$, $A_2^k=O$, where $k>0$. This decomposition formula is called core-power zero decomposition. It has the following properties:

• $A_1=AAdA$, where $dA=A^+$ is Drazin inverse (generalized inverse).
• $A_2=A-AAdA=(I-A^+A)A=A(I-AA^+)$.
• $d(A_1)=d(A)$, $d(A_2)=0$, where $d(A)$ represents the minimum polynomial (minimum number of degrees and divisible feature polynomial).
• $s(A_1)=s(A)$, $s(A_2)=0$, where $s(A)$ represents the spectral radius (maximum modulus eigenvalue).

Core-power zero decomposition refers to decomposing a $n \times n$ complex matrix $A$ into the sum of two matrices $A_1$ and $A_2$, where $A_1$ is the core part of $A$, that is, the part that satisfies $A_1^2$=$A_1$, and $A_2$ is the power zero part, that is, the part that satisfies $A_2^k=0$, and these two parts are orthogonal to each other, that is, $A_1A_2=A_2A_1=0$. This decomposition can be given by the following theorem:

Assume $A$ is $n \times n$ complex matrix, then $n \times n$ complex matrix $A_1$, $A_2$ satisfies, $A=A1 +A2$ and

• $\operatorname{rank}(A_1^2)=\operatorname{rank}(A_1)$,
• $A_1A_2=A_2A_1=0$
• $A_2$ is a power zero matrix, that is, there is a positive integer $k$, such that $A_2^k=0$.

2. Hartwig-Spindelböck Decomposition: Any $n\times n$ complex matrix $A$ can be uniquely decomposed into

$$
A=U\begin{pmatrix}\Sigma K & \Sigma L \\ O & O \end{pmatrix}U^H,
$$

Where $\Sigma=\operatorname{diag}(\sqrt{\lambda_1},\ldots,\sqrt{\lambda_r})$, $\lambda_1,\ldots,\lambda_r>0$ is a non-zero singular value (non-zero eigenvalue), $K, L, O, U, U^H$ are all unitary matrices (satisfies $UU^H=U^HU=I$, $I$ is a unit matrix), and $KK^H+LL^H=I$ is a unit matrix). This decomposition formula is called Hartwig-Spindelböck decomposition. It has the following properties:

• $K,L,O,U,U^H$ are all compatible with $\Sigma,A,A^H,AA^H,A^HA$ (satisfies the exchange law).
• $\Sigma K,\Sigma L,O,O$ are all incompatible with $\Sigma,A,A^H,AA^H,A^HA$ (does not satisfy the commutational law).
• $\Sigma K,\Sigma L, O, O$ are all compatible with $\Sigma K,\Sigma L, O, O$ (satisfies the commutative law).
• $\Sigma K,\Sigma L, O, O$ are all inaccurate with $\Sigma K,\Sigma L, O, O$ (cannot be further decomposed).

3. Core-EP Decomposition: Decompose a matrix into the sum of a half-single matrix and an invertible matrix, where a half-single matrix refers to a matrix similar to a diagonal matrix.
   The advantage of this decomposition is that the eigenvalues ​​and eigenvectors of the matrix can be separated, so as to facilitate matrix operation and analysis.

4. EP-power zero decomposition: Decompose a matrix into the sum of an invertible matrix and a power zero matrix, where the power zero matrix refers to the existence of a positive integer k such that the matrix has a zero matrix to the k power.
   The advantage of this decomposition is that it can separate the rank and core parts of the matrix, so as to facilitate the study of partial order and generalized inverse of the matrix.

5. Generalized inverse matrix: Any $m\times n$ complex matrix $X$ has a $n\times m$ complex matrix $X^{(k)}$ satisfying the following four conditions:
   $$
   XX^{(k)}X=X, \quad X^{(k)}XX^{(k)}=X^{(k)}, \quad (XX^{(k)})^H=XX^{(k)}, \quad (X^{(k)}X)^H=X^{(k)}X.
   $$

This matrix $X^{(k)}$ is called the generalized inverse matrix of $X$, where $k$ is a non-negative integer representing the rank of $X^{(k)}$. A generalized inverse matrix has the following properties:

• The generalized inverse matrix is ​​unique, that is, if $X^{(k)}$ and $Y^{(k)}$ are both generalized inverse matrices of $X$, then $X^{(k)}=Y^{(k)}$.
• The rank of the generalized inverse matrix is ​​equal to the rank of the original matrix, that is, $\operatorname{rank}(X)=\operatorname{rank}(X^{(k)})=k$.
• The generalized inverse matrix of the generalized inverse matrix is ​​equal to the original matrix, that is, $(X^{(k)})^{(k)}=X$.
• Generalized inverse matrix can be used to solve linear system of equations or linear least squares problems, i.e. if $AX=B$ is a linear system of equations or linear least squares problems, then $A^{(k)}B$ is a solution, where $A^{(k)}$ is a generalized inverse matrix of $A$.
• Generalized inverse matrix can be obtained by Hartwig-Spindelböck decomposition, i.e.
  $$
  A=U\begin{pmatrix}\Sigma K & \Sigma L \\ O & O \end{pmatrix}U^*
  $$
  is the Hartwig-Spindelböck decomposition of $A$,
  $$
  A^{(k)}=U\begin{pmatrix}K\Sigma^{-1} & O \\ O & O \end{pmatrix}U^*
  $$ is a generalized inverse matrix of $A$.

(Note: The $A^H$ appears in the text refers to the conjugated transposition of $A$, that is, $A^H=\bar{A}^T$)