Positive Definite

Relationships among full-rank, positive definite and non-negative definite are clearlty summarized below.

level 0

Suppose A is a (square) nxn matrix,

• non-singular $\Leftrightarrow$ rank( A )=n $\Leftrightarrow$ $det( \bf{A} ) \neq 0$ $\Leftrightarrow$ $\bf{A^{-1}}$ exist

• determinant $det( \bf{A} )$ can be negative.

• singular $\Leftrightarrow$ $rank(\bf{A})\neq n$ $\Leftrightarrow$ $det( \bf{A} ) = 0$ $\Leftrightarrow$ $\bf{A^{-1}}$ fail to exist

• If A is non-negative, then $det(\bf{A}) \ge 0$

  • A may be singular or non-singular.

• For a non-negative definite A

• i.e. non-negative + non-singular = positive

level 1

Suppose A is a nxn symmetric matrix, (positive definite matrix is always symmetric)

• By the Fundamental Theorem of Algebra, A has n eigenvalues.

• rank( A ) is equal to the number of non-zero eigenvalues of A.

• A is non-negative definite $\Leftrightarrow$ all eigenvalues of A are non-negative.

• A is positive definite $\Leftrightarrow$ all eigenvalues of A are positive.

• positive definite $\Rightarrow$ full rank

level 2

• It is worth noting that for a real non-negative definite square matrix (a covariance matrix), the singular value decomposition is the eigen decomposition, U = V, columns of these matrices are unit eigenvectors, and the SVD singular values are the corresponding eigenvalues.

  • covariance matrix is non-negative definite.(stat542 P171)

  • If A is non-negative, there exist symmetrix matrix B such that BB=A. This is important for AITKEN model.

application

1. $X^{T}X$ is always semi-positive definite, If $X$ is full rank, $X^{T}X$ is positive definite.

$a'X'Xa=(Xa)'(Xa)=\sum{k^2} >= 0.$

$X$ is full column rank, $Xa$ is the linear combination of columns of X, it won't be $0$. Thus $a'X'Xa >= 0$