Commutation matrix

In mathematics, especially in linear algebra and matrix theory, the commutation matrix is used for transforming the vectorized form of a matrix into the vectorized form of its transpose. Specifically, the commutation matrix K^(m,n) is the nm × mn matrix which, for any m × n matrix A, transforms vec(A) into vec(A^T):

K^(m,n) vec(A) = vec(A^T) .

Here vec(A) is the mn × 1 column vector obtain by stacking the columns of A on top of one another:

\operatorname {vec} (\mathbf {A} )=[\mathbf {A} _{1,1},\ldots ,\mathbf {A} _{m,1},\mathbf {A} _{1,2},\ldots ,\mathbf {A} _{m,2},\ldots ,\mathbf {A} _{1,n},\ldots ,\mathbf {A} _{m,n}]^{\mathrm {T} }

where A = [A_i,j].

The commutation matrix is a special type of permutation matrix, and is therefore orthogonal. Replacing A with A^T in the definition of the commutation matrix shows that K^(m,n) = (K^(n,m))^T. Therefore in the special case of m = n the commutation matrix is an involution and symmetric.

The main use of the commutation matrix, and the source of its name, is to commute the Kronecker product: for every m × n matrix A and every r × q matrix B,

\mathbf {K} ^{(r,m)}(\mathbf {A} \otimes \mathbf {B} )\mathbf {K} ^{(n,q)}=\mathbf {B} \otimes \mathbf {A} .

It is much used in developing the higher order statistics of Wishart covariance matrices.[1]

An explicit form for the commutation matrix is as follows: if e_r,j denotes the j-th canonical vector of dimension r (i.e. the vector with 1 in the j-th coordinate and 0 elsewhere) then

\mathbf {K} ^{(r,m)}=\sum _{i=1}^{r}\sum _{j=1}^{m}\left(\mathbf {e} _{r,i}{\mathbf {e} _{m,j}}^{\mathrm {T} }\right)\otimes \left(\mathbf {e} _{m,j}{\mathbf {e} _{r,i}}^{\mathrm {T} }\right).

Example

Let M be a 2×2 square matrix.

$\mathbf {M} ={\begin{bmatrix}a&b\\c&d\\\end{bmatrix}}$

Then we have

$\operatorname {vec} (\mathbf {M} )={\begin{bmatrix}a\\c\\b\\d\\\end{bmatrix}}$

And K^(2,2) is the 4×4 square matrix that will transform vec(M) into vec(M^T)

${\begin{bmatrix}1&0&0&0\\0&0&1&0\\0&1&0&0\\0&0&0&1\\\end{bmatrix}}\cdot {\begin{bmatrix}a\\c\\b\\d\\\end{bmatrix}}={\begin{bmatrix}a\\b\\c\\d\\\end{bmatrix}}=\operatorname {vec} (\mathbf {M} ^{T})$

For both square and rectangular matrices of M rows and N columns, the commutation matrix can be generated by this pseudocode, which is similar to an article at Stack Exchange[2] and demonstrably gives the correct result though is presented without proof.

for i = 1 to M
  for j = 1 to N
    K(i + M*(j - 1), j + N*(i - 1)) = 1
  end
end

Thus the following $3\times 2$ matrix $\mathbf {A}$ has two possible vectorizations as follows:

\mathbf {A} ={\begin{bmatrix}1&4\\2&5\\3&6\\\end{bmatrix}},\;\;\;\;V_{1}=\operatorname {vec} (\mathbf {A} )={\begin{bmatrix}1\\2\\3\\4\\5\\6\\\end{bmatrix}},\;\;\;\;\;V_{2}=\operatorname {vec} (\mathbf {A} ^{\mathrm {T} })={\begin{bmatrix}1\\4\\2\\5\\3\\6\\\end{bmatrix}}

and the code above yields

K={\begin{bmatrix}1&\cdot &\cdot &\cdot &\cdot &\cdot \\\cdot &\cdot &1&\cdot &\cdot &\cdot \\\cdot &\cdot &\cdot &\cdot &1&\cdot \\\cdot &1&\cdot &\cdot &\cdot &\cdot \\\cdot &\cdot &\cdot &1&\cdot &\cdot \\\cdot &\cdot &\cdot &\cdot &\cdot &1\\\end{bmatrix}}

giving the expected results

K^{T}K=KK^{T}=\mathbf {I} _{6}

K^{T}\times V_{1}=V_{2}

K\times V_{2}=V_{1}

References

von Rosen, Dietrich (1988). "Moments for the Inverted Wishart Distribution". Scand J Statistics. 15: 97–109.
"Kronecker product and the commutation matrix". Stack Exchange. 2013.

Jan R. Magnus and Heinz Neudecker (1988), Matrix Differential Calculus with Applications in Statistics and Econometrics, Wiley.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] von Rosen, Dietrich (1988). "Moments for the Inverted Wishart Distribution". Scand J Statistics. 15: 97–109.

[2] "Kronecker product and the commutation matrix". Stack Exchange. 2013.