Tricks for matrix derivation

this document from link 学生古
The trace of a square matrix is the sum of the elements on the main diagonal. That is, for an n by n square matrix A, the trace of A is

This might not seem too exciting at first. However, the trace operator has a neat quasi-commutative property: for matrices U and V, so long as the internal dimensions work out, it is true that

The proof isn’t too hard so I’ll skip it. If we had a third matrix W (again assuming the internal dimensions work out), since matrix multiplication is associative, it is also true that

It’s not truly commutative, since you can only do cyclic shifts of the arguments. So, e.g., tr(UVW) is not equal to tr(WVU) in general.

What can you do with this? For one thing, note that the trace of a scalar a is itself: tr(a) = a. So if you have a matrix multiplication that results in a scalar, you can use trace to rearrange the arguments.

For instance, let U be a 1 by n row vector, and let V be an n by n matrix. If U’ is the transpose of U, then UVU’ is a scalar. This kind of expression comes up pretty often in jointly Gaussian distributions.

Now say U is a zero-mean vector with covariance matrix E[U’U], and I want to know E[UVU’].Using the trace trick, I can express this expectation in terms of E[U’U]: first, we can write

and since expectation distributes over the trace sum, we have

As a result, if you know the covariance E[U’U], there’s no need to recalculate any expectations.

原文（翻*墙）http://andreweckford.blogspot.com/2009/09/trace-tricks.html

用到该trace trick的有：
1）Michael I. Jordan： Jeffreys priors http://www.cs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture6.pdf
2）Kevin P. Murphy： Conjugate Bayesian analysis of the Gaussian distribution http://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf
3）Leon Gu： Multivariate Gaussian Distribution http://www.cs.cmu.edu/~epxing/Class/10701-08s/recitation/gaussian.pdf
4）Tom SF Haines： Gaussian Conjugate Prior Cheat Sheet http://thaines.com/content/misc/gaussian_conjugate_prior_cheat_sheet.pdf

LaTex support
formulation $y_t w_{t+1} \dot x_t = y_t w_t \dot x_t + \eta ||x_t||^2$

Written with StackEdit.