6.4 The Covariance Matrix

The terms of the covariance matrix $V_{ij}$ are defined by:

$$V_{ij} = \left< \left(u_i - u^0_i\right) \left(u_j - u^0_j\right) \right>$$ (6.8)

Its leading diagonal terms may be recognised as equalling the variances of each of our $n_\mathrm{u}$ variables; its cross terms measure the correlation between the variables. If a component $V_{ij} > 0$, it implies that higher estimates of the coefficient $u_i$ make higher estimates of $u_j$ more favourable also; if $V_{ij} < 0$, the converse is true.

It is a standard statistical result that $\mathbf{V} = (-\mathbf{A})^{-1}$. In the remainder of this section we prove this; readers who are willing to accept this may skip onto section 6.5.

Using $\Delta u_i$ to denote $\left(u_i - u^0_i\right)$, we may proceed by rewriting equation (6.8) as:

$$V_{ij} = \idotsint_{u_i=-\infty}^{\infty} \Delta u_i \Delta u_j \mathrm{P}... ...thbf{x}_i, f_i, \sigma_i \right\} \right) \mathrm{d}^{n_\mathrm{u}}\mathbf{u}$$ (6.9)

$$= \frac{ \idotsint_{u_i=-\infty}^{\infty} \Delta u_i \Delta u_j \ex... ...dotsint_{u_i=-\infty}^{\infty} \exp(-Q) \mathrm{d}^{n_\mathrm{u}}\mathbf{u} }$$

The normalisation factor in the denominator of this expression, which we denote as $Z$, the partition function, may be evaluated by $n_\mathrm{u}$-dimensional Gaussian integration, and is a standard result:

$$Z = \idotsint_{u_i=-\infty}^{\infty} \exp\left(\frac{1}{2} \Delta \ma... ...f{T} \mathbf{A} \Delta \mathbf{u} \right) \mathrm{d}^{n_\mathrm{u}}\mathbf{u}$$ (6.10)

$$= \frac{(2\pi)^{n_\mathrm{u}/2}}{\mathrm{Det}(\mathbf{-A})}$$

Differentiating $\log_e(Z)$ with respect of any given component of the Hessian matrix $A_{ij}$ yields:

$$-2 \frac{\partial}{\partial A_{ij}} \left[ \log_e(Z) \right] = \f... ...}^{\infty} \Delta u_i \Delta u_j \exp(-Q) \mathrm{d}^{n_\mathrm{u}}\mathbf{u}$$ (6.11)

which we may identify as equalling $V_{ij}$:

$$V_{ij} = -2 \frac{\partial}{\partial A_{ij}} \left[ \log_e(Z) \right]$$ (6.12)

$$= -2 \frac{\partial}{\partial A_{ij}} \left[ \log_e((2\pi)^{n_\mathrm{u}/2}) - \log_e(\mathrm{Det}(\mathbf{-A})) \right]$$

$$= 2 \frac{\partial}{\partial A_{ij}} \left[ \log_e(\mathrm{Det}(\mathbf{-A})) \right]$$

This expression may be simplified by recalling that the determinant of a matrix is equal to the scalar product of any of its rows with its cofactors, yielding the result:

$$\frac{\partial}{\partial A_{ij}} \left[\mathrm{Det}(\mathbf{-A})\right] = -a_{ij}$$ (6.13)

where $a_{ij}$ is the cofactor of $A_{ij}$. Substituting this into equation (6.12) yields:

$$V_{ij} = \frac{-a_{ij}}{\mathrm{Det}(\mathbf{-A})}$$ (6.14)

Recalling that the adjoint $\mathbf{A}^\dagger$ of the Hessian matrix is the matrix of cofactors of its transpose, and that $\mathbf{A}$ is symmetric, we may write:

$$V_{ij} = \frac{-\mathbf{A}^\dagger}{\mathrm{Det}(\mathbf{-A})} \equiv (-\mathbf{A})^{-1}$$ (6.15)

which proves the result stated earlier.