Quiz

1. Differentiate \(f(\mathbf{x})=\mathbf{x}^{T}\mathbf{A}\mathbf{x}\)

Consider \(f(\mathbf{x})=\mathbf{x}^{T}\mathbf{A}\mathbf{x}\), where \(\mathbf{x} \in \mathbb{R}^{n}\) and \(\mathbf{A} \in \mathbb{R}^{n \times n}\). Show that for a symmetric matrix \(\mathbf{A}^{T} =\mathbf{A}\), the gradient of \(f(\mathbf{x})\) is given as \(2\mathbf{A} \mathbf{x}\).

Solution

Let \(\mathbf{x}^{n\times 1}=(x_1,\dots ,x_n)'\) be a vector, the derivative of \(\mathbf y=f(\mathbf x)\) with respect to the vector \(\mathbf{x}\) is defined by

\[\begin{split}\frac{\partial f}{\partial \mathbf x}=\begin{pmatrix} \frac{\partial f}{\partial x_1} \\ \vdots\\ \frac{\partial f}{\partial x_n} \end{pmatrix}\end{split}\]

Let

\[\begin{split}\begin{align} \mathbf y&=f(\mathbf x)\\&=\mathbf x'A\mathbf x \\&=\sum_{i=1}^n\sum_{j=1}^n a_{ij}x_ix_j\\&=\sum_{i=1}^na_{i1}x_ix_1+\sum_{j=1}^na_{1j}x_1x_j+\sum_{i=2}^n\sum_{j=2}^n a_{ij}x_ix_j \\\frac{\partial f}{\partial x_1} &=\sum_{i=1}^na_{i1}x_i+\sum_{j=1}^na_{1j}x_j\\&=\sum_{i=1}^na_{1i}x_i+\sum_{i=1}^na_{1i}x_i \,[\text{since}\,\, a_{1i}=a_{1i}]\\ &=2 \sum_{i=1}^na_{1i}x_i \\ \frac{\partial f}{\partial \mathbf x}&=\begin{pmatrix} 2 \sum_{i=1}^na_{1i}x_i \\ \vdots\\ 2 \sum_{i=1}^na_{ni}x_i \end{pmatrix} \\&=2\begin{pmatrix} a_{11} & a_{12} & \dots & a_{1n}\\ \vdots & \vdots &\ddots & \vdots \\ a_{11} & a_{12} & \dots & a_{1n} \end{pmatrix}\begin{pmatrix}x_1 \\ \vdots \\ x_n \end{pmatrix}\\ &= 2A\mathbf x \end{align}\end{split}\]