11.4 Matrix-Matrix Multiplication via Dot Products

#Type/Definition #Topic/Linear_Algebra

Types: N/A
Examples: 11.4.1 Example of Matrix-Matrix Multiplication via Dot Products
Constructions: N/A
Generalizations: N/A

Properties: 11.6 Algebraic Properties of Matrix-Matrix Multiplication
Sufficiencies: N/A
Questions: N/A

Matrix-Matrix Multiplication via Dot Products

Let matrices $A \in R^{m \times p}$ and $X \in R^{p \times n}$ . The product $B = A \cdot X$ is the $m \times n$ matrix whose value in the $i$ th row and $k$ th entry is given by

b_{i k} = {Row}_{i} (A) \cdot {Column}_{k} (X) = A (i, :) \cdot X (:, k)

for $i \in {1, 2, \dots, m}$ and $k \in {1, 2, \dots, n}$ .

Remark. When we want to find the individual entries of a matrix-matrix multiplication product, the dot product definition is a good tool for this. We are chopping the left matrix into rows and the right matrix into columns.

\begin{aligned} B & = A X \\ = {[\begin{array}{c} A (1, :) \\ A (2, :) \\ ⋮ \\ a (m, :) \end{array}]}_{m \times p} {[\begin{array}{c} X (:, 1) & X (:, 2) & \dots & X (:, n) \end{array}]}_{p \times n} \\ = {[\begin{array}{c} b_{11} & b_{12} & \dots & b_{1 n} \\ b_{21} & b_{22} & \dots & b_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ b_{m 1} & b_{m 2} & \dots & b_{m n} \end{array}]}_{m \times n} \end{aligned}

$⧫$

Remark. Let's take a look at the individual entries on the left and right side of this equation.

\begin{aligned} b_{11} & = [A (1, :)]_{1 \times p} [X (:, 1)]_{p \times 1} \\ = {[\begin{array}{c} a_{11} & a_{12} & \dots & a_{1 p} \end{array}]}_{1 \times p} {[\begin{array}{c} x_{11} \\ x_{21} \\ ⋮ \\ x_{p 1} \end{array}]}_{p \times 1} \\ = x_{11} a_{11} + x_{21} a_{12} + \dots + x_{p 1} a_{1 p} \\ = a_{11} x_{11} + a_{12} x_{21} + \dots + a_{1 p} x_{p 1} \\ = \sum_{j = 1}^{p} a_{1 j} x_{j 1} \\ = [{Row}_{1} (A)]_{1 \times p} \cdot [{Column}_{1} (x)]_{p \times 1} \\ = [{Column}_{1} (A^{T})]_{p \times 1} \cdot [{Column}_{1} (x)]_{p \times 1} \end{aligned}

Let's look at the next entry of $b$ now.

\begin{aligned} b_{21} & = [A (2, :)]_{1 \times p} [X (:, 1)]_{p \times 1} \\ = {[\begin{array}{c} a_{21} & a_{22} & \dots & a_{2 p} \end{array}]}_{1 \times p} {[\begin{array}{c} x_{11} \\ x_{21} \\ ⋮ \\ x_{p 1} \end{array}]}_{p \times 1} \\ = x_{11} a_{21} + x_{21} a_{22} + \dots + x_{p 1} a_{2 p} \\ = a_{21} x_{11} + a_{22} x_{21} + \dots + a_{2 p} x_{p 1} \\ = \sum_{j = 1}^{p} a_{2 j} x_{j 1} \\ = [{Row}_{2} (A)]_{1 \times p} \cdot [{Column}_{1} (x)]_{p \times 1} \\ = [{Column}_{2} (A^{T})]_{p \times 1} \cdot [{Column}_{1} (x)]_{p \times 1} \end{aligned}

& Note that our dimensions are not matching but we are still able to do dot product. This is because we are using a version of dot product for matrix-matrix multiplication, which is different from 5.1 Inner Product Between Vectors.

Can we guess a pattern? Let's try to construct our general form.

\begin{aligned} b_{i k} & = [A (i, :)]_{1 \times p} [X (:, k)]_{p \times 1} \\ = {[\begin{array}{c} a_{i 1} & a_{i 2} & \dots & a_{i p} \end{array}]}_{1 \times p} {[\begin{array}{c} x_{1 k} \\ x_{2 k} \\ ⋮ \\ x_{p k} \end{array}]}_{p \times 1} \\ = x_{1 k} a_{i 1} + x_{2 k} a_{i 2} + \dots + x_{p k} a_{i p} \\ = a_{i 1} x_{1 k} + a_{i 2} x_{2 k} + \dots + a_{i p} x_{p k} \\ = \sum_{j = 1}^{p} a_{i j} x_{j k} \\ = [{Row}_{i} (A)]_{1 \times p} [{Column}_{k} (X)]_{p \times 1} \\ = [{Column}_{i} (A^{T})]_{p \times 1} [{Column}_{k} (X)]_{p \times 1} \end{aligned}

$⧫$