11.2.1 Example of Matrix-Matrix Multiplication via Linear Combination of Columns

Types: N/A
Examples: N/A
Constructions: N/A
Generalizations: N/A

Properties: N/A
Sufficiencies: N/A
Questions: N/A

Matrix-Matrix Multiplication via Linear Combination of Columns

Let $A \in R^{3 \times 2}$ and $X \in R^{2 \times 3}$ . Find the matrix-matrix product $B = A \cdot X$ using 11.2 Matrix-Matrix Multiplication via Linear Combination of Columns.

A = {[\begin{matrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{matrix}]}_{3 \times 2}, X = {[\begin{matrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{matrix}]}_{2 \times 3}

Solution. The inner dimensions are both equal to $2$ , so we can do matrix-matrix multiplication. The outer dimensions are both $3$ , so our output will be a $3 \times 3$ matrix. Through our linear combination of columns approach, we can find the resulting matrix through a column-by-column approach.

Let's begin with column 1. Let's start by creating a general plan for what we are going to compute.

\begin{aligned} B (:, 1) & = A \cdot X (:, 1) \\ = {[\begin{array}{c} a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32} \end{array}]}_{2 \times 3} \cdot {[\begin{array}{c} x_{11} \\ x_{21} \end{array}]}_{2 \times 1} & Matrix-vector product \\ = x_{11} [\begin{array}{c} a_{11} \\ a_{21} \\ a_{31} \end{array}] + x_{21} [\begin{array}{c} a_{12} \\ a_{22} \\ a_{32} \end{array}] & Linear combination \\ = [\begin{array}{c} x_{11} \cdot a_{11} \\ x_{11} \cdot a_{21} \\ x_{11} \cdot a_{31} \end{array}] + [\begin{array}{c} x_{21} \cdot a_{12} \\ x_{21} \cdot a_{22} \\ x_{21} \cdot a_{32} \end{array}] & Scalar-vector multiplication \\ ⟹ B (:, 1) & = [\begin{array}{c} x_{11} a_{11} + x_{21} a_{12} \\ x_{11} a_{21} + x_{21} a_{22} \\ x_{11} a_{31} + x_{21} a_{32} \end{array}] & Vector-vector addition \end{aligned}

Now that we have this in mind, let's plug in our values.

\begin{aligned} ⟹ B (:, 1) & = A \cdot X (:, 1) \\ = {[\begin{array}{c} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array}]}_{3 \times 2} \cdot {[\begin{array}{c} 1 \\ 4 \end{array}]}_{2 \times 1} \\ = 1 {[\begin{array}{c} 1 \\ 3 \\ 5 \end{array}]}_{3 \times 1} + 4 {[\begin{array}{c} 2 \\ 4 \\ 6 \end{array}]}_{3 \times 1} \\ = {[\begin{array}{c} 1 \\ 3 \\ 5 \end{array}]}_{3 \times 1} + {[\begin{array}{c} 8 \\ 16 \\ 24 \end{array}]}_{3 \times 1} \\ B (:, 1) & = {[\begin{array}{c} 9 \\ 19 \\ 29 \end{array}]}_{3 \times 1} \end{aligned}

We continue this process for our column 2

\begin{aligned} ⟹ B (:, 2) & = A \cdot X (:, 2) \\ = {[\begin{array}{c} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array}]}_{3 \times 2} \cdot {[\begin{array}{c} 2 \\ 5 \end{array}]}_{2 \times 1} \\ = 2 {[\begin{array}{c} 1 \\ 3 \\ 5 \end{array}]}_{3 \times 1} + 5 {[\begin{array}{c} 2 \\ 4 \\ 6 \end{array}]}_{3 \times 1} \\ = {[\begin{array}{c} 2 \\ 6 \\ 10 \end{array}]}_{3 \times 1} + {[\begin{array}{c} 10 \\ 20 \\ 30 \end{array}]}_{3 \times 1} \\ B (:, 2) & = {[\begin{array}{c} 12 \\ 26 \\ 40 \end{array}]}_{3 \times 1} \end{aligned}

and column 3

\begin{aligned} ⟹ B (:, 3) & = A \cdot X (:, 3) \\ = {[\begin{array}{c} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array}]}_{3 \times 2} \cdot {[\begin{array}{c} 3 \\ 6 \end{array}]}_{2 \times 1} \\ = 3 {[\begin{array}{c} 1 \\ 3 \\ 5 \end{array}]}_{3 \times 1} + 6 {[\begin{array}{c} 2 \\ 4 \\ 6 \end{array}]}_{3 \times 1} \\ = {[\begin{array}{c} 3 \\ 9 \\ 15 \end{array}]}_{3 \times 1} + {[\begin{array}{c} 12 \\ 24 \\ 36 \end{array}]}_{3 \times 1} \\ B (:, 3) & = {[\begin{array}{c} 15 \\ 33 \\ 51 \end{array}]}_{3 \times 1} \end{aligned}

Putting our columns 1, 2, and 3 together, we get our final matrix-matrix product

\begin{aligned} B & = A \cdot X \\ = {[\begin{array}{c} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array}]}_{3 \times 2} {[\begin{array}{c} 1 & 2 & 3 \\ 4 & 5 & 6 \end{array}]}_{2 \times 2} \\ ⟹ B & = {[\begin{array}{c} 9 & 12 & 15 \\ 19 & 26 & 33 \\ 29 & 40 & 51 \end{array}]}_{3 \times 3} \end{aligned}

Remark. We can check our answer using Octave with the following code:

A = [1, 2; 3, 4; 5, 6]
X = [1, 2, 3; 4, 5, 6]
B = A * X

$⧫$

Modify a single column of a matrix

Let a $4 \times 3$ modeling matrix $A$ be

[A]_{m \times n} = {[\begin{matrix} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 1 \\ 0 & 0 & - 1 \end{matrix}]}_{4 \times 3}

Suppose we wish to double column 1 of $A$ and leave the other columns untouched. In other words, we want to multiply $A (:, 1)$ by 2 and leave the other columns untouched.

Solution. Recall 11.1 Anatomy of Matrix-Matrix Multiplication. To do algebraic work on the columns of our modeling matrix, we can multiply on the right.

What type of matrix would allow us to leave all other columns untouched?

To double column 1 and leave all other columns untouched, we multiply on the right by a 9.9 Dilation Matrix.

Dilation Matrix

Let natural number $n \in N$ and scalar $c \in R$ . For $j \in {1, 2, \dots, n}$ , we define an $n \times n$ dilation matrix
$D_{j} (c) = I_{n} + (c - 1) \cdot {\vec{e}}_{j} \cdot {\vec{e}}_{j}^{T}$

Recall that a dilation matrix modifies the $j$ th value of the 8.6 Identity Matrix.

Since we are trying to double column 1 of our modeling matrix $A$ , we set $j = 1$ and $c = 2$ for our dilation matrix.

This is how our current setup would be written:

\underset{modeling matrix}{[A]_{4 \times 3}} \cdot \underset{algebraic worker}{[D_{1} (2)]_{3 \times 3}} = {[\begin{matrix} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 1 \\ 0 & 0 & - 1 \end{matrix}]}_{4 \times 3} {[\begin{matrix} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}]}_{3 \times 3} = [B]_{4 \times 3}

We use an $n = 3$ dilation matrix because we want the output matrix to have to same dimensions as the input (recall that the outer dimensions define the dimensions of $B$ )

Let's compute our matrix-matrix product column-by-column.

\begin{aligned} [B (:, 1)]_{4 \times 1} & = {Column}_{1} (A \cdot D_{1} (2)) \\ = [A]_{4 \times 3} \cdot [{Column}_{1} (D_{1} (2))]_{3 \times 1} \\ = {[\begin{array}{c} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 1 \\ 0 & 0 & - 1 \end{array}]}_{4 \times 3} \cdot {[\begin{array}{c} 2 \\ 0 \\ 0 \end{array}]}_{3 \times 1} \\ = 2 [\begin{array}{c} 1 \\ - 1 \\ 0 \\ 0 \end{array}] + 0 [\begin{array}{c} 0 \\ 1 \\ - 1 \\ 0 \end{array}] + 0 [\begin{array}{c} 0 \\ 0 \\ 1 \\ - 1 \end{array}] \\ = [\begin{array}{c} 2 \\ - 2 \\ 0 \\ 0 \end{array}] + [\begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \end{array}] + [\begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \end{array}] \\ ⟹ [B (:, 1)]_{4 \times 1} & = [\begin{array}{c} 2 \\ - 2 \\ 0 \\ 0 \end{array}] \end{aligned}

Let's continue with column 2.

\begin{aligned} [B (:, 2)]_{4 \times 1} & = {Column}_{2} (A \cdot D_{1} (2)) \\ = [A]_{4 \times 3} \cdot [{Column}_{2} (D_{1} (2))]_{3 \times 1} \\ = {[\begin{array}{c} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 1 \\ 0 & 0 & - 1 \end{array}]}_{4 \times 3} \cdot {[\begin{array}{c} 0 \\ 1 \\ 0 \end{array}]}_{3 \times 1} \\ = 0 [\begin{array}{c} 1 \\ - 1 \\ 0 \\ 0 \end{array}] + 1 [\begin{array}{c} 0 \\ 1 \\ - 1 \\ 0 \end{array}] + 0 [\begin{array}{c} 0 \\ 0 \\ 1 \\ - 1 \end{array}] \\ ⟹ [B (:, 2)]_{4 \times 1} & = [\begin{array}{c} 0 \\ 1 \\ - 1 \\ 0 \end{array}] \end{aligned}

Again, we continue with column 3.

\begin{aligned} [B (:, 3)]_{4 \times 1} & = {Column}_{3} (A \cdot D_{1} (2)) \\ = [A]_{4 \times 3} \cdot [{Column}_{3} (D_{1} (2))] \\ = {[\begin{array}{c} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 1 \\ 0 & 0 & - 1 \end{array}]}_{4 \times 3} \cdot {[\begin{array}{c} 0 \\ 0 \\ 1 \end{array}]}_{3 \times 1} \\ = 0 [\begin{array}{c} 1 \\ - 1 \\ 0 \\ 0 \end{array}] + 0 [\begin{array}{c} 0 \\ 1 \\ - 1 \\ 0 \end{array}] + 1 [\begin{array}{c} 0 \\ 0 \\ 1 \\ - 1 \end{array}] \\ ⟹ [B (:, 3)]_{4 \times 1} & = [\begin{array}{c} 0 \\ 0 \\ 1 \\ - 1 \end{array}] \end{aligned}

Finally, we can combine our results to form our output.

\begin{aligned} [A]_{4 \times 5} \cdot [D_{1} (2)] & = [B]_{4 \times 3} \\ {[\begin{array}{c} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & - 1 \end{array}]}_{4 \times 3} {[\begin{array}{c} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}]}_{3 \times 3} & = {[\begin{array}{c} 2 & 0 & 0 \\ - 2 & 1 & 0 \\ 0 & - 1 & 0 \\ 0 & 0 & - 1 \end{array}]}_{4 \times 3} \end{aligned}

Based on this example, we can generalize a pattern for scaling the $k$ th column of matrix $A \in R^{m \times n}$ by the number $c$ . To scale the $k$ th column of $A$ , we multiply $A$ on the right by the $n \times n$ dilation matrix $D_{k} (c)$ .