10.2 Matrix-Column-Vector Multiplication via Dot Products

#Type/Definition #Topic/Linear_Algebra

Types: N/A
Examples: 10.2.1 Example of Matrix-Column-Vector Multiplication Using Dot Products
Constructions: N/A
Generalizations: N/A

Properties: N/A
Sufficiencies: N/A
Questions: N/A

Matrix-Column-Vector Multiplication via Dot Products

Let matrix $A \in R^{m \times n}$ and 3.1 Column Vector $\vec{x} \in R^{n \times 1}$ . We define matrix-column-vector product $A \cdot \vec{x} = \vec{b} \in R^{m \times 1}$ with 5.1 Inner Product Between Vectors using an entry-by-entry approach. The $i$ th entry of the product $\vec{b} = A \cdot \vec{x}$ can be calculated via dot product between the $i$ th row of $A$ and the vector $\vec{x}$ .

b_{i} = {entry}_{(i, 1)} (A \cdot \vec{x}) = (A (i, :))^{T} \cdot \vec{x} = \sum_{k = 1}^{n} a_{i k} x_{k}

for row index $i \in {1, 2, \dots, m}$ . When we calculate each entry of the output vector, we produce:

A \vec{x} = [\begin{matrix} a_{11} x_{1} + a_{12} x_{2} + \dots + a_{1 n} x_{n} \\ a_{21} x_{1} + a_{22} x_{2} + \dots + a_{2 n} x_{n} \\ ⋮ \\ a_{m 1} x_{1} + a_{m 2} x_{2} + \dots + a_{m n} x_{n} \end{matrix}] = [\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{m} \end{matrix}]

! The inner dimensions must agree!

Remark. To analyze the dot product definition of matrix-column-vector multiplication, let's cut up our matrix $A$ into row partitions and output vector $\vec{b}$ into individual scalars.

[A]_{m \times n} \cdot [\vec{n}]_{n \times 1} = {[\begin{matrix} A (1, :)_{1 \times n} \\ A (2, :)_{1 \times n} \\ ⋮ \\ A (m, :)_{1 \times n} \end{matrix}]}_{m \times n} \cdot [\vec{x}]_{n \times 1} = {[\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{m} \end{matrix}]}_{m \times 1} = [\vec{b}]_{m \times 1}

Let's take a look at how the individual entries are computed.

& Recall that the inner product requires the dimensions of the operands to be the same. This is why we have to transpose the row partitions of $A$ .

\begin{aligned} ⟹ b_{1} & = {Entry}_{(1, 1)} (\vec{b}) \\ = \underset{first row of A}{\underset{⏟}{[A (1, :)]_{n \times 1}^{\overset{convert row to column}{\overset{⏞}{T}}}}} \cdot \underset{column vector}{\underset{⏟}{{\vec{x}}_{n \times 1}}} \\ = [\begin{array}{c} a_{11} \\ a_{12} \\ ⋮ \\ a_{1 n} \end{array}] \cdot [\begin{array}{c} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{array}] \\ = a_{11} x_{1} + a_{12} x_{2} + \dots + a_{1 n} x_{n} \\ = \sum_{k = 1}^{n} a_{i k} x_{k} \end{aligned}

We use $k$ for the sum because when working with row partitions, we are iterating across the columns. This process would continue for the rest of the entries $b_{2}, b_{3}, \dots, b_{m}$ .
$⧫$

Remark. Let's connect our dot product definition with our 10.1 Matrix-Column-Vector Multiplication via Linear Combinations definition. Using commutativity, we can rearrange our variables to shift our perspective.

\begin{aligned} [\begin{array}{c} b_{1} \\ b_{2} \\ ⋮ \\ b_{m} \end{array}] & = [\begin{array}{c} a_{11} x_{1} + a_{12} x_{2} + \dots + a_{1 n} x_{n} \\ a_{21} x_{1} + a_{22} x_{2} + \dots + a_{2 n} x_{n} \\ ⋮ \\ a_{m 1} x_{1} + a_{m 2} x_{2} + \dots + a_{m n} x_{n} \end{array}] \\ = [\begin{array}{c} x_{1} a_{11} + x_{2} a_{12} + \dots + x_{n} a_{1 n} \\ x_{1} a_{21} + x_{2} a_{22} + \dots + x_{n} a_{2 n} \\ ⋮ \\ x_{1} a_{m 1} + x_{2} a_{m 2} + \dots + x_{n} a_{m n} \end{array}] \\ = [\begin{array}{c} x_{1} a_{11} \\ x_{1} a_{21} \\ ⋮ \\ x_{1} a_{m 1} \end{array}] + [\begin{array}{c} x_{2} a_{12} \\ x_{2} a_{22} \\ ⋮ \\ x_{2} a_{m 2} \end{array}] + \dots + [\begin{array}{c} x_{n} a_{1 n} \\ x_{n} a_{2 n} \\ ⋮ \\ x_{n} a_{m n} \end{array}] \\ = x_{1} [\begin{array}{c} a_{11} \\ a_{21} \\ ⋮ \\ a_{m 1} \end{array}] + x_{2} [\begin{array}{c} a_{12} \\ a_{22} \\ ⋮ \\ a_{m 2} \end{array}] + \dots + x_{n} [\begin{array}{c} a_{1 n} \\ a_{2 n} \\ ⋮ \\ a_{m n} \end{array}] & Linear combination \end{aligned}

The outputs are identical, but the operational units of each version are different.

Operational unit of dot product version is a scalar.
Operational unit of linear combination version is a column partition.

$⧫$