10.1 Matrix-Column-Vector Multiplication via Linear Combinations

#Type/Definition #Topic/Linear_Algebra

Types: N/A
Examples: 10.1.1 Example of Matrix-Column-Vector Multiplication via Linear Combinations
Constructions: 11.2 Matrix-Matrix Multiplication via Linear Combination of Columns
Generalizations: N/A

Properties: 10.3 Properties of Matrix-Column-Vector Multiplication
Sufficiencies: N/A
Questions: N/A

Matrix-Column-Vector Multiplication via Linear Combinations

Let matrix $A \in R^{m \times n}$ and 3.1 Column Vector $\vec{x} \in R^{n \times 1}$ .

Note that the number of rows of $\vec{x}$ matches the number of columns of $A$ .
! The inner dimensions must agree!

Then, the linear combination version of the matrix-column-vector product $A \vec{x} = \vec{b} \in R^{m \times 1}$ is given by

A \vec{x} = x_{1} [\begin{matrix} a_{11} \\ a_{21} \\ ⋮ \\ a_{m 1} \end{matrix}] + x_{2} [\begin{matrix} a_{12} \\ a_{22} \\ ⋮ \\ a_{m 2} \end{matrix}] + \dots + x_{n} [\begin{matrix} a_{1 n} \\ a_{2 n} \\ ⋮ \\ a_{m n} \end{matrix}] = \vec{b}

The matrix vector product can also be written using 8.11 Colon Notation and summation notation.

A \vec{x} = x_{1} A (:, 1) + x_{2} A (:, 2) + \dots + x_{n} A (:, n) = \sum_{k = 1}^{n} x_{k} A (:, k) = \vec{b}

Remark. In the real world when the matrix is on the left, it usually represents some real world data that we have collected. The right vector $\vec{x}$ is the algebraic work we are trying to do on this data.

\underset{modeling matrix}{\underset{⏟}{{[\begin{matrix} A \end{matrix}]}_{m \times n}}} \cdot \underset{algebraic work}{\underset{⏟}{{[\begin{matrix} \vec{x} \end{matrix}]}_{n \times 1}}}

Let's cut up our modeling matrix into columns and vector into individual scalars.

[A]_{m \times n} = {[\begin{matrix} A (:, 1) & A (:, 2) & \dots & A (:, n) \end{matrix}]}_{m \times n} \cdot {[\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}]}_{n \times 1}

We multiply by "pairing" each column of $A$ with a corresponding scalar of $\vec{x}$ .

\begin{aligned} = x_{1} A (:, 1) + x_{2} A (:, 2) + \dots + x_{n} A (:, n) \\ = x_{1} {[\begin{array}{c} a_{11} \\ a_{21} \\ ⋮ \\ a_{m 1} \end{array}]}_{m \times 1} + x_{2} {[\begin{array}{c} a_{12} \\ a_{22} \\ ⋮ \\ a_{m 2} \end{array}]}_{m \times 1} + \dots + x_{n} {[\begin{array}{c} a_{1 n} \\ a_{2 n} \\ ⋮ \\ a_{m n} \end{array}]}_{m \times 1} \end{aligned}

Why must the inner dimensions agree?

Because the product of matrix-column-vector multiplication is a linear combination, there has to be a matching scalar for each column of $A$ (no column left behind!). In other words, every column must have a scalar value to be matched with.

$⧫$

Remark. We know what the input will look like, but what about the output? The size of the output will be the size of the sum of adding $m \times 1$ vectors.

⟹ [A]_{m \times n} \cdot [\vec{x}]_{n \times 1} = {\vec{b}}_{m \times 1}

A nice way to think of this would be the $n$ dimensions "cancelling" each other out.
$⧫$

Remark. Instead of a linear combination, we can also see the product as entry-by-entry values. Recall that in 4.1 Scalar-Vector Multiplication, we can bring the scalar inside the vector.

A_{m \times n} \cdot {\vec{x}}_{n \times 1} = [\begin{matrix} x_{1} a_{11} \\ x_{1} a_{21} \\ ⋮ \\ x_{1} a_{m 1} \end{matrix}] + [\begin{matrix} x_{2} a_{12} \\ x_{2} a_{22} \\ ⋮ \\ x_{2} a_{m 2} \end{matrix}] + \dots + [\begin{matrix} x_{n} a_{1 n} \\ x_{n} a_{2 n} \\ ⋮ \\ x_{n} a_{m n} \end{matrix}]

We can go even further by looking at the sums using 4.2 Column Vector Addition.

{[\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{m} \end{matrix}]}_{m \times 1} = {[\begin{matrix} x_{1} a_{11} + x_{2} a_{12} + \dots + x_{n} a_{1 n} \\ x_{2} a_{12} + x_{2} a_{22} + \dots + x_{n} a_{2 n} \\ ⋮ \\ x_{1} a_{m 1} + x_{2} a_{m 2} + \dots + x_{n} a_{m n} \end{matrix}]}_{m \times 1}

Claim

Every individual entry of $\vec{b}$ can be calculated as an 5.1 Inner Product Between Vectors.

$⧫$