Vector spaces, basis, and dimension
Video: Matrices, Linear Systems, and Determinants — 1:40:28
This section corresponds to 1:40:28 in Proof of Fermat Last Theorem FROM SCRATCH.
Vector spaces
We stated earlier that mathematics is not only about numbers, it is also about structure. A vector space is one of the most important examples of such a structure.
Vector Space: Axioms
A vector space \(V\) over a field \(F\) is a set with addition (\(+, V\times V \rightarrow V\)) and scalar multiplication (\(\cdot, V\times F \rightarrow V\)) satisfying these axioms for all \(x,y,z\in V\) and \(a, b\in F\):
Here we intentionally want to generalize the concept of a vector!
VS 1: \(x + y = y + x\) (addition: commutativity)
VS 2: \((x + y) + z = x + (y + z)\) (addition: associativity)
VS 3: \(\exists {\bf 0} \in V\ \text{s.t.}\ x + {\bf 0}= x\) (addition: zero vector)
VS 4: \(\forall x\in V, \exists(-x) \in V\ \text{s.t.}\ x + (-x) = {\bf 0}\) (addition: inverse)
VS 5: \(1 \cdot x = x\) (scalar multiplication: identity)
VS 6: \(1 (ab)\cdot x = a\cdot(b\cdot x)\) (scalar multiplication: associativity)
VS 7: \(a\cdot(x + y) = a\cdot x + a\cdot y\) (scalar multiplication: distributive law)
VS 8: \((a + b)\cdot x = a\cdot x + b\cdot x\) (scalar multiplication: distributive law)
Note that addition is also a generalized concept here, an operation which name happens to be addition, hence we cannot automatically assume that \(x + y = y + x\) always holds.
For example, assume \(V=\mathbb{R}\) and we define \(a + b := a^b + e^ab^{72}\in \mathbb{R}\). This definition of addition does not statisfy VS 1 and VS 2!
Example vector spaces
\(\mathbb{R}\), \(\mathbb{C}\), and the cartesian product \(\mathbb{R}^n\) are all examples of vector spaces.
For the first two, we define addition and multiplication in the familiar way. For the last one, we define addition as follows:
\[ (x_1, x_2, \dots, x_n) + (y_1, y_2, \dots, y_n) = (x_1+y_1, x_2+y_2, \dots, x_n+y_n) \]
and scalar multiplication
\[ a (x_1, x_2, \dots, x_n) = (ax_1, ax_2, \dots, ax_n) \]
Another example is \(\mathbb{R}[x]\), the set of all polynomials with real coefficients. So how do we know that \(\mathbb{R}[x]\) is a vector space? We check if all axioms hold! For example, if we add two polynomials and change the order under addition, do we get the same result back? Yes, we do!
\[ (1 + x) + (3 + x^2) = (3 + x^2) + (1 + x) \]
Similarly, we can check that all other axioms hold.
Also, the set of al continuous functions \(C_0(\mathbb{R})\) is a vector space, as is \(Mat_2(\mathbb{R})\).
However, \(\mathbb{N}\) is not a vector space, as 0 is not in the set. Also, if we pick any natural number, the negative is not in the set.
Subspace
A subspace of a vector space \(V\) is a non-empty subset \(W\) of $$ that is itself a vector space under the operations inherited from \(V\).
For example, \(\mathbb{R}\) is a subspace of \(\mathbb{C}\), \(\mathbb{R}\subset \mathbb{C}\).
Another example is
\[ W = \{(x,y)\in\mathbb{R} |\ y = 2x\} \subset \mathbb{R}^2 \]
We can check that \(W\) is both a vector space and a subspace of \(\mathbb{R}^2\). However, \(W'\)
\[ W' = \{(x,y)\in\mathbb{R} |\ y = x + 1\} \subset \mathbb{R}^2 \]
is not a vector space, as it does not have a zero vector in it \((0, 0) \notin W'\). Consequently, we cannot call it a subspace of \(\mathbb{R}^2\) either!
Another example is the set of all real coefficient polynomials with maximum degree two, \(P_2[\mathbb{R}]\). This is a vector space as well as a subspace of all real coefficient polynomials.
Linear combination
We are slowly moving towards the concept of a basis. So what is a basis? A basis is one of the most important ideas in linear algebra. Roughly speaking, a basis is a minimal set of building blocks for a vector space.
For example, if we look at \(\mathbb{R}^2\), a basis may be \(\{(1, 0), (0, 1)\}\). This constitutes a valid basis, as we can pick any vector \((a,b)\in\mathbb{R}^2\) and write it as a linear combination of the basis vectors:
\[ (a, b) = a(1, 0) + b(0, 1) \]
So if we know the basis, we can build every vector in the space from those basis vectors in a unique way.
But before we can define a basis properly, we need to define a few smaller concepts first.
A linear combination of vectors in a set \(S=\{v1, v2, \dots, v_k\}\) is any vector \({\bf u}\) of the form
\[ {\bf u} = a_1v_1 + a_2v_2 + \dots + a_kv_k \]
where \(a_1, a_2, \dots, a_k\) are scalars in the field \(F\).
Example 1: a linear combinations in \(\mathbb{R}^2\)?
Let’s take the subset of vectors
\[ S=\left\{ \begin{pmatrix} 1 \\ 2 \end{pmatrix}, \begin{pmatrix} 3 \\ 4 \end{pmatrix} \right\} \]
Then the linear combinations of \(S\) would be something like
\[ S= a\begin{pmatrix} 1 \\ 2 \end{pmatrix} +b \begin{pmatrix} 3 \\ 4 \end{pmatrix} \]
Now we can ask if \(\begin{pmatrix} 5 \\ 8\end{pmatrix}\) is a linear combination of \(S\). To this extent, we check if there exists an \(a\) and \(b\), such that
\[ a\begin{pmatrix} 1 \\ 2 \end{pmatrix} +b \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 5 \\ 8\end{pmatrix} \]
And indeed, this holds true for \(a=2\) and \(b=1\), so \(\begin{pmatrix} 5 \\ 8\end{pmatrix}\) is a linear combination of \(S\)!
Example 2: a linear combination in \(\mathbb{R}^3\)?
Let’s take the subset of vectors
\[ S=\left\{ \begin{pmatrix} 1 \\ 2 \\ 0 \end{pmatrix}, \begin{pmatrix} 4 \\ 7 \\ 0 \end{pmatrix} \right\} \]
We can ask if \(\begin{pmatrix} 3 \\ 4 \\ 5 \end{pmatrix}\) is a linear combination of \(S\):
\[ a\begin{pmatrix} 1 \\ 2 \\0 \end{pmatrix} +b \begin{pmatrix} 4 \\ 7 \\ 0 \end{pmatrix} = \begin{pmatrix} 3 \\ 4 \\ 5\end{pmatrix} \]
The answer is no, as the third coordinate will always remain zero. So \(\begin{pmatrix} 3 \\ 4 \\ 5 \end{pmatrix}\) is not a linear combination of \(S\).
Span
Let \(S\) be a non-empty subset of a vector space \(V\). The span of \(S\), denoted by \(\text{span}(S)\), is the smallest subset of \(V\) containing \(S\). In practice, it is the set of all linear combinations of elements of \(S=\{ v_1, v_2, \cdots, v_n\}\):
\[ \text{span}(S) = \{ a_1v_1+\cdots+a_n v_n | n \geq 1, v_i\in S, a_i \in F \} \]
Linear independence and dependence
A set of vectors \(S=\{ v_1, v_2, \cdots, v_k\}\) is linearly independent if the only solution to the equation
\[ a_1v_1+a_2v_2+\cdots+a_k v_k = {\bf 0} \]
is the trivial solution \(a_1=a_2=\cdots=a_k=0\). If a set is not linearly independent, it is called linearly dependent.
Example 1
In \(\mathbb{R}^2\), are \(\{(1, 2), (1, 0)\}\) linearly independent? To answer this quesiton, we check if the only solution to the following equation is the trivial solution:
\[ a_1(1, 2) + a_2(1, 0) = (a_1 + a_2, 2a_1) = (0, 0) \Rightarrow a_1=0 \Rightarrow a_2 = 0 \]
So these vectors are linearly independent.
Example 2
In \(\mathbb{R}^3\), are \(\{(1, 2, 3), (1, 3, 5), {2, 5, 8)\}\) linearly independent? To answer this quesiton, we check if the only solution to the following equation is the trivial solution:
\[ a_1(1, 2, 3) + a_2(1, 3, 5) + a_3(2, 5, 8) = (0, 0, 0) \Rightarrow a_1=0 \Rightarrow a_2 = 0 \]
Now we find a solution for \(a1 = 1, a2 = 1, a3= -1\), so these vectors are linearly dependent.
Example 3
In \(\mathbb{R}_{[x]}\), the collection of all real coefficient polynomials, is \(1 + x, x^2 + x, 2 + 3x +x^2\) linearly dependent or independent?
It is linearly dependent, because there is a solution:
\[ 2\cdot(1 + x) + 1\cdot(x^2 + x) -1\cdot(3x + x^2) = 0 \]
Now finally we can define the concepts of basis and dimension.
Basis and dimension
A basis for a vector space \(V\) is a linearly independent subset of \(V\) that spans \(V\)
The dimension of \(V\), \(\text{dim}(V)\), is the number of vectors in a basis
Example 1: \(\mathbb{R}^2\)
Basis:
\[ \left\{ \begin{pmatrix} 1 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \end{pmatrix} \right\} \Rightarrow \begin{pmatrix} a \\ b \end{pmatrix}= a\begin{pmatrix} 1 \\ 0 \end{pmatrix}+ b\begin{pmatrix} 0 \\ 1 \end{pmatrix} \]
Note that we want to have a set that is large enough to span the whole space, but at the same time is small enough to contain only the necessary information.
Example 2: \(\mathbb{R}^3\)
Most of the time, we take as basis:
\[ \left\{ \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \right\} \]
Since the number of elements in this basis is three, the dimension of \(\mathbb{R}^3\) is three.
Note that we could have taken any other set of three linearly independent basis vectors, such as
\[ \left\{ \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} \right\} \]
Example 3: \(P_2(\mathbb{R})\)
Possible basis are \(\{1, x, x^2\}\) and \(\{7, x + x^2, -x\}\). Since there are three vectors in the basis, the dimension of \(P_2(\mathbb{R})\) is three.
Example 4: $Mat_2()
The most easy basis would be
\[ \left\{ \begin{pmatrix} 1 && 0 \\ 0 && 0 \end{pmatrix}, \begin{pmatrix} 0 && 1 \\ 0 && 0 \end{pmatrix}, \begin{pmatrix} 0 && 0 \\ 1 && 0 \end{pmatrix}, \begin{pmatrix} 0 && 0 \\ 0 && 1 \end{pmatrix} \right\} \]
So the dimension of this vector space is four.