Skip to main content

Section D Dimension

Almost every vector space we have encountered has been infinite in size (an exception is Example VSS). But some are bigger and richer than others. Dimension, once suitably defined, will be a measure of the size of a vector space, and a useful tool for studying its properties. You probably already have a rough notion of what a mathematical definition of dimension might be — try to forget these imprecise ideas and go with the new ones given here.

Subsection D Dimension

Definition D. Dimension.

Suppose that \(V\) is a vector space and \(\set{\vectorlist{v}{t}}\) is a basis of \(V\text{.}\) Then the dimension of \(V\) is defined by \(\dimension{V}=t\text{.}\) If \(V\) has no finite bases, we say \(V\) has infinite dimension.

This is a very simple definition, which belies its power. Grab a basis, any basis, and count up the number of vectors it contains. That is the dimension. However, this simplicity causes a problem. Given a vector space, you and I could each construct different bases — remember that a vector space might have many bases. And what if your basis and my basis had different sizes? Applying Definition D we would arrive at different numbers! With our current knowledge about vector spaces, we would have to say that dimension is not “well-defined.” Fortunately, there is a theorem that will correct this problem.

In a strictly logical progression, the next two theorems would precede the definition of dimension. Many subsequent theorems will trace their lineage back to the following fundamental result.

We want to prove that any set of \(t+1\) or more vectors from \(V\) is linearly dependent. So we will begin with a totally arbitrary set of vectors from \(V\text{,}\) \(R=\set{\vectorlist{u}{m}}\text{,}\) where \(m\gt t\text{.}\) We will now construct a nontrivial relation of linear dependence on \(R\text{.}\)

Each vector \(\vectorlist{u}{m}\) can be written as a linear combination of the vectors \(\vectorlist{v}{t}\) since \(S\) is a spanning set of \(V\text{.}\) This means there exist scalars \(a_{ij}\text{,}\) \(1\leq i\leq t\text{,}\) \(1\leq j\leq m\text{,}\) so that

\begin{align*} \vect{u}_1&=a_{11}\vect{v}_1+a_{21}\vect{v}_2+a_{31}\vect{v}_3+\cdots+a_{t1}\vect{v}_t\\ \vect{u}_2&=a_{12}\vect{v}_1+a_{22}\vect{v}_2+a_{32}\vect{v}_3+\cdots+a_{t2}\vect{v}_t\\ \vect{u}_3&=a_{13}\vect{v}_1+a_{23}\vect{v}_2+a_{33}\vect{v}_3+\cdots+a_{t3}\vect{v}_t\\ &\quad\quad\vdots\\ \vect{u}_m&=a_{1m}\vect{v}_1+a_{2m}\vect{v}_2+a_{3m}\vect{v}_3+\cdots+a_{tm}\vect{v}_t \end{align*}

Now we form, unmotivated, the homogeneous system of \(t\) equations in the \(m\) variables, \(x_1,\,x_2,\,x_3,\,\ldots,\,x_m\text{,}\) where the coefficients are the just-discovered scalars \(a_{ij}\text{,}\)

\begin{align*} a_{11}x_1+a_{12}x_2+a_{13}x_3+\cdots+a_{1m}x_m&=0\\ a_{21}x_1+a_{22}x_2+a_{23}x_3+\cdots+a_{2m}x_m&=0\\ a_{31}x_1+a_{32}x_2+a_{33}x_3+\cdots+a_{3m}x_m&=0\\ \vdots\quad\quad&\\ a_{t1}x_1+a_{t2}x_2+a_{t3}x_3+\cdots+a_{tm}x_m&=0 \end{align*}

This is a homogeneous system with more variables than equations (our hypothesis is expressed as \(m\gt t\)), so by Theorem HMVEI there are infinitely many solutions. Choose a nontrivial solution and denote it by \(x_1=c_1,\,x_2=c_2,\,x_3=c_3,\,\ldots,\,x_m=c_m\text{.}\) As a solution to the homogeneous system, we then have

\begin{align*} a_{11}c_1+a_{12}c_2+a_{13}c_3+\cdots+a_{1m}c_m&=0\\ a_{21}c_1+a_{22}c_2+a_{23}c_3+\cdots+a_{2m}c_m&=0\\ a_{31}c_1+a_{32}c_2+a_{33}c_3+\cdots+a_{3m}c_m&=0\\ \vdots\quad\quad&\\ a_{t1}c_1+a_{t2}c_2+a_{t3}c_3+\cdots+a_{tm}c_m&=0 \end{align*}

As a collection of nontrivial scalars, \(c_1,\,c_2,\,c_3,\,\dots,\,c_m\) will provide the nontrivial relation of linear dependence we desire,

\begin{align*} &\lincombo{c}{u}{m}\\ &=c_{1}\left(a_{11}\vect{v}_1+a_{21}\vect{v}_2+a_{31}\vect{v}_3+\cdots+a_{t1}\vect{v}_t\right)&& \knowl{./knowl/definition-SSVS.html}{\text{Definition SSVS}}\\ &\quad\quad+c_{2}\left(a_{12}\vect{v}_1+a_{22}\vect{v}_2+a_{32}\vect{v}_3+\cdots+a_{t2}\vect{v}_t\right)\\ &\quad\quad+c_{3}\left(a_{13}\vect{v}_1+a_{23}\vect{v}_2+a_{33}\vect{v}_3+\cdots+a_{t3}\vect{v}_t\right)\\ &\quad\quad\quad\quad\vdots\\ &\quad\quad+c_{m}\left(a_{1m}\vect{v}_1+a_{2m}\vect{v}_2+a_{3m}\vect{v}_3+\cdots+a_{tm}\vect{v}_t\right)\\ &=c_{1}a_{11}\vect{v}_1+c_{1}a_{21}\vect{v}_2+c_{1}a_{31}\vect{v}_3+\cdots+c_{1}a_{t1}\vect{v}_t&& \knowl{./knowl/property-DVA.html}{\text{Property DVA}}\\ &\quad\quad+c_{2}a_{12}\vect{v}_1+c_{2}a_{22}\vect{v}_2+c_{2}a_{32}\vect{v}_3+\cdots+c_{2}a_{t2}\vect{v}_t\\ &\quad\quad+c_{3}a_{13}\vect{v}_1+c_{3}a_{23}\vect{v}_2+c_{3}a_{33}\vect{v}_3+\cdots+c_{3}a_{t3}\vect{v}_t\\ &\quad\quad\quad\quad\vdots\\ &\quad\quad+c_{m}a_{1m}\vect{v}_1+c_{m}a_{2m}\vect{v}_2+c_{m}a_{3m}\vect{v}_3+\cdots+c_{m}a_{tm}\vect{v}_t\\ &=\left(c_{1}a_{11}+c_{2}a_{12}+c_{3}a_{13}+\cdots+c_{m}a_{1m}\right)\vect{v}_1&& \knowl{./knowl/property-DSA.html}{\text{Property DSA}}\\ &\quad\quad+\left(c_{1}a_{21}+c_{2}a_{22}+c_{3}a_{23}+\cdots+c_{m}a_{2m}\right)\vect{v}_2\\ &\quad\quad+\left(c_{1}a_{31}+c_{2}a_{32}+c_{3}a_{33}+\cdots+c_{m}a_{3m}\right)\vect{v}_3\\ &\quad\quad\quad\quad\vdots\\ &\quad\quad+\left(c_{1}a_{t1}+c_{2}a_{t2}+c_{3}a_{t3}+\cdots+c_{m}a_{tm}\right)\vect{v}_t\\ &=\left(a_{11}c_{1}+a_{12}c_{2}+a_{13}c_{3}+\cdots+a_{1m}c_{m}\right)\vect{v}_1&& \knowl{./knowl/property-CMCN.html}{\text{Property CMCN}}\\ &\quad\quad+\left(a_{21}c_{1}+a_{22}c_{2}+a_{23}c_{3}+\cdots+a_{2m}c_{m}\right)\vect{v}_2\\ &\quad\quad+\left(a_{31}c_{1}+a_{32}c_{2}+a_{33}c_{3}+\cdots+a_{3m}c_{m}\right)\vect{v}_3\\ &\quad\quad\quad\quad\vdots\\ &\quad\quad+\left(a_{t1}c_{1}+a_{t2}c_{2}+a_{t3}c_{3}+\cdots+a_{tm}c_{m}\right)\vect{v}_t\\ &=0\vect{v}_1+0\vect{v}_2+0\vect{v}_3+\cdots+0\vect{v}_t&& c_j\text{ as solution}\\ &=\zerovector+\zerovector+\zerovector+\cdots+\zerovector&& \knowl{./knowl/theorem-ZSSM.html}{\text{Theorem ZSSM}}\\ &=\zerovector&& \knowl{./knowl/property-Z.html}{\text{Property Z}}\text{.} \end{align*}

That does it. \(R\) has been undeniably shown to be a linearly dependent set.

The proof just given has some monstrous expressions in it, mostly owing to the double subscripts present. Now is a great opportunity to show the value of a more compact notation. We will rewrite the key steps of the previous proof using summation notation, resulting in a more economical presentation, and even greater insight into the key aspects of the proof. So here is an alternate proof — study it carefully.

Alternate Proof: We want to prove that any set of \(t+1\) or more vectors from \(V\) is linearly dependent. So we will begin with a totally arbitrary set of vectors from \(V\text{,}\) \(R=\setparts{\vect{u}_j}{1\leq j\leq m}\text{,}\) where \(m\gt t\text{.}\) We will now construct a nontrivial relation of linear dependence on \(R\text{.}\)

Each vector \(\vect{u_j}\text{,}\) \(1\leq j\leq m\) can be written as a linear combination of \(\vect{v}_i\text{,}\) \(1\leq i\leq t\) since \(S\) is a spanning set of \(V\text{.}\) This means there are scalars \(a_{ij}\text{,}\) \(1\leq i\leq t\text{,}\) \(1\leq j\leq m\text{,}\) so that

\begin{align*} \vect{u}_j&=\sum_{i=1}^{t}a_{ij}\vect{v}_i&&1\leq j\leq m \end{align*}

Now we form, unmotivated, the homogeneous system of \(t\) equations in the \(m\) variables, \(x_j\text{,}\) \(1\leq j\leq m\text{,}\) where the coefficients are the just-discovered scalars \(a_{ij}\text{,}\)

\begin{align*} \sum_{j=1}^{m}a_{ij}x_j=0&&1\leq i\leq t \end{align*}

This is a homogeneous system with more variables than equations (our hypothesis is expressed as \(m\gt t\)), so by Theorem HMVEI there are infinitely many solutions. Choose one of these solutions that is not trivial and denote it by \(x_j=c_j\text{,}\) \(1\leq j\leq m\text{.}\) As a solution to the homogeneous system, we then have \(\sum_{j=1}^{m}a_{ij}c_{j}=0\) for \(1\leq i\leq t\text{.}\) As a collection of nontrivial scalars, \(c_j\text{,}\) \(1\leq j\leq m\text{,}\) will provide the nontrivial relation of linear dependence we desire,

\begin{align*} \sum_{j=1}^{m}c_{j}\vect{u}_j &=\sum_{j=1}^{m}c_{j}\left(\sum_{i=1}^{t}a_{ij}\vect{v}_i\right)&& \knowl{./knowl/definition-SSVS.html}{\text{Definition SSVS}}\\ &=\sum_{j=1}^{m}\sum_{i=1}^{t}c_{j}a_{ij}\vect{v}_i&& \knowl{./knowl/property-DVA.html}{\text{Property DVA}}\\ &=\sum_{i=1}^{t}\sum_{j=1}^{m}c_{j}a_{ij}\vect{v}_i&& \knowl{./knowl/property-C.html}{\text{Property C}}\\ &=\sum_{i=1}^{t}\sum_{j=1}^{m}a_{ij}c_{j}\vect{v}_i&& \knowl{./knowl/property-CMCN.html}{\text{Property CMCN}}\\ &=\sum_{i=1}^{t}\left(\sum_{j=1}^{m}a_{ij}c_{j}\right)\vect{v}_i&& \knowl{./knowl/property-DSA.html}{\text{Property DSA}}\\ &=\sum_{i=1}^{t}0\vect{v}_i&& c_j\text{ as solution}\\ &=\sum_{i=1}^{t}\zerovector&& \knowl{./knowl/theorem-ZSSM.html}{\text{Theorem ZSSM}}\\ &=\zerovector&& \knowl{./knowl/property-Z.html}{\text{Property Z}}\text{.} \end{align*}

That does it. \(R\) has been undeniably shown to be a linearly dependent set.

Notice how the swap of the two summations is so much easier in the third step above, as opposed to all the rearranging and regrouping that takes place in the previous proof. And using only about half the space. And there are no ellipses (…).

Theorem SSLD can be viewed as a generalization of Theorem MVSLD. We know that \(\complex{m}\) has a basis with \(m\) vectors in it (Theorem SUVB), so it is a set of \(m\) vectors that spans \(\complex{m}\text{.}\) By Theorem SSLD, any set of more than \(m\) vectors from \(\complex{m}\) will be linearly dependent. But this is exactly the conclusion we have in Theorem MVSLD. Maybe this is not a total shock, as the proofs of both theorems rely heavily on Theorem HMVEI. The beauty of Theorem SSLD is that it applies in any vector space. We illustrate the generality of this theorem, and hint at its power, in the next example.

In Example SSP4 we showed that

\begin{equation*} S=\set{x-2,\,x^2-4x+4,\,x^3-6x^2+12x-8,\,x^4-8x^3+24x^2-32x+16} \end{equation*}

is a spanning set for \(W=\setparts{p(x)}{p\in P_4,\ p(2)=0}\text{.}\) So we can apply Theorem SSLD to \(W\) with \(t=4\text{.}\) Here is a set of five vectors from \(W\text{,}\) as you may check by verifying that each is a polynomial of degree 4 or less and has \(x=2\) as a root,

\begin{align*} T&=\set{p_1,\,p_2,\,p_3,\,p_4,\,p_5}\subseteq W\\ &\ \\ p_1&=x^4-2x^3+2x^2-8x+8\\ p_2&=-x^3+6x^2-5x-6\\ p_3&=2x^4-5x^3+5x^2-7x+2\\ p_4&=-x^4+4x^3-7x^2+6x\\ p_5&=4x^3-9x^2+5x-6 \end{align*}

By Theorem SSLD we conclude that \(T\) is linearly dependent, with no further computations.

Theorem SSLD is indeed powerful, but our main purpose in proving it right now was to make sure that our definition of dimension (Definition D) is well-defined. Here is the theorem.

Suppose that \(C\) has more vectors than \(B\text{.}\) (Allowing for the possibility that \(C\) is infinite, we can replace \(C\) by a subset that has more vectors than \(B\text{.}\)) As a basis, \(B\) is a spanning set for \(V\) (Definition B), so Theorem SSLD says that \(C\) is linearly dependent. However, this contradicts the fact that as a basis \(C\) is linearly independent (Definition B). So \(C\) must also be a finite set, with size less than, or equal to, that of \(B\text{.}\)

Suppose that \(B\) has more vectors than \(C\text{.}\) As a basis, \(C\) is a spanning set for \(V\) (Definition B), so Theorem SSLD says that \(B\) is linearly dependent. However, this contradicts the fact that as a basis \(B\) is linearly independent (Definition B). So \(C\) cannot be strictly smaller than \(B\text{.}\)

The only possibility left for the sizes of \(B\) and \(C\) is for them to be equal.

Theorem BIS tells us that if we find one finite basis in a vector space, then they all have the same size. This (finally) makes Definition D unambiguous.

Subsection DVS Dimension of Vector Spaces

We can now collect the dimension of some common, and not so common, vector spaces.

Example BP provides two bases with \(n+1\) vectors. Take your pick.

It should now be plausible that

\begin{equation*} Z=\setparts{\begin{bmatrix}a&b\\c&d\end{bmatrix}}{2a+b+3c+4d=0,\,-a+3b-5c-2d=0} \end{equation*}

is a subspace of the vector space \(M_{22}\) (Example VSM). (It is.) To find the dimension of \(Z\) we must first find a basis, though any old basis will do.

First concentrate on the conditions relating \(a,\,b,\,c\) and \(d\text{.}\) They form a homogeneous system of two equations in four variables with coefficient matrix

\begin{equation*} \begin{bmatrix} 2 & 1 & 3 & 4\\ -1 & 3 & -5 & -2 \end{bmatrix}\text{.} \end{equation*}

We can row-reduce this matrix to obtain

\begin{equation*} \begin{bmatrix} \leading{1} & 0 & 2 & 2\\ 0 & \leading{1} & -1 & 0 \end{bmatrix}\text{.} \end{equation*}

Rewrite the two equations represented by each row of this matrix, expressing the dependent variables (\(a\) and \(b\)) in terms of the free variables (\(c\) and \(d\)), and we obtain,

\begin{align*} a&=-2c-2d\\ b&=c \end{align*}

We can now write a typical entry of \(Z\) strictly in terms of \(c\) and \(d\text{,}\) and we can decompose the result,

\begin{equation*} \begin{bmatrix}a&b\\c&d\end{bmatrix}= \begin{bmatrix}-2c-2d&c\\c&d\end{bmatrix}= \begin{bmatrix}-2c&c\\c&0\end{bmatrix}+ \begin{bmatrix}-2d&0\\0&d\end{bmatrix}= c\begin{bmatrix}-2&1\\1&0\end{bmatrix}+ d\begin{bmatrix}-2&0\\0&1\end{bmatrix}\text{.} \end{equation*}

This equation says that an arbitrary matrix in \(Z\) can be written as a linear combination of the two vectors in

\begin{equation*} S=\set{\begin{bmatrix}-2&1\\1&0\end{bmatrix},\,\begin{bmatrix}-2&0\\0&1\end{bmatrix}} \end{equation*}

so we know that

\begin{equation*} Z=\spn{S}= \spn{\set{ \begin{bmatrix}-2&1\\1&0\end{bmatrix},\, \begin{bmatrix}-2&0\\0&1\end{bmatrix} }}\text{.} \end{equation*}

Are these two matrices (vectors) also linearly independent? Begin with a relation of linear dependence on \(S\text{,}\)

\begin{align*} a_1\begin{bmatrix}-2&1\\1&0\end{bmatrix}+ a_2\begin{bmatrix}-2&0\\0&1\end{bmatrix}&=\zeromatrix\\ \begin{bmatrix}-2a_1-2a_2&a_1\\a_1&a_2\end{bmatrix}&= \begin{bmatrix}0&0\\0&0\end{bmatrix} \end{align*}

From the equality of the two entries in the last row, we conclude that \(a_1=0\text{,}\) \(a_2=0\text{.}\) Thus the only possible relation of linear dependence is the trivial one, and therefore \(S\) is linearly independent (Definition LI). So \(S\) is a basis for \(Z\) (Definition B). Finally, we can conclude that \(\dimension{Z}=2\) (Definition D) since \(S\) has two elements.

In Example BSP4 we showed that

\begin{equation*} S=\set{x-2,\,x^2-4x+4,\,x^3-6x^2+12x-8,\,x^4-8x^3+24x^2-32x+16} \end{equation*}

is a basis for \(W=\setparts{p(x)}{p\in P_4,\ p(2)=0}\text{.}\) Thus, the dimension of \(W\) is four, \(\dimension{W}=4\text{.}\)

Note that \(\dimension{P_4}=5\) by Theorem DP, so \(W\) is a subspace of dimension 4 within the vector space \(P_4\) of dimension 5, illustrating the upcoming Theorem PSSD.

In Example BC we determined that the set \(R=\set{(1,\,0),\,(6,\,3)}\) from the crazy vector space, \(C\) (Example CVS), is a basis for \(C\text{.}\) By Definition D we see that \(C\) has dimension 2, \(\dimension{C}=2\text{.}\)

It is possible for a vector space to have no finite bases, in which case we say it has infinite dimension. Many of the best examples of this are vector spaces of functions, which lead to constructions like Hilbert spaces. We will focus exclusively on finite-dimensional vector spaces. OK, one infinite-dimensional example, and then we will focus exclusively on finite-dimensional vector spaces.

Define the set \(P\) by

\begin{equation*} P=\setparts{p}{p(x)\text{ is a polynomial in }x}\text{.} \end{equation*}

Our operations will be the same as those defined for \(P_n\) (Example VSP).

With no restrictions on the possible degrees of our polynomials, any finite set that is a candidate for spanning \(P\) will come up short. We will give a proof by contradiction (Proof Technique CD). To this end, suppose that the dimension of \(P\) is finite, say \(\dimension{P}=n\text{.}\)

The set \(T=\set{1,\,x,\,x^2,\,\ldots,\,x^n}\) is a linearly independent set (check this!) containing \(n+1\) polynomials from \(P\text{.}\) However, a basis of \(P\) will be a spanning set of \(P\) containing \(n\) vectors. This situation is a contradiction of Theorem SSLD, so our assumption that \(P\) has finite dimension is false. Thus, we say \(\dimension{P}=\infty\text{.}\)

Sage D. Dimension.

Now we recognize that every basis has the same size, even if there are many different bases for a given vector space. The dimension is an important piece of information about a vector space, so Sage routinely provides this as part of the description of a vector space. But it can be returned by itself with the vector space method .dimension(). Here is an example of a subspace with dimension 2.

Subsection RNM Rank and Nullity of a Matrix

For any matrix, we have seen that we can associate several subspaces — the null space (Theorem NSMS), the column space (Theorem CSMS), row space (Theorem RSMS) and the left null space (Theorem LNSMS). As vector spaces, each of these has a dimension, and for the null space and column space, they are important enough to warrant names.

Definition NOM. Nullity Of a Matrix.

Suppose that \(A\) is an \(m\times n\) matrix. Then the nullity of \(A\) is the dimension of the null space of \(A\text{,}\) \(\nullity{A}=\dimension{\nsp{A}}\text{.}\)

Definition ROM. Rank Of a Matrix.

Suppose that \(A\) is an \(m\times n\) matrix. Then the rank of \(A\) is the dimension of the column space of \(A\text{,}\) \(\rank{A}=\dimension{\csp{A}}\text{.}\)

Let us compute the rank and nullity of

\begin{equation*} A=\begin{bmatrix} 2 & -4 & -1 & 3 & 2 & 1 & -4\\ 1 & -2 & 0 & 0 & 4 & 0 & 1\\ -2 & 4 & 1 & 0 & -5 & -4 & -8\\ 1 & -2 & 1 & 1 & 6 & 1 & -3\\ 2 & -4 & -1 & 1 & 4 & -2 & -1\\ -1 & 2 & 3 & -1 & 6 & 3 & -1 \end{bmatrix}\text{.} \end{equation*}

To do this, we will first row-reduce the matrix since that will help us determine bases for the null space and column space,

\begin{equation*} \begin{bmatrix} \leading{1} & -2 & 0 & 0 & 4 & 0 & 1\\ 0 & 0 & \leading{1} & 0 & 3 & 0 & -2\\ 0 & 0 & 0 & \leading{1} & -1 & 0 & -3\\ 0 & 0 & 0 & 0 & 0 & \leading{1} & 1\\ 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{bmatrix}\text{.} \end{equation*}

From this row-equivalent matrix in reduced row-echelon form we record \(D=\set{1,\,3,\,4,\,6}\) and \(F=\set{2,\,5,\,7}\text{.}\)

For each index in \(D\text{,}\) Theorem BCS creates a single basis vector. In total the basis will have \(4\) vectors, so the column space of \(A\) will have dimension \(4\) and we write \(\rank{A}=4\text{.}\)

For each index in \(F\text{,}\) Theorem BNS creates a single basis vector. In total the basis will have \(3\) vectors, so the null space of \(A\) will have dimension \(3\) and we write \(\nullity{A}=3\text{.}\)

There were no accidents or coincidences in the previous example — with the row-reduced version of a matrix in hand, the rank and nullity are easy to compute.

Theorem BCS provides a basis for the column space by choosing columns of \(A\) that have the same indices as the pivot columns of \(B\text{.}\) In the analysis of \(B\text{,}\) each leading 1 provides one nonzero row and one pivot column. So there are \(r\) column vectors in a basis for \(\csp{A}\text{.}\)

Theorem BNS provides a basis for the null space by creating basis vectors of the null space of \(A\) from entries of \(B\text{,}\) one basis vector for each column that is not a pivot column. So there are \(n-r\) column vectors in a basis for \(\nullity{A}\text{.}\)

Every archetype (Appendix A) that involves a matrix lists its rank and nullity. You may have noticed as you studied the archetypes that the larger the column space is the smaller the null space is. A simple corollary states this trade-off succinctly. (See Proof Technique LC.)

Let \(r\) be the number of nonzero rows in a row-equivalent matrix in reduced row-echelon form. By Theorem CRN,

\begin{equation*} \rank{A}+\nullity{A}= r+(n-r)=n\text{.} \end{equation*}

When we first introduced \(r\) as our standard notation for the number of nonzero rows in a matrix in reduced row-echelon form you might have thought \(r\) stood for “rows.” Not really — it stands for “rank”!

Sage RNM. Rank and Nullity of a Matrix.

The rank and nullity of a matrix in Sage could be exactly what you would have guessed. But we need to be careful. The rank is the rank. But nullity in Sage is the dimension of the left nullspace. So we have matrix methods .nullity(), .left_nullity(), .right_nullity(), where the first two are equal and correspond to Sage's preference for rows, and the third is the column version used by the text. That said, a “row version” of Theorem RPNC is also true.

Subsection RNNM Rank and Nullity of a Nonsingular Matrix

Let us take a look at the rank and nullity of a square matrix.

The matrix

\begin{equation*} E=\begin{bmatrix} 0 & 4 & -1 & 2 & 2 & 3 & 1\\ 2 & -2 & 1 & -1 & 0 & -4 & -3\\ -2 & -3 & 9 & -3 & 9 & -1 & 9\\ -3 & -4 & 9 & 4 & -1 & 6 & -2\\ -3 & -4 & 6 & -2 & 5 & 9 & -4\\ 9 & -3 & 8 & -2 & -4 & 2 & 4\\ 8 & 2 & 2 & 9 & 3 & 0 & 9 \end{bmatrix} \end{equation*}

is row-equivalent to the matrix in reduced row-echelon form,

\begin{equation*} \begin{bmatrix} \leading{1} & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & \leading{1} & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & \leading{1} & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & \leading{1} & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & \leading{1} & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & \leading{1} & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & \leading{1} \end{bmatrix}\text{.} \end{equation*}

With \(n=7\) columns and \(r=7\) nonzero rows Theorem CRN tells us the rank is \(\rank{E}=7\) and the nullity is \(\nullity{E}=7-7=0\text{.}\)

The value of either the nullity or the rank are enough to characterize a nonsingular matrix.

(1 \(\Rightarrow\) 2) Theorem CSNM says that if \(A\) is nonsingular then \(\csp{A}=\complex{n}\text{.}\) If \(\csp{A}=\complex{n}\text{,}\) then the column space has dimension \(n\) by Theorem DCM, so the rank of \(A\) is \(n\text{.}\)

(2 \(\Rightarrow\) 3) Suppose \(\rank{A}=n\text{.}\) Then Theorem RPNC gives

\begin{align*} \nullity{A}&=n-\rank{A}&& \knowl{./knowl/theorem-RPNC.html}{\text{Theorem RPNC}}\\ &=n-n&& \text{Hypothesis}\\ &=0\text{.} \end{align*}

(3 \(\Rightarrow\) 1) Suppose \(\nullity{A}=0\text{,}\) so a basis for the null space of \(A\) is the empty set. This implies that \(\nsp{A}=\set{\zerovector}\) and Theorem NMTNS says \(A\) is nonsingular.

With a new equivalence for a nonsingular matrix, we can update our list of equivalences (Theorem NME5) which now becomes a list requiring double digits to number.

Sage NME6. Nonsingular Matrix Equivalences, Round 6.

Recycling the nonsingular matrix from Sage NME5 we can use Sage to verify the two new equivalences of Theorem NME6.

Reading Questions D Reading Questions

1.

What is the dimension of the vector space \(P_6\text{,}\) the set of all polynomials of degree 6 or less?

2.

How are the rank and nullity of a matrix related?

3.

Explain why we might say that a nonsingular matrix has “full rank.”

Exercises D Exercises

C20.

The archetypes listed below are matrices, or systems of equations with coefficient matrices. For each, compute the nullity and rank of the matrix. This information is listed for each archetype (along with the number of columns in the matrix, so as to illustrate Theorem RPNC), and notice how it could have been computed immediately after the determination of the sets \(D\) and \(F\) associated with the reduced row-echelon form of the matrix. Archetype A, Archetype B, Archetype C, Archetype D/Archetype E, Archetype F, Archetype G/Archetype H, Archetype I, Archetype J, Archetype K, Archetype L

C21.

Find the dimension of the subspace \(W = \setparts{\colvector{a + b\\ a + c\\a + d \\ d}}{a, b, c, d \in\complexes}\) of \(\complex{4}\text{.}\)

Solution.

The subspace \(W\) can be written as

\begin{align*} W &= \setparts{\colvector{a + b\\ a + c\\a + d \\ d}}{a, b, c, d \in\complexes}\\ &= \setparts{ a\colvector{1\\1\\1\\0} + b\colvector{1\\0\\0\\0} + c\colvector{0\\1\\0\\0} + d\colvector{0\\0\\1\\1} } {a, b, c, d \in\complexes}\\ &= \spn{\set{ \colvector{1\\1\\1\\0}, \colvector{1\\0\\0\\0}, \colvector{0\\1\\0\\0}, \colvector{0\\0\\1\\1} }} \end{align*}

Since the set of vectors

\begin{equation*} \set{ \colvector{1\\1\\1\\0},\, \colvector{1\\0\\0\\0},\, \colvector{0\\1\\0\\0},\, \colvector{0\\0\\1\\1} } \end{equation*}

is a linearly independent set (why?), it forms a basis of \(W\text{.}\) Thus, \(W\) is a subspace of \(\complex{4}\) with dimension 4 (and must therefore equal \(\complex{4}\)).

C22.

Find the dimension of the subspace \(W = \setparts{a + bx + cx^2 + dx^3}{a + b + c + d = 0}\) of \(P_3\text{.}\)

Solution.

The subspace \(W = \setparts{a + bx + cx^2 + dx^3}{a + b + c + d = 0}\) can be written as

\begin{align*} W&= \setparts{a + bx + cx^2 + (-a -b -c)x^3}{a,b,c\in\complexes}\\ &= \setparts{a(1-x^3) + b(x - x^3) + c(x^2 - x^3)}{a,b,c\in\complexes}\\ &= \spn{\set{1 - x^3, x - x^3, x^2 - x^3}} \end{align*}

Since these vectors are linearly independent (why?), \(W\) is a subspace of \(P_3\) with dimension 3.

C23.

Find the dimension of the subspace\(W = \setparts{\begin{bmatrix} a & b\\c & d \end{bmatrix}}{a + b = c, b + c = d, c + d = a}\) of \(M_{22}\text{.}\)

Solution.

The equations specified are equivalent to the system

\begin{align*} a + b - c &= 0\\ b + c - d &= 0\\ a - c - d &= 0\text{.} \end{align*}

The coefficient matrix of this system row-reduces to

\begin{align*} \begin{bmatrix} \leading{1} & 0 & 0& -3\\ 0 & \leading{1} & 0 & 1\\ 0 & 0 & \leading{1} & -2 \end{bmatrix}\text{.} \end{align*}

Thus, every solution can be decribed with a suitable choice of \(d\text{,}\) together with \(a = 3d\text{,}\) \(b = -d\) and \(c = 2d\text{.}\) Thus the subspace \(W\) can be described as

\begin{align*} W &= \setparts{\begin{bmatrix} 3d & -d \\ 2d & d\end{bmatrix}}{d\in\complexes} =\spn{\set{\begin{bmatrix} 3 & -1 \\ 2 & 1\end{bmatrix}}} \end{align*}

So, \(W\) is a subspace of \(M_{22}\) with dimension 1.

C30.

For the matrix \(A\) below, compute the dimension of the null space of \(A\text{,}\) \(\dimension{\nsp{A}}\text{.}\)

\begin{equation*} A= \begin{bmatrix} 2 & -1 & -3 & 11 & 9 \\ 1 & 2 & 1 & -7 & -3 \\ 3 & 1 & -3 & 6 & 8 \\ 2 & 1 & 2 & -5 & -3 \end{bmatrix} \end{equation*}
Solution.

Row reduce \(A\text{.}\)

\begin{equation*} A\rref \begin{bmatrix} \leading{1} & 0 & 0 & 1 & 1 \\ 0 & \leading{1} & 0 & -3 & -1 \\ 0 & 0 & \leading{1} & -2 & -2 \\ 0 & 0 & 0 & 0 & 0 \end{bmatrix} \end{equation*}

So \(r=3\) for this matrix. Then

\begin{align*} \dimension{\nsp{A}}&=\nullity{A}&& \knowl{./knowl/definition-NOM.html}{\text{Definition NOM}}\\ &=\left(\nullity{A}+\rank{A}\right)-\rank{A}\\ &=5-\rank{A}&& \knowl{./knowl/theorem-RPNC.html}{\text{Theorem RPNC}}\\ &=5-3&& \knowl{./knowl/theorem-CRN.html}{\text{Theorem CRN}}\\ &=2 \end{align*}

We could also use Theorem BNS and create a basis for \(\nsp{A}\) with \(n-r=5-3=2\) vectors (because the solutions are described with 2 free variables) and arrive at the dimension as the size of this basis.

C31.

The set \(W\) below is a subspace of \(\complex{4}\text{.}\) Find the dimension of \(W\text{.}\)

\begin{equation*} W=\spn{\set{ \colvector{2\\-3\\4\\1},\, \colvector{3\\0\\1\\-2},\, \colvector{-4\\-3\\2\\5} }} \end{equation*}
Solution.

We will appeal to Theorem BS (or you could consider this an appeal to Theorem BCS). Put the three column vectors of this spanning set into a matrix as columns and row-reduce.

\begin{equation*} \begin{bmatrix} 2 & 3 & -4 \\ -3 & 0 & -3 \\ 4 & 1 & 2 \\ 1 & -2 & 5 \end{bmatrix} \rref \begin{bmatrix} \leading{1} & 0 & 1 \\ 0 & \leading{1} & -2 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix} \end{equation*}

The pivot columns are \(D=\set{1,2}\) so we can “keep” the vectors of \(W\) with the same indices and set

\begin{equation*} T=\set{ \colvector{2\\-3\\4\\1},\, \colvector{3\\0\\1\\-2}} \end{equation*}

and conclude that \(W=\spn{T}\) and \(T\) is linearly independent. In other words, \(T\) is a basis with two vectors, so \(W\) has dimension 2.

C35.

Find the rank and nullity of the matrix \(A\text{.}\)

\begin{equation*} A = \begin{bmatrix} 1 & 0 & 1\\ 1 & 2 & 2\\ 2 & 1 & 1\\ -1 & 0 & 1\\ 1 & 1 & 2 \end{bmatrix} \end{equation*}
Solution.

Analyzing the row-reduced form of matrix below shows that the rank of \(A\) (number of pivot columns) is \(3\text{,}\) and the nullity is \(0\text{.}\)

\begin{equation*} \begin{bmatrix} \leading{1} & 0 & 0\\ 0 & \leading{1} & 0\\ 0 & 0 & \leading{1}\\ 0 & 0 & 0\\ 0 & 0 & 0 \end{bmatrix} \end{equation*}

C36.

Find the rank and nullity of the matrix

\begin{equation*} A = \begin{bmatrix} 1 & 2 & 1 & 1 & 1\\ 1 & 3 & 2 & 0 & 4\\ 1 & 2 & 1 & 1 & 1 \end{bmatrix}\text{.} \end{equation*}
Solution.

The row reduced form of matrix \(A\) is

\begin{equation*} \begin{bmatrix} \leading{1} & 0 & -1 & 3 & 5\\ 0 & \leading{1} & 1& -1 & 3\\ 0 & 0 & 0 & 0 & 0 \end{bmatrix}\text{,} \end{equation*}

so the rank of \(A\) (the number of pivot columns) is \(2\text{,}\) and the nullity is \(5 - 2 = 3\text{.}\)

C37.

Find the rank and nullity of the matrix

\begin{equation*} A = \begin{bmatrix} 3 & 2 & 1 & 1 & 1\\ 2 & 3 & 0 & 1 & 1\\ -1 & 1 & 2 & 1 & 0\\ 1 & 1 & 0 & 1 & 1\\ 0 & 1 & 1 & 2 & -1 \end{bmatrix}\text{.} \end{equation*}
Solution.

This matrix \(A\) row reduces to the \(5\times 5\) identity matrix, so it has full rank. The rank of \(A\) is 5, and the nullity is \(0\text{.}\)

C40.

In Example LDP4 we determined that the set of five polynomials, \(T\text{,}\) is linearly dependent by a simple invocation of Theorem SSLD. Prove that \(T\) is linearly dependent from scratch, beginning with Definition LI.

M20.

\(M_{22}\) is the vector space of \(2\times 2\) matrices. Let \(S_{22}\) denote the set of all \(2\times 2\) symmetric matrices. That is

\begin{equation*} S_{22}=\setparts{A\in M_{22}}{\transpose{A}=A} \end{equation*}

  1. Show that \(S_{22}\) is a subspace of \(M_{22}\text{.}\)

  2. Exhibit a basis for \(S_{22}\) and prove that it has the required properties.

  3. What is the dimension of \(S_{22}\text{?}\)

Solution.

(1) We will use the three criteria of Theorem TSS. The zero vector of \(M_{22}\) is the zero matrix, \(\zeromatrix\) (Definition ZM), which is a symmetric matrix. So \(S_{22}\) is not empty, since \(\zeromatrix\in S_{22}\text{.}\)

Suppose that \(A\) and \(B\) are two matrices in \(S_{22}\text{.}\) Then we know that \(\transpose{A}=A\) and \(\transpose{B}=B\text{.}\) We want to know if \(A+B\in S_{22}\text{,}\) so test \(A+B\) for membership,

\begin{align*} \transpose{\left(A+B\right)}&=\transpose{A}+\transpose{B}&& \knowl{./knowl/theorem-TMA.html}{\text{Theorem TMA}}\\ &=A+B&&A,\,B\in S_{22} \end{align*}

So \(A+B\) is symmetric and qualifies for membership in \(S_{22}\text{.}\)

Suppose that \(A\in S_{22}\) and \(\alpha\in\complexes\text{.}\) Is \(\alpha A\in S_{22}\text{?}\) We know that \(\transpose{A}=A\text{.}\) Now check that,

\begin{align*} \transpose{\alpha A}&=\alpha\transpose{A}&& \knowl{./knowl/theorem-TMSM.html}{\text{Theorem TMSM}}\\ &=\alpha A&&A\in S_{22} \end{align*}

So \(\alpha A\) is also symmetric and qualifies for membership in \(S_{22}\text{.}\)

With the three criteria of Theorem TSS fulfilled, we see that \(S_{22}\) is a subspace of \(M_{22}\text{.}\)

(2) An arbitrary matrix from \(S_{22}\) can be written as

\begin{equation*} \begin{bmatrix} a&b\\b&d \end{bmatrix}\text{.} \end{equation*}

We can express this matrix as

\begin{align*} \begin{bmatrix} a&b\\b&d \end{bmatrix} &= \begin{bmatrix} a&0\\0&0 \end{bmatrix}+ \begin{bmatrix} 0&b\\b&0 \end{bmatrix}+ \begin{bmatrix} 0&0\\0&d \end{bmatrix}\\ &= a \begin{bmatrix} 1&0\\0&0 \end{bmatrix}+ b \begin{bmatrix} 0&1\\1&0 \end{bmatrix}+ d \begin{bmatrix} 0&0\\0&1 \end{bmatrix} \end{align*}

this equation says that the set

\begin{equation*} T=\set{ \begin{bmatrix} 1&0\\0&0 \end{bmatrix},\, \begin{bmatrix} 0&1\\1&0 \end{bmatrix},\, \begin{bmatrix} 0&0\\0&1 \end{bmatrix} } \end{equation*}

spans \(S_{22}\text{.}\) Is it also linearly independent?

Write a relation of linear dependence on \(S\text{,}\)

\begin{align*} \zeromatrix&= a_1 \begin{bmatrix} 1&0\\0&0 \end{bmatrix}+ a_2 \begin{bmatrix} 0&1\\1&0 \end{bmatrix}+ a_3 \begin{bmatrix} 0&0\\0&1 \end{bmatrix}\\ \begin{bmatrix} 0&0\\0&0 \end{bmatrix} &= \begin{bmatrix} a_1&a_2\\a_2&a_3 \end{bmatrix}\text{.} \end{align*}

The equality of these two matrices (Definition ME) tells us that \(a_1=a_2=a_3=0\text{,}\) and the only relation of linear dependence on \(T\) is trivial. So \(T\) is linearly independent, and hence is a basis of \(S_{22}\text{.}\)

(3) The basis \(T\) found in part (2) has size 3. So by Definition D, \(\dimension{S_{22}}=3\text{.}\)

M21.

A \(2\times 2\) matrix \(B\) is upper triangular if \(\matrixentry{B}{21}=0\text{.}\) Let \(UT_2\) be the set of all \(2\times 2\) upper triangular matrices. Then \(UT_2\) is a subspace of the vector space of all \(2\times 2\) matrices, \(M_{22}\) (you may assume this). Determine the dimension of \(UT_2\) providing all of the necessary justifications for your answer.

Solution.

A typical matrix from \(UT_2\) looks like

\begin{equation*} \begin{bmatrix} a & b \\ 0 & c \end{bmatrix} \end{equation*}

where \(a,\,b,\,c\in\complex{}\) are arbitrary scalars. Observing this we can then write

\begin{equation*} \begin{bmatrix} a & b \\ 0 & c \end{bmatrix} = a \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} + b \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} + c \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} \end{equation*}

which says that

\begin{equation*} R=\set{ \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} ,\, \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} ,\,\begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} } \end{equation*}

is a spanning set for \(UT_2\) (Definition SSVS). Is \(R\) is linearly independent? If so, it is a basis for \(UT_2\text{.}\) So consider a relation of linear dependence on \(R\text{,}\)

\begin{equation*} \alpha_1 \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} + \alpha_2 \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} + \alpha_3 \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} = \zeromatrix = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix} \end{equation*}

From this equation, one rapidly arrives at the conclusion that \(\alpha_1=\alpha_2=\alpha_3=0\text{.}\) So \(R\) is a linearly independent set (Definition LI), and hence is a basis (Definition B) for \(UT_2\text{.}\) Now, we simply count up the size of the set \(R\) to see that the dimension of \(UT_2\) is \(\dimension{UT_2}=3\text{.}\)