Lecture 9: Vector Algebra

• Linear combination of vectors• Geometric interpretation

• Interpreting as Matrix-Vector Multiplication

• Span of a set of vectors

• Vector Spaces and Subspaces

• Linearly Independent/Dependent set of vectors and bases

• Spanning set

• Dimension

1

Linear Combination of Vectors

• Let 𝑣 =

𝑣1𝑣2𝑣3.𝑣𝑝

∈ 𝑅𝑝 and c1, c2, …, cp be scalars then vector y defined by:

is called a linear combination of v1, v2, …. vp with weights c1, c2, … cp.

• The weights in a linear combination can be any real numbers including zero.

• And ‘y’ is said to be linearly dependent on v1, v2, …. vp.

Linear Dependence

• Definition:

v1, v2, ……vn are linearly dependent if vi can be written down as a linearcombination of rest of the prceeding vectors for any i.

• A zero vector is linearly dependent on any set of N-dimensional vectors where 𝑥1, 𝑥2, … are all 0.

𝑣𝑖 = 𝑐1𝑣1 + 𝑐2𝑣2+ 𝑐3𝑣3+ …… . . 𝑐𝑖−1𝑣𝑖−1 𝑓𝑜𝑟 𝑎𝑛𝑦 𝑖 = 1,2… . . 𝑟

Linear Dependence

If there is a solution for the vector equation

In other words, there exists values for 𝑎1, 𝑎2, … 𝑎𝑛 such that the above equation is satisfied, then b is linearly dependent on 𝑥1, 𝑥2, … 𝑥𝑛.

Linear Independence

• If there is no value for 𝑎1, 𝑎2, … exists such that the below equation is satisfied then b is linearly independent of the set of vectors 𝒗𝟏, 𝒗𝟐, … 𝒗𝒏.

• In other words, 𝑥1𝒂𝟏 + 𝑥2𝒂𝟐 +⋯𝑥𝑛𝒂𝒏 ≠ 𝒃 for any value of 𝑥1, 𝑥2, …

or 𝑥1𝒂𝟏 + 𝑥2𝒂𝟐 +⋯𝑥𝑛𝒂𝒏 − 𝒃 ≠ 𝟎

means b is linearly independent of the set of vectors 𝒂𝟏, 𝒂𝟐, …𝒂𝒏

In general, the set of vectors {𝒂𝟏, 𝒂𝟐, …𝒂𝒏, 𝒃} is linearly independent if 𝑥1𝒂𝟏 + 𝑥2𝒂𝟐 +⋯𝑥𝑛𝒂𝒏 + 𝑥𝑛+1𝒃 = 𝟎, only if 𝑥1, 𝑥2, … are all 0.

Linear Independence

Linearly Independent/Dependent Set

• A set of vectors that are linearly independent is called a Linearly Independent Set.

• If at least one vector in a set of vectors is linearly dependent on other vectors, then that set is called a Linearly Dependent Set.

• A set of vectors with a zero vector has to be a Linearly Dependent Set.

.

Examples of Linear Dependence

• A vector and its scalar multiple are linear dependent. (By definition.) 𝑥1𝒂 = 𝒃

• A vector that is a linear combination of a set of vectors is linearly dependent on those vectors. (By definition.)

8

Examples of Linear Dependence

𝐿𝑒𝑡 𝑥1 = 1 2 3 , 𝑥2 = 2 4 5 𝑎𝑛𝑑 𝑏 = [4 8 11]

Then the rank of the matrix M is

𝑀 =1 2 32 4 54 8 11

- the number of linearly independent vectors

Span of a set of vectors

A vector that is in the span of a set of vectors is linearly dependent on those vectors. (By definition.)

Span of a set of vectors

• Example 1:

Span of a set of vectors

• Example 2:

𝑃1

𝒗𝟐

𝒗𝟏

𝑃2

0

Same R2 plane can be spanned by other vectors too…

• Let a=21

and b=12

.

• Any point in R2 can be represented

as a linear combination of the a and b axes.

• 𝑃𝑎𝑟𝑏𝑖𝑡𝑟𝑎𝑟𝑦 = 𝑐1𝒂 + 𝑐2𝒃

• Example:

• 𝑃1 =20

=𝟒

𝟑𝒂 +

−2

3𝒃

• 𝑃2 =−12

= −4

3𝒂 +

5

3𝒃

𝒂 𝑎𝑛𝑑 𝒃 vectors SPAN the R2 plane.

13

𝑃1

𝒃

𝒂

𝑃2𝟒

𝟑𝒂

−2

3𝒃

−𝟒

𝟑𝒂

5

3𝒃

a-coordinate b-coordinate

Span of a set of vectors (geometric description)A GEOMETRIC DESCRIPTION OF SPAN V

Let v be a nonzero vector in R3 . Then Span v is the set of all scalar

mult iples of v, which is the set of points on the line in R3 through v

and 0. See the figure below

Xiaohui Xie (UCI) ICS 6N January 17, 2017 17 / 18

Span of a set of vectors (geometric description)A GEOMETRIC DESCRIPTION OF SPAN U, V

If u and v are nonzero vectors in R3 , with v not a multiple of u, then

Span u, v is the plane in R3 that contains u, v, and 0.

In particular, Span u, v contains the line in R3 through u and 0 and

the line through v and 0. See the figure below.

Xiaohui Xie (UCI) ICS 6N January 17, 2017 18 / 18

How many minimum number of vectors are necessary to span a space? • ONE non-zero vector is required and sufficient to span a 1D “space” (in other

words, a line). The dimension of the vectors has to be 1 or more.

• TWO linearly independent vectors are required and sufficient to span a 2D plane such as the XY plane. The dimension of the vectors has to be 2 or more.

• THREE linearly independent vectors are required and sufficient to span a 3D space (such as XYZ volume). The dimension of the vectors has to be 3 or more.

• N linearly independent vectors are required and sufficient to span an N-D space. The dimension of the vectors has to be N or more.

• The span of these vectors is called the “vector space” (or “subspace”, if it is a subset of a vector space).

• (Later we will see that a vector space is more generic than a span. But in this course, we will only see vector spaces that are spans of a sets of vectors.)

16

Span of a set of vectors

• Maximal subset consisting of linearly independent vectors is called the basis of the span of x1, x2, x3, …… xn

• And the number of elements in the basis is called the dimension of the span.

Spanning Set

• A basis is an efficient spanning set that contains no unnecessary vectors.

• A basis can be constructed from a spanning set by discarding unneeded vectors

• Basis construction

Spanning Set

𝒗4=𝟐𝒗1 − 𝒗2. So, by Spanning Set Theorem, {𝒗1, 𝒗2, 𝒗3} span the same subspace as {𝒗1, 𝒗2, 𝒗3, 𝒗4}. We also see that {𝒗1, 𝒗2, 𝒗3} are linearly independent. So that is the basis of W.

Vector Spaces

• Span of vectors also form a vector space.

• Now we will look more into the definition of vector spaces and their properties.

Vector Spaces

7) 𝑐 𝑢 + 𝑣 = 𝑐𝑢 + 𝑐𝑣 for any scalar c

Vector Spaces

Vector Subspace

• If a subset of a vector space also forms a vector space, then that subset is called a vector subspace.

• Hence, every vector subspace is also a vector space. And also every vector space is also a vector subspace (of itself and possibly of larger spaces).

Vector Subspace

Vector Subspace

• Example

Vector Subspace

• Example

Vector Basis

Vector Basis

Example:

• No, because the Span{𝒗1, 𝒗2} ≠ 𝐻. • Span{𝒗1, 𝒗2} is the entire XY plane, not just the vectors of the form [s,s,0].

Spanning Set

• The following three sets in R3 show how linearly independent set can be enlarged to a basis and how further enlargement destroys the linear independence of the set.

Dimension of a Vector Space

• We already discussed • Dimension of a vector: Number of components of the vector• Dimension of a vector space (or subspace): Minimum number of linearly independent

vectors required to span that space.

• New definition• Dimension of a vector space (or subspace): Number of vectors in its basis.

• What is Basis of vector space again? • A Linearly Independent Set (of vectors) whose span is the vector space.

• This implies:• Basis has the minimum number of linearly independent vectors that spans the space.

• Note: Basis is not unique. But all bases that span the same space has the same number of vectors.

30

• Solving a system of 𝑚 equation and 𝑛 unknown in Algebra can be converted in matrix-vector multiplication form.

𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 = 𝑏1𝑎21𝑥1 + 𝑎22𝑥2 + 𝑎23𝑥3 = 𝑏2𝑎31𝑥1 + 𝑎32𝑥2 + 𝑎33𝑥3 = 𝑏3

• 𝐴 is a 3 × 3 matrix, 𝒙 and 𝒃 are 3 × 1 column vectors. 𝐴 and 𝒃 are known, where 𝒙 is unknown. This is a matrix equation.

𝐴𝒙 = 𝒃,

𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎23𝑎31 𝑎32 𝑎33

𝑥1𝑥2𝑥3

=

𝑏1𝑏2𝑏3

Matrix equation

Matrix equation as a vector equation

• Matrix equation can be interpreted as a vector equation.• The left side of equation (𝐴𝒙) can be written as:

𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎23𝑎31 𝑎32 𝑎33

𝑥1𝑥2𝑥3

=

𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3𝑎21𝑥1 + 𝑎22𝑥2 + 𝑎23𝑥3𝑎31𝑥1 + 𝑎32𝑥2 + 𝑎33𝑥3

= 𝑥1

𝑎11𝑎21𝑎31

+ 𝑥2

𝑎12𝑎22𝑎32

+ 𝑥3

𝑎13𝑎23𝑎33

𝒄𝒐𝒍1 𝒄𝒐𝒍2 𝒄𝒐𝒍3 𝑥1𝒄𝒐𝒍1 𝑥2𝒄𝒐𝒍2 𝑥3𝒄𝒐𝒍3

Represented as a linear combination of column vectors

Matrix equation as a vector equation

• Now we have a vector equation, stating the same problem as 𝐴𝒙 = 𝒃

𝑥1

𝑎11𝑎21𝑎31

+ 𝑥2

𝑎12𝑎22𝑎32

+ 𝑥3

𝑎13𝑎23𝑎33

=

𝑏1𝑏2𝑏3

• So, we can represent any 𝐴𝒙 = 𝒃 equation as a vector equation.

• If 𝒃 is in the span of column vectors, the system has either ONE or INFINTEsolution.

• If 𝒃 is NOT in the span of column vectors, the system has NO solution. But, we can find projection of 𝒃 on the vector space, as best estimate.

Matrix equation (ONE solution)

• Example:

• 2𝑥 + 𝑦 = 5

4𝑥 − 2𝑦 = −2

• Row representation of above equation:

•2 14 −2

𝑥𝑦 =

5−2

13

𝑥

𝑦

Vector equation (ONE solution)

• Vector equation representation

of previous example:

• 𝑥24+ 𝑦

1−2

=5−2

• b is in the span of col1 and

col2.

• The only solution for this

equation is 𝑥 = 1, and 𝑦 = 3

dim1

dim2 𝒄𝒐𝒍1

𝒄𝒐𝒍2𝒃

Span of col1

Span of col2

Matrix equation (INFINITE solution)

• Example:

• 2𝑥 + 𝑦 = 34𝑥 + 2𝑦 = 6

• Row representation of above equation:

•2 14 2

𝑥𝑦 =

36

• Both equations are the same line

• All points on the line are solutions

𝑥

𝑦

Vector equation (INFINITE solution)

• Vector equation representation of previous example:

• 𝑥24+ 𝑦

12

=36

• 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2 are linearly dependent• 𝒃 is in the span of 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2

• 2𝑥 + 𝑦12

=36

• Any combination of 𝑥 and 𝑦 thatsatisfies 2𝑥 + 𝑦 = 3 is a solution.

dim1

dim2

𝒄𝒐𝒍1

𝒄𝒐𝒍2

𝒃

Span of col1 and Span of col2

Matrix equation (NO solution)

• Example:

• 2𝑥 + 𝑦 = 34𝑥 + 2𝑦 = 4

• Row representation of above equation:

•2 14 2

𝑥𝑦 =

34

• The lines are parallel.

• There is no solution.

𝑥

𝑦

Vector equation (NO solution)

• Vector equation representation

of previous example:

• 𝑥24+ 𝑦

12

=34

• 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2 are linearly dependent

• 𝒃 is NOT in the span of 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2• There is no solution

dim1

dim2

𝒄𝒐𝒍1

𝒄𝒐𝒍2

𝒃

Span of col1 and Span of col2

Vector equation (NO solution)

Hence, in order to solve thiswe project b onto the span of col1 and col2 using Least Squares

dim1

dim2

𝒄𝒐𝒍1

𝒄𝒐𝒍2

𝒃

Projection of b