Homework 1: Linear Algebra

CS 556-B: Mathematical Foundations of Machine Learning
Homework 1: Linear Algebra (100 points)
Note: Calculators allowed for trigonometric operations & arithmetic operations
(i.e., addition, subtraction, multiplication or division of scalars). All solutions methods must be full explained.
Vectors
.
1. (5 points) Find the magnitude of the vector x =






2
−3
2
4
−4






2. (5 points) Consider two vectors 
0
1

,

1
0

(in vector space R
2
), what is their span? Briefly explain your reasoning
leveraging the definition of the span of a set of vectors.
Dot Product
3. (10 points) If two vectors a, b have magnitudes 3 and 5 respectively and the angle between them is π
2
radians,
what is their dot product?
4. (10 points) Let vector u =

1
3

v =

−2
7

, calculate the dot product of u and v also calculate the angle
between (i.e., not the cosine of the angle but the actual angle in radians or degrees) u and v.
Linear Independence
5. (15 points) Check if the vectors x =


−1
0
2

, y =


3
−2
2

, z =


5
2
−6


are linearly independent. Note: The condition for linear independence is that given a set S of vectors x, y, z
and coefficients a, b, c, ax + by + cz = 0 if and only if a = b = c = 0.
6. (15 points) Given a subset of vectors S = {x1, x2, .., xk} for k ∈ N of a vector space V , prove that S is linearly
independent iff a linear combination of elements of S with non-zero coefficients does not yield 0. Hint: To
prove iff statements, i.e., A iff B (A ⇐⇒ B), first prove A → B, then prove B ← A.
Matrices
7. (10 points) Demonstrate the distributive property of matrix multiplication over addition.
Given A =

3 4
1 2
, B =

6 2
3 2
, C =

1 −2
4 −1

, demonstrate: A(B+C) = AB + AC
8. (15 points) Calculate the inverse of matrix A =


3 1 2
−2 −4 1
5 −3 2

. Note: It is acceptable to leave the final solution
with fractional entities in the matrix (i.e., no requirement to convert fractions to decimal numbers).
1
Change of Bases
9. (15 points) Consider the three columns in matrix A (problem 8) to be our new basis of interest in R3
. If a
vector x =


1
2
3

 defined on the natural basis in R3
(i.e., e1 =


1
0
0

, e2 =


0
1
0

, e1 =


0
0
1

), how would vector x
be represented in the basis defined by the matrix A in problem 8.
2