Lab 1: Introduction to Python

Lab 1: Introduction to Python
Welcome to the Spring 2021 offering of ECE 365: Data Science and Engineering.

In this course, we will be using Python 3 as our programming environment for our labs.

The labs will be distributed as Jupyter (IPython) Notebooks linked to a Python script. This is a nice way to organize Python code and visualize outputs. In the Machine Learning section of the course, only the Python script is required for submission.

For this course, we expect mathematical maturity at the level of Junior ECE students: familiarity with basic probability (at the level of ECE 313) and basic linear algebra (at the level of MATH 415, i.e. you know how to multiply matrices, transposes, determinants, equations in matrix form, etc.).

For the programming assignments, we do not expect you to be familiar with Python before the course. However, we expect you to have some knowledge of programming (i.e. you know what a function is, if statements, loops, etc.).

This lab will not be graded -- it serves to introduce you to Python. Do not turn in this lab.

Using Jupyter Notebooks
A Jupyter notebook consists of a collection of cells. Cells are blocks of code, Markdown (text with formatting, can use HTML) or headings. Generally, problem statements and directions will be written in cells made of Markdown and headings.

To insert a cell, use the Insert menu, and you can insert a cell above or below the current cell. To change the type of cell (code or Markdown), use the dropdown on the toolbar or go to Cell->Cell Type->(Desired Type of cell). A cell can be deleted by using the Edit menu.

To run a code cell, click on it and press the Play button (or hit Shift+Enter). You can also run all cells or run a cell from the cell menu. You can run the cells in different orders than they appear in the notebook (though for readability, you want your notebook to usually be executed from the top down). Note that code cells do support tab completion.

# This is an example of a code cell. You can put Python code here, and run it. # denotes the start of a comment in Python.
# The order in which you ran the cells will be put in bracket next to In [], such as In[2]
# and the corresponding output will appear below. 

# Example (run this):
print("Hello World!")

# Note that in Python 3, "print" is used as a function.
# In Python 2, you need to write
# print "Hello World!"
Hello World!
To edit a Markdown cell or Heading cell, double click on it. When you're done, press the Play button (or hit Shift+Enter) to display it with appropriate formatting.

Jupyter Notebooks have checkpoints built in (i.e. they save every few minutes automatically), but you can also save them using the save icon.

You can find many examples of Jupyter Notebooks here.

Introduction to Python
For this course, we will only be using a small subset of Python. A good tutorial for the parts of the Python language you will need is available at Software Carpentry (primarily sections 1,2,5,6). I encourage you to complete the Software Carpentry tutorial.

Another good resource is the UIUC CSE Workshops (particularly Introduction to Python, Numerical & Scientific Programming with Python and Plotting in Python).

If you want the whole nine yards, you can look at the official Python documentation or Learn Python the Hard Way.

Python is a common choice for big data work because it is easy to learn, has reasonable performance and there is a wide variety of packages available to do pretty much anything you want:

In the first part of the course, we will be using the following packages (libraries, modules):

NumPy - For matrices, vectors, mathematical operations on them, etc.
SciPy - More math functions (e.g. calculate distances between data points)
Scikit-Learn - Smorgasboard of machine learning tools (SVMs, K-Means, K-NN, PCA etc.)
Matplotlib - Easy way to make plots.
Basic Language Features
To load a library "numpy", you can use "import numpy". Then, functions in numpy can be accessed as numpy.function().

However, typing out the whole library name every time is annoying. So, we can use "as" to give it our own name.

import numpy as np
This loads numpy with the name np. If we want to calculate 10−−√ using numpy's square root function, we can now do

np.sqrt(10)
3.1622776601683795
Unlike C, Python does not require you to declare variables. To make a variable x with value 52, we can simply do

x=5**2
print(x)
25
Scopes of code are also determined by indentation. For example, the following C code, which uses a loop to print the numbers from 1 to 10:

int i;
printf("Let's print the numbers 1 to 10!\n");
for (i=1;i<=10;i++)
{
    printf("%i\n",i);
}
printf("Done!\n");
becomes

print("Let's print the numbers 1 to 10!")
for i in range(1,11):
    print (i)
print ("Done!")
Let's print the numbers 1 to 10!
1
2
3
4
5
6
7
8
9
10
Done!
Note that we did not need semicolons, and tab(=4 spaces) was used rather than braces to denote the body of the loop. range(n) gives a class which contains a list of numbers 0,1,…,n−1 and range(n,m) gives a class which contains a list of numbers n,n+1,…,m−2,m−1. The keyword in makes i take on all the values in range(1,11).

We can also do if-else statements. Consider the following C code:

x=2;
if (x==0)
{
    printf("zero\n");
} else if (x==1)
{
    printf("one");
} else 
{
    printf("not zero or one");
}
In Python, we use indentation rather than braces. Else if is also shortened to elif:

x=1
if x==0:
    print ("zero")
elif x==1:
    print ("one")
else:
    print ("not zero or one")
one
Here's a little exercise for you to try using a for loop and if statement. Note that logical and, or and not are given by and, or, not respectively in Python (rather than &&,||,! in C). The break and continue keywords work as in C. Modulus is %.

Exercise 1 (FizzBuzz): Prints the numbers from 1 to 100. For multiples of three, print “Fizz” instead of the number and for the multiples of five print “Buzz” instead. For numbers which are multiples of both three and five, print “FizzBuzz”.

#Put your code here
for i in range(1,101):
    if i%3==0 and i%5==0:
        print('FizzBuzz')
    elif i%3==0:
        print('Fizz')
    elif i%5==0:
        print('Buzz')
    else:
        print(i)
1
2
Fizz
4
Buzz
Fizz
7
8
Fizz
Buzz
11
Fizz
13
14
FizzBuzz
16
17
Fizz
19
Buzz
Fizz
22
23
Fizz
Buzz
26
Fizz
28
29
FizzBuzz
31
32
Fizz
34
Buzz
Fizz
37
38
Fizz
Buzz
41
Fizz
43
44
FizzBuzz
46
47
Fizz
49
Buzz
Fizz
52
53
Fizz
Buzz
56
Fizz
58
59
FizzBuzz
61
62
Fizz
64
Buzz
Fizz
67
68
Fizz
Buzz
71
Fizz
73
74
FizzBuzz
76
77
Fizz
79
Buzz
Fizz
82
83
Fizz
Buzz
86
Fizz
88
89
FizzBuzz
91
92
Fizz
94
Buzz
Fizz
97
98
Fizz
Buzz
Nesting loops works the same way that it does in C:

for row in range(1,5):
    for col in range(1,5):
        print ("(",row,",",col,") ",)
    print ("\n")
( 1 , 1 ) 
( 1 , 2 ) 
( 1 , 3 ) 
( 1 , 4 ) 


( 2 , 1 ) 
( 2 , 2 ) 
( 2 , 3 ) 
( 2 , 4 ) 


( 3 , 1 ) 
( 3 , 2 ) 
( 3 , 3 ) 
( 3 , 4 ) 


( 4 , 1 ) 
( 4 , 2 ) 
( 4 , 3 ) 
( 4 , 4 ) 


Most arithmetic works the same way as it does in C.

For Python 2, one thing "quirk" is in division: if both numbers are integers, the result will be an integer. While this is consistent with int/int or long/long in C, it can be confusing since we don't declare things to have a type in Python.

5/9 will result in 0 in Python 2.

5.0/9 will be 0.5555555555555556 in Python 2.

Try this example in Python 3:

5/9
0.5555555555555556
5.0/9
0.5555555555555556
5//9
0
Most useful codes consist of functions. These are declared with the def keyword. For example, lets say we didn't know that x2 could be calculated in Python as x**2 and we wanted to write a function square to do this:

def square(x):
    s=x*x
    return s
And we can call this function to calculate 42 as

square(4)
16
Another way to declare a function in Python is to use the lambda expressions:

also_a_square = lambda x : x**2
also_a_square(4)
16
One good thing about the lambda expressions is that you can pass these function around (however, not recommended for beginners). For example:

def pwr(n):
    return lambda x:x**n
f2 = pwr(2)
f2(4)
16
f3 = f2
f3(4)
16
For a comparison of regular functions and lambda expressions, see https://treyhunner.com/2018/09/stop-writing-lambda-expressions/#The_pros_and_cons_of_lambda.

While not necessary for the first part of the course, it may be useful to learn how functions treat their parameters in Python.

Now, we need to talk about lists. For example, we can have a list of numbers 0,…,5, a:

a=[0,1,2,3,4,5]
or more succinctly

a=list(range(6))
print(a)
[0, 1, 2, 3, 4, 5]
Lists have indices starting from 0.

print(a[0],a[1],a[4])
0 1 4
But you can also access them backwards with negative numbers

print(a[0],a[-1],a[-2])
0 5 4
Like MATLAB, you can slice a list with start:end:step (in MATLAB it is start:step:end). This gives you a list consisting of the elements start, start+step, start + 2 step, ..., up to end (not included!). Omitting step will give step=1. Omitting start and end will give the beginning and end of the array respectively. For example, to print the even indexed elements:

print(a[::2])
[0, 2, 4]
and to access a[1],a[2] (Key: end is not included!)

print(a[1:3])
[1, 2]
And you can modify the list through a slice

a[1:3]=[6,7]
print(a)
[0, 6, 7, 3, 4, 5]
Exercise 2: Write a function rev(x) which reverses an input list x (use a slice).

def rev(x):
    #Put your code here
    a=x[::-1]
    return a
x="hello world!" #strings aren't lists, but you can slice them. 
print(rev(x))
print(x)
!dlrow olleh
hello world!
Be careful with "copying" items in Python. Python uses something called deep and shallow copies. Roughly speaking, when you do

y=x
y points to the same thing as x. If x is a compound object (e.g. a list), then changing y can change x.

To avoid this, use

import copy
y=copy.copy(x) # shallow copy
y=copy.deepcopy(x) # deep copy
which makes a deep copy (or an appropriate copy constructor).

Note that is will return True if two variables point to the same object, == if the objects referred to by the variables are equal.

# Simple 
x=5
y=x
print ("x is y:", x is y)
y=2
print ("x=",x)
print ("y=",y)

# A list
x=[4,5,6]
y=x
print ("x is y:", x is y)
y[1]=2
print ("x=",x)
print ("y=",y)

# Copying a list with copy.copy
import copy
x=[4,5,6]
y=copy.copy(x)
print ("x is y:", x is y)
y[1]=2
print ("x=",x)
print ("y=",y)

# Copying a list with an appropriate constructor
x=[4,5,6]
y=list(x)
print ("x is y:", x is y)
y[1]=2
print ("x=",x)
print ("y=",y)
x is y: True
x= 5
y= 2
x is y: True
x= [4, 2, 6]
y= [4, 2, 6]
x is y: False
x= [4, 5, 6]
y= [4, 2, 6]
x is y: False
x= [4, 5, 6]
y= [4, 2, 6]
Finally, to help this jupyter notebook to connect to your Python script, you can use %run <script_name>. In all our labs, the linked script will be named as main.py. Do not change the file name!

%run main.py
<Figure size 432x288 with 0 Axes>
The following line creates an object from the class in main.py. Do not change the class name and function headers!

module = Lab1()
Note that if you change your function in main.py, you need to run the above two lines again to get an updated result.

The NumPy Library
NumPy is the library which gives us the numpy array, and along with its partner in crime, SciPy, gives us a bunch of linear algebra tools. A quick tutorial is available here. For those who are used to MATLAB, some common paradigms are translated from MATLAB to numpy here. I recommend you read this documentation. This is by no means an exhaustive list of functions you will be using from numpy in the course.

You can load the numpy library with the name np with

import numpy as np
which we did earlier.

The fundamental data type we will be using is the numpy array, which can represent vectors, matrices, tensors, etc. You can make a numpy array in several ways:

np.zeros( k ), np.ones(k) : make a vector of k zeros
np.zeros ( (k,l) ) , np.ones( (k,l) ): make a matrix with k rows and l columns
np.array( list ) : make a numpy array from list or matrix
np.arange (stop) : make a vector of numbers 0,1,...,stop-1 
np.copy(m) : make a copy of numpy array m
You can enter a matrix in as [[a,b],[c,d]] to get [acbd]. A vector can be entered as [a,b,c]. Some examples are given below.

a=np.zeros(5)
print ("a=",a)
m = np.ones( (3,2))
print ("m=",m)
v=np.array([1,2])
print ("v=",v)
n= np.array([[1,2],[3,4]])
print ("n=",n)
a= [0. 0. 0. 0. 0.]
m= [[1. 1.]
 [1. 1.]
 [1. 1.]]
v= [1 2]
n= [[1 2]
 [3 4]]
You can get the shape and size of an array using .shape and .size

print ("v=",v)
print ("v.shape=",v.shape,"v.size=",v.size)
print ("n=",n)
print ("n.shape=",n.shape,"n.size=",n.size)
v= [1 2]
v.shape= (2,) v.size= 2
n= [[1 2]
 [3 4]]
n.shape= (2, 2) n.size= 4
Note that the vector is specified as (length,). This is a useful (though counterintuitive) feature of numpy: it does not distinguish between row and column vectors, but automatically interprets the math by the way it makes sense. One can understand it better with array storage (see this answer).

You can also reshape an array using np.reshape.

onethrough6=np.arange(6)+1
print ("The Numbers 1-6 as a Vector:\n", onethrough6)
print ("The Numbers 1-6 as a matrix:\n", np.reshape(onethrough6,(2,3)))
print ("The Numbers 1-6 as another matrix:\n", np.reshape(onethrough6,(3,-1))) # -1 will determine the last dim. automatically
The Numbers 1-6 as a Vector:
 [1 2 3 4 5 6]
The Numbers 1-6 as a matrix:
 [[1 2 3]
 [4 5 6]]
The Numbers 1-6 as another matrix:
 [[1 2]
 [3 4]
 [5 6]]
We can transpose a matrix by using .transpose() (or simply .T). But it doesn't do anything for vectors.

print ("v=\n",v)
print ("v.transpose()=\n",v.transpose()) # this is just v, because numpy doesnt distinguish between row and column vectors
print ("n=\n",n )
print ("n.transpose()=\n",n.transpose()) # this is n^T
print ("n.T=\n",n.T)
v=
 [1 2]
v.transpose()=
 [1 2]
n=
 [[1 2]
 [3 4]]
n.transpose()=
 [[1 3]
 [2 4]]
n.T=
 [[1 3]
 [2 4]]
Array math is by default elementwise.

m1= np.array([[1,2],[3,4]])
m2= np.array([[5,6],[7,8]])
print ("m1=\n",m1)
print ("m2=\n",m2)
print ("m1+m2=\n",m1+m2)
print ("m1-m2=\n",m1-m2)
print ("m1/m2 elementwise:\n",m1/m2)
print ("m1*m2 elementwise:\n",m1*m2)
m1=
 [[1 2]
 [3 4]]
m2=
 [[5 6]
 [7 8]]
m1+m2=
 [[ 6  8]
 [10 12]]
m1-m2=
 [[-4 -4]
 [-4 -4]]
m1/m2 elementwise:
 [[0.2        0.33333333]
 [0.42857143 0.5       ]]
m1*m2 elementwise:
 [[ 5 12]
 [21 32]]
To do proper matrix multiplication, use dot. If A,B are matrices, A.dot(B) calculates AB. For a vector v (which mathematically we always assume is a column vector), and matrix A, vTA is calculated as v.dot(A) and Av is calculated as A.dot(v).

For two vectors v,w, the dot product is v.dot(w). For the outer product between them vwT, one must do np.outer(v,w). This is a bit cumbersome.

You can invert a matrix using np.linalg.inv(matrix).

One note on the numerical issue. Avoid inverting a matrix at best! Matrix inversion is much less stable than matrix multiplication. Most often, one is inverting a matrix when solving linear equations. In that case, an equation solver is much quicker (O(n2), compared with O(n3) using inversion and multiplication) and more accurate. See this post.

print ("m1*m2=\n", m1.dot(m2))
print ("m2*m1=\n", m2.dot(m1))
print ("v^T*m1=\n", v.dot(m1))
print ("m1*v=\n", m1.dot(v))

v=np.array([1,2])
w=np.array([4,7])
print ("v w^T\n", np.outer(v,w))
print ("inverse(m1)=\n",np.linalg.inv(m1))
m1*m2=
 [[19 22]
 [43 50]]
m2*m1=
 [[23 34]
 [31 46]]
v^T*m1=
 [ 7 10]
m1*v=
 [ 5 11]
v w^T
 [[ 4  7]
 [ 8 14]]
inverse(m1)=
 [[-2.   1. ]
 [ 1.5 -0.5]]
You can also slice and index numpy arrays.

print ("m1=\n",m1)
print ("m1[1,1]=\n",m1[1,1])
print ("m1[:,1]=\n",m1[:,1])
m1=
 [[1 2]
 [3 4]]
m1[1,1]=
 4
m1[:,1]=
 [2 4]
You can also index numpy arrays with boolean arrays or other arrays representing indices. Examples are given here. You can use this to write your k-Nearest Neighbors code compactly in Lab 2.

This list is not extensive -- if you think a function should exist in numpy, chances are it does. Check the documentation. For example, numpy.fliplr and numpy.flipud will flip an array left to right and up and down respectively.

Here are some exercises for you to get familiar with numpy array math (and plotting).

# This is a line magic -- a script which Jupyter uses to do nice things.  Line magics start with %
# This one will give us inline plotting in the notebook. Don't worry about the warnings in this case. 
%pylab inline
Populating the interactive namespace from numpy and matplotlib
Exercise 3: Solve Ax=b for x where A=[1324],b=[89] using matrix inversion. You can verify your solution by calculating Ax.

A=np.array([[1,2],[3,4]])
b=np.array([8,9])
res = module.solver(A,b)
print(res)
[-7.   7.5]
Another way is to use an linear equation solver, as below:

y = np.linalg.solve(A, b)
print(y)
[-7.   7.5]
Exercise 4 (Curve fitting): We are given a set of points {(xi,yi)}Ni=1, and want to find a straight line which fits these points well.

If the line is y=ax+b, we can write a system of equations

y1=ax1+b
y2=ax2+b
⋮
yn=axn+b
which we can write using matrices: ⎡⎣⎢⎢⎢⎢x1x2⋮xn11⋮1⎤⎦⎥⎥⎥⎥[ab]=⎡⎣⎢⎢⎢⎢y1y2⋮yn⎤⎦⎥⎥⎥⎥ (verify that this is correct on your own!).

# I'll give you a set of points
x=np.linspace(-5,5,100) # get a list of 100 evenly spaced points in (-5,5)
y=2*x+1+np.random.randn(100) # the true line is y=2x+1, but the data points are corrupted by some noise.
# Now we'll plot these
plot(x,y) # plot the data
xlim(-5,5) # set the horizontal limits
ylim(y.min(),y.max()) # set the vertical limits to the smallest and largest data points
(-11.264327647510312, 12.491708823929557)

Now, if the matrix ⎡⎣⎢⎢⎢⎢x1x2⋮xn11⋮1⎤⎦⎥⎥⎥⎥ was invertible, we'd just write [ab]=⎡⎣⎢⎢⎢⎢x1x2⋮xn11⋮1⎤⎦⎥⎥⎥⎥−1⎡⎣⎢⎢⎢⎢y1y2⋮yn⎤⎦⎥⎥⎥⎥.

But this doesn't make sense, since the matrix ⎡⎣⎢⎢⎢⎢x1x2⋮xn11⋮1⎤⎦⎥⎥⎥⎥ won't even be square normally.

As we will see in the linear regression section, a good solution is to use

[ab]≈⎡⎣⎢⎢⎢⎢x1x2⋮xn11⋮1⎤⎦⎥⎥⎥⎥†⎡⎣⎢⎢⎢⎢y1y2⋮yn⎤⎦⎥⎥⎥⎥ where † denotes the pseudoinverse, which acts like an inverse (in some sense) when you cannot invert a matrix. The pseudoinverse can be calculated using numpy.linalg.pinv with the same syntax as matrix inversion. numpy.column_stack may also be useful.

Use the pseudoinverse to calculate [ab]. Put your coefficients in a numpy array called coeff.

coeff = module.fitting(x,y)
print(coeff)
[1.99758778 1.14421745]
Now, lets see how the line fits up.

scatter(x,y) # scatter plot the data
xlim(-5,5) # set the horizontal limits
ylim(y.min(),y.max()) # set the vertical limits to the smallest and largest data points
plot(x,coeff[0]*x+coeff[1], '-r') # plot the best fit line
[<matplotlib.lines.Line2D at 0x15e2804c808>]

Now, we will talk a bit on code vectorization. The exercises will have (a) parts and (b) parts. You should be able to complete the (a) parts easily. The (b) parts are harder, but it is good (but not necessary) to know how to complete them).

In many cases, we recieve our feature vectors in the form of a matrix X where each row of X is a feature vector, i.e.
X=⎡⎣⎢⎢x11⋮xN1x12⋮xN2…⋮…x1d⋮xNd⎤⎦⎥⎥
or more compactly,
X=⎡⎣⎢⎢x⊤1⋮x⊤N⎤⎦⎥⎥
where xi=⎡⎣⎢⎢xi1⋮xid⎤⎦⎥⎥.

We often want to calculate quantities like Sx where S is some matrix of appropriate size.

# Lets first make a matrix S and some data
S=np.asarray([[2,1,3],[1,2,1]])
X=(np.arange(15)).reshape((5,3))
print ("S=")
print (S)
print ("X=")
print (X)
S=
[[2 1 3]
 [1 2 1]]
X=
[[ 0  1  2]
 [ 3  4  5]
 [ 6  7  8]
 [ 9 10 11]
 [12 13 14]]
# Now, lets calculate S x_i for each feature vector x_i and store this as a row in the matrix Sx
# in the naive way
def naiveSx(S,X):
    Sx = np.zeros((X.shape[0],S.shape[0]))
    for i in range(X.shape[0]):
        Sx[i]=np.dot(S,X[i])
    return Sx

print (naiveSx(S,X))
[[ 7.  4.]
 [25. 16.]
 [43. 28.]
 [61. 40.]
 [79. 52.]]
So, your loop will end up having N iterations, and doing a matrix * vector multiply in each one.

However, some simple linear algebra gives us a nice way to calculate them all in one shot with one matrix*matrix multiply.

You can easily check the following matrix multiplication is true:
XS⊤=⎡⎣⎢⎢x⊤1⋮x⊤N⎤⎦⎥⎥S⊤=⎡⎣⎢⎢x⊤1S⊤⋮x⊤NS⊤⎤⎦⎥⎥=⎡⎣⎢⎢(Sx1)⊤⋮(SxN)⊤⎤⎦⎥⎥
# Lets try calculating S x using this.
def matrixSx(S,x):
    return np.dot(X,S.T)
print (matrixSx(S,X))
[[ 7  4]
 [25 16]
 [43 28]
 [61 40]
 [79 52]]
Now, one may say, "Is there an advantage of doing one over the other?"

Well, lets see how they perform.

d=10
N=1000
S=np.random.randn(d,d) # We'll generate some random S (d x d) and data X (N x d)
X=np.random.randn(N,d)
%timeit naiveSx(S,X)
%timeit matrixSx(S,X)
1.09 ms ± 22 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
20.1 µs ± 115 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
You should see that the naive version (2.17 ms on my computer) is much slower with d=10, N=1000 than the matrix version (33.5 μs; a factor of 65x). And, with a bit of experience, the matrix multiply version will yield much more readable code.

In many cases, for fast, readable code, you'll want to use built-in operations which can operate on the data array (or large portions of it at one time), rather than iterating over each feature vector. Of course, you need to do some thinking and experimentation to determine what is faster at the end of the day (but don't caught up with premature optimization!).

Another quantity one may often want to calculate is x⊤Ay (which is a number) for some matrix A. The most important special case is that of a quadratic form Quadratic form, where A is some symmetric matrix and you want to calculate x⊤Ax.

Exercise 5: You will compute x⊤Ay for a collection of of vectors {xi} and {yj}, where A is of appropriate size such that the product is defined. You will receive the vectors in the form of two matrices:
X=⎡⎣⎢⎢x⊤1x⊤2⋮⎤⎦⎥⎥
Y=⎡⎣⎢⎢y⊤1y⊤2⋮⎤⎦⎥⎥
Your code will return a matrix such that the (i,j)-th entry will be x⊤iAyj.

Part (a): Calculate the matrix with (i,j)-th entry as x⊤iAyj by looping over the rows of X,Y.

Part (b): Repeat part (a), but using only matrix operations (no loops!).

Hint: You can do this with just numpy.dot and the transpose operations. Try writing out an appropriate matrix multiplication as above, and noting x⊤Ay=(x⊤A)y=(ATx)⊤y.

#Generate some random data
A=np.reshape(np.arange(15),(5,3))
Y=np.reshape(np.arange(30),(10,3))
X=np.reshape(np.arange(10),(2,5))
print (module.naive5(X,A,Y))
print (module.matrix5(X,A,Y))
print ("This should be small (about zero): ", np.sum((module.naive5(X,A,Y)-module.matrix5(X,A,Y))**2))

# Your result should be:
# [[  320  1220  2120  3020  3920  4820  5720  6620  7520  8420]
#  [  895  3370  5845  8320 10795 13270 15745 18220 20695 23170]]
[[  320.  1220.  2120.  3020.  3920.  4820.  5720.  6620.  7520.  8420.]
 [  895.  3370.  5845.  8320. 10795. 13270. 15745. 18220. 20695. 23170.]]
[[  320  1220  2120  3020  3920  4820  5720  6620  7520  8420]
 [  895  3370  5845  8320 10795 13270 15745 18220 20695 23170]]
This should be small (about zero):  0.0
Your result should be

[[  320  1220  2120  3020  3920  4820  5720  6620  7520  8420]
 [  895  3370  5845  8320 10795 13270 15745 18220 20695 23170]]

A related problem is to evaluate f(x)=x⊤Ax where A is a square matrix for a collection of vectors {xi}. You could take the result of Exercise 5 and extract the diagonal (e.g. using numpy.diag) and returning that. But, that is wasteful if you don't care about x⊤iAxj when i≠j.

Exercise 6: You will compute x⊤Ax for a collection of N vectors {xi} which are provided as a matrix X (as in Exercise 5). You should return a vector length N where the i-th entry is x⊤iAxi.

Part (a): Calculate a vector with i-th component x⊤iAxi by looping over the rows of X.

Part (b): Repeat part (a) using matrix operations (no loops!).

Hint: You can do this with the np.dot, elementwise multiplication and np.sum (along an axis) operations. The solution is easily found online, but try it first on your own.

A=np.reshape(np.arange(9),(3,3))
X=np.reshape(np.arange(30),(10,3))
print (module.naive6(X,A))
print (module.matrix6(X,A))
print ("This should be small:", np.sum((module.naive6(X,A)-module.matrix6(X,A))**2))

# Your result should be: [   60   672  1932  3840  6396  9600 13452 17952 23100 28896]
[   60.   672.  1932.  3840.  6396.  9600. 13452. 17952. 23100. 28896.]
[   60   672  1932  3840  6396  9600 13452 17952 23100 28896]
This should be small: 0.0
And this concludes Lab 1!