BME 590L: Homework 1

BME 590L: Homework 1

Please write-up solutions to all problems (however you’d like) and submit them in class
or via Github by the above due date. It is important that you show all of your steps - points
will be deducted for simply writing the answer without showing any intermediate steps on
how you arrived at your answer. You may work together to solve these problems, but please
write up your own answer in your own way.
Remember that there is a coding component to this homework assignment as well,
which can be found on Github in the corresponding folder: https://github.com/Ouwen/
BME-590-Machine-Learning-in-Imaging
Problem 1: Linear algebra refresher. We have two short column vectors,
u =




1
5
1
4




, v =




0
1
5
1




For these two vectors, please compute,
(a) Their inner product, u
T v
(b) Their outer product (tensor product), u ⊗ v = uvT
(c) Their Hadamard product (element-wise product), u ◦ v
Next, consider the following two complex vectors of the form Aeiθ:
a =




1
e
iπ/4
e
iπ/2
e
iπ/4




, b =




0
1e
iπ
5e
iπ
1e
−iπ




For these two complex vectors, please compute,
(d) Their inner product, a
Hb, where H denotes conjugate transpose
(e) The inner products, a
Ha and b
Hb
1
(f) The Hadamard product (element-wise product) of a ◦ a
∗ = kak
2
, where ∗ denotes
complex conjugate (no transpose)
(g) The Hadamard product (element-wise product) of b ◦ b
∗ = kbk
2
(h) Check to make sure that |a
Hb| ≤ kak2kbk2. Here, the single bars |x| mean to take the
absolute value of x, while the double bars denote what is called a “Euclidean norm”,
or an “L2 norm”, and are defined as kxk2 =
pPn
i=1 |x[i]|
2
. This is called the Schwarz
inequality and is a really useful rule of thumb.
Problem 2: Convolutions in 1D. Consider the following two short vectors (written out
as row vectors to save space):
u = [1, −1] (1)
v = [0, 1, 2, 3, 4, 5] (2)
(a) Compute the discrete convolution, w = u ∗ v. You should first try to do this by hand,
and then if you’d like, you can use a computer as an aid to check your result.
(b) The above convolution can be expressed as a matrix-vector product, w = Uv, where
the matrix U ∈ R
7×6 holds the blur kernel u. Write out U. What generalized operation
is it performing to the vector v? Note: I am looking for something besides a convolution
here - this particular convolution kernel is approximating a particular mathematical
operation that we perform all the time. It is helpful to ignore the “edge” effects to
establish what this particular operation of U on v is.
(c) One can also try to “undo” a convolution, which is called a deconvolution. This is
helpful, for example, to take blurry images (that have been already convolved with a
blur kernel) and to try to remove the effects of the blur, which can help create clearer
images. Determine the deconvolution matrix D that undoes the operation of U to
recover v from w. What generalized operation is D performing?
(d) (bonus problem) Is DDT
invertible? Why? What does its inverse represent?
Problem 3: Fourier transforms - gotta know these. Please sketch or plot the following functions, U(x), then write down what function Uˆ(fx) is the Fourier transform of each
U(x), and then sketch or plot Uˆ(fx) (Note: sketches can be rough):
(a) U(x) = rect(2x)
(b) U(x) = e
−x
2/4
(c) U(x) = δ(x − 3), where δ is the Dirac delta function
(d) U(x) = ReLU(x), where ReLU = max(0, x)
2
(e) Prove this important Fourier transform theorem:
F [U(x)V (x)] = F [U(x)] ∗ F [V (x)] (3)
where ∗ denotes convolution.
Problem 4: Quick and conceptual. For the following problems, please provide a sketch
and short explanation of the result. We are looking mostly for a correct conceptual understanding of these questions.
(a) Let’s consider an optical field U(x, y) that has passed through a camera lens and
has reached the plane right before an optical detector. As we’ll learn, the camera
lens will block a lot of the high spatial frequencies within U(x, y). Let’s assume that
at this plane, the maximum spatial frequency contained within U(x, y) is 0.1 µm−1
.
Furthermore, let’s assume that the image sensor that detects the incoming optical field
acts like a comb function, as we discussed in class, and has a pixel pitch of 10 µm
in both x and y (this means that the interval spacing of the comb sampling is 10
µm). Would would one be able to faithfully re-create the original field U(x, y) from
the detected field (i.e., discretized field)? Why or why not? How about if the pixel
pitch is 4 µm?
(b) Sketch the function f(x) = x
2
sin(x) on the interval x ∈ [−6, 6]. Please mark the local
minima and global minimum of f(x) on this interval. If we were to perform gradient
descent to determine the global minimum of this function, are we always guaranteed
to locate it? What would be a good value of x to use as an initial estimate to perform
gradient descent to find the global minimum? And what would be a bad initial value
of x?
(c) Finally, let’s consider the linear equation f(x) = sign
wT x


, where the column vector
w = [3, 2, 1]T
, and our two-dimensional variable x = [1, x1, x2]
T
. While x does have 3
coordinates, we’ll say it is two-dimensional because the first coordinate is fixed. “sign”
is an operator such that sign(x) = 1 if x > 0, sign(x) = 0 if x = 0, and sign(x) = −1 if
x < 0. Show that the regions of f(x) = +1 and f(x) = −1 are separated by a line, and
draw a sketch of the function f(x) in the coordinate space (x1, x2) (Note: you don’t
have to draw the exactly correct slope or anything, just a conceptual sketch will do).