Image Processing Lab

Image Processing Lab
All of the programming assignments are to be done in Python using additional libraries specified in the assignments. There are many libraries available, some of which we will be using, and you are welcome to use them with one exception: if the library or a function within it performs the specific function you are asked to code, you may not use that other than perhaps as a reference to compare against. All of the code you submit must be your own. You are welcome to turn in a completed jupyter notebook.

The following code will load an image you can use for this lab. If needed make sure to install PIL using pip install PIL or conda install PIL.

Pre-lab Notes
Since it is our first time working with color images, we need to be careful of a couple of things. Most RGB color images store each RGB value as an 8-bit number. This is fine for displaying images as shown below:

import matplotlib.pyplot as plt
import numpy as np

racoon = plt.imread('racoon.jpg')
plt.imshow(racoon); plt.show()

This is fine for displaying, but becomes a problem once we try to do any image manipulations. We have to beware of overflow. For example, lets say we just want to add 20 to all of the RGB values.

plt.imshow(racoon + 20)
plt.show()

Notice what happens near the white areas of the image. We only had 8-bits to respresent each RGB value (0 to 255). White areas will have values near 255. So when we add 20, the colors go crazy because values have overflowed (ex. 240 + 20 = 4).

You maybe tempted to try something like this:

plt.imshow(np.minimum(racoon + 20, 255))
plt.show()

Notice that this still doesn't work because the overflow occurs before the maximum check.

The way to beat this problem is to convert the image into a higher bit representation, do the manipulations, then convert it back down to the 8-bit representation.

racoon_32 = np.array(racoon, dtype=np.int32)
racoon_32 = np.minimum(racoon_32 + 20, 255)
racoon_8 = np.array(racoon_32, dtype=np.uint8)

plt.imshow(racoon_8)
plt.show()

For convenience, we will leave all data in int32 representation. Then, we will simply define a function that converts the image back to 8-bit representation before plotting.

def plotImage(image, title=""):
    im = np.array(image, dtype=np.uint8)
    plt.imshow(im, vmin = 0, vmax = 255)
    plt.title(title)
    plt.show()

racoon = plt.imread('racoon.jpg')
racoon = np.array(racoon, dtype=np.int32)
plotImage(racoon)

You are welcome to use the function above for plotting your own color images.

In this lab, you will also need to be able to convert between RGB values and HSB values. We have provided function that allow you to easily go back and forth while staying in the 0-255 representation for images. You are welcome to use the functions below in this lab.

def toHSB(image):
    from matplotlib import colors
    temp = 255*colors.rgb_to_hsv(image/255.0)
    return temp.astype(np.int32)
    
def toRGB(image):
    from matplotlib import colors
    temp = 255*colors.hsv_to_rgb(image/255.0)
    return temp.astype(np.int32)
Now that you have that understanding, you are ready to start the lab.

Implement each of the following functions. Use the provided test cases to test your functions.

Function 1: Convert to grayscale
Takes in a color image and returns a grayscale image using the following formula: Gray = 0.299 Red + 0.587 Green + 0.114 Blue

def toGrayScale(image):
    dcopy = np.array(image, dtype = np.int32)
    shape = np.shape(dcopy)
    rows, cols = shape[0], shape[1]
    for row in range(rows):
        for col in range(cols):
            pix = dcopy[row, col]
            dcopy[row, col] = 0.299*pix[0] + 0.587*pix[1] + 0.114*pix[2]
    return np.array(dcopy, dtype=np.uint8)
# Test Case
gray_racoon = toGrayScale(racoon)
plt.imshow(gray_racoon,cmap="Greys_r",vmin=0,vmax=255)
plt.title("Grayscale Racoon")
plt.show()

Function 2: Brightness Adjustment
Takes in a color image and returns the brightened version of that image according to a passed in parameter. Use a max image value of 255.

def pixelBrightAdjust(hsb_pixel, c):
    if c > 0:
        hsb_pixel[2] = 255 if hsb_pixel[2] > 255 - c else hsb_pixel[2] + c
    else:
        hsb_pixel[2] = 0 if hsb_pixel[2] < -1*c else hsb_pixel[2] + c

def brightAdjust(image,c):
    if c > 255:
        print("TO WTF")
    im = toHSB(image)
    # Your code here
    dcopy = im.copy() #not sure if "im" is a reference, and don't care to check
    
    shape = np.shape(dcopy)
    rows, cols = shape[0], shape[1]
    
    for i in range(rows):
        for j in range(cols):
            pixelBrightAdjust(dcopy[i, j], c)
    return toRGB(dcopy)
# Test Case
bright_racoon = brightAdjust(racoon,100)
plotImage(bright_racoon, "Bright Racoon")
dark_racoon = brightAdjust(racoon,-100)
plotImage(dark_racoon, "Dark Racoon")


Function 3: Contrast Adjustment
Takes in a color image and returns the contrasted version of that image according to a passed in parameter. Use a max image value of 255.

Also, rather than a straight linear operation, we will use a mapping similar to what Photoshop does. In particular, the contrast will be in the range [-100,100] where 0 denotes no change, -100 denotes complete loss of contrast, and 100 denotes maximum enhancement (8x multiplier). If c is the contrast parameter, then the level operation applied is:

$$s = \left(\frac{c+100}{100}\right)^4 (r-128) + 128$$
Make sure you work in floating point, not integers. Integer division would not be very acurate.

def pixelContrastAdjust(hsb_pixel, c):
    hsb_pixel[1] = int((((c + 100)/100.0)**4) * (hsb_pixel[1] - 128) + 128)

def contrastAdjust(image, c):
    im = toHSB(image)
    # Your code here
    dcopy = im.copy() #not sure if "im" is a reference, and don't care to check
    
    shape = np.shape(dcopy)
    rows, cols = shape[0], shape[1]
    
    for i in range(rows):
        for j in range(cols):
            pixelContrastAdjust(dcopy[i, j], c)
            
    return toRGB(np.array(dcopy, dtype=np.uint8))
contrast_racoon = contrastAdjust(racoon, 30)
plotImage(contrast_racoon, "High Contrast Racoon")

Function 4: Image Blending
Takes in 2 color images of the same size. Given an alpha value it returns the blended image according to the alpha value. Note that your alpha value can be a single number or a mask image of the same size. The alpha values will be between 0 and 1.

def alphaBlend(image1, image2, alpha = .5):
    return alpha*image1 + (1-alpha)*image2
# Test Cases
man = plt.imread("man.jpg")
city = plt.imread("city.jpg")
blended = alphaBlend(man, city, .7)
plotImage(blended, "Alpha Blend with Single Value")

mask1 = plt.imread("alphamask1.jpg")/255.0
blended1 = alphaBlend(man, city, mask1)
plotImage(blended1, "Alpha Blend with Mask 1")

beach = plt.imread("beach.jpg")
boat = plt.imread("boat.jpg")
mask2 = plt.imread("alphamask2.jpg")/255.0
blended2 = alphaBlend(boat, beach, mask2)
plotImage(blended2, "Alpha Blend with Mask 2")



Function 5: Cross Dissolve
Takes in 2 color images of the same size. Returns an array of alpha blend of those two images, where the first picture is an alpha value of 1, the last picture is an alpha value of 0, and the middle pictures slowly decrease until reaching zero. Allow the user to specify the number of steps in the cross dissolve. You can then feed this array into our animation function to view the cross dissolve.

def crossDissolve(image1, image2, numsteps = 10):
    blend = []
    
    for i in range(numsteps):
        alpha = (numsteps - i)*(1/numsteps)
        blend.append(alpha*image1 + (1-alpha)*image2)
    
    blend.append(alphaBlend(image1, image2, 0))
    return blend
#Test Case
import matplotlib.animation as animation

beach = plt.imread("beach.jpg")
boat = plt.imread("boat.jpg")
dis = crossDissolve(beach, boat)

fig = plt.figure()
ims = []
#doesn't work
# for im in dis:
#     im = np.array(im, dtype=np.uint8)
#     result = plt.imshow(im, vmin=0, vmax=255, animated=True)
#     ims.append([result])

# ani = animation.ArtistAnimation(fig, ims, interval=500, blit=True)
# plt.show()

for im in dis:
    plotImage(im)