1. Definition

An computer image is composed of small squares, called pixels. Each pixel has an associated set of attributes that define its color and opacity.

2. Terminology

• RGB - Red, Green, Blue
• CMYK - Cyan, Magenta, Yellow, and Black
• Colormap - In MATLAB, it means the list of allowed colors to which numbers in a matrix are "mapped" to (e.g., matrix elements with values in the range of 0.01 to 0.02 = black).
• Number representations - how numbers are represented internally in computer memory.
• uint8 - "unsigned 8-bit integer". Possible values are 000000002 through 111111112, corresponding to 0 through 28-1 = 255 in decimal.
• uint16 - "unsigned 16-bit integer". Possible values are 0 through 216-1 = 65535 in decimal.
• single - A single precision decimal number. Each number is represented with 32 bits.
• double - A double precision decimal number. Each number is represented with 64 bits. Allowed values range from 2.2251·10-308 to 1.7977·10+308.
• Bitdepth or Color Depth - The total number of bits used to represent the color of a pixel. If the amount of red in a pixel can have values of 0 through 255, eight bits are required. If the same is true of green and blue, then a total of 24 bits is required to specify the color of a pixel.
• Truecolor - Red, green, and blue can have a total of 256 possible values. The total number of color combinations is 224.
• Encoding - A rule for how a symbol or number is represented in binary. For example, the symbol "A" is 1000001 when encoded in 7-bit ASCII.
• Lossless compression - When an image is re-written to require fewer bits, but the original image can be uniquely recovered.
• Lossy compression - When an image is re-written to require fewer bits, but in such a way that the original image cannot be uniquely recovered.

3. Image File Formats

There are many different ways in which data for images are stored in a file. They all have different capabilities and limitations. All commonly used formats use binary - you can't just open the file in Notepad and read numbers. You need a special program to interpret the numbers.

• BMP - Bitmap Image File
• TIFF - Tagged Image File Format
• JPEG - Joint Photographic Experts Group
• PNG - Portable Network Graphics
• Interchange Format GIF - Graphics Interchange Format

The most common scientific format for images (typically astronomy or solar) is FITS.

4. Review

You should be able to answer most the problems listed at the bottom of the following two pages:

5. MATLAB Functions

5.1. Color Representations

MATLAB uses two black and white representations for displaying a matrix as an image:

• A black pixel is indicated by 0 and a white pixel by 255. Values (must be integers) in between are varying levels of grey.
• A black pixel is indicated by 0 and a white pixel by 1. Values (must be fractional) in between are varying levels of grey.

The functions that display images determine which representation to use based on the "class" of the matrix that is being plotted. If the matrix contains uint8 values, it assumes the first representation. If the matrix contains double precision values, it assumes the second representation.

uint8 numbers may only have the values 0, 1, ..., 255. uint8 numbers are represented internally as a patern of 8 bits.

double precision numbers may have fractional values between 2.2251·10-308 (2.2251e-308) and 1.79977·10308 (1.7977e+308). Double precision values are represented internally as pattern of 64 bits. (The allowed values in this range may be determined using the function eps. For example, the next possible double precision number after 2.0 is 2.0 + eps(2.0) = 2.0 + 1.4211e-14.)

Example:

 A = imread('cameraman.tif')
% shows that matrix contains uint8 numbers
% which have a min value of 0 and max value of 255.
whos A
% Open a figure window named "Figure 1"
figure(1)
% will show the image.  255 is assumed to mean "white".
imshow(A)
% Convert values in A to double precision values
B = double(A);
% Verify that values are double precision
whos B
% Open a figure window named "Figure 2"
figure(2)
% Will show a white square.  Because B contains
% double precision values, 1 means white.  But
% most values in B are above 1, so they are assumed
% to be 1.
imshow(B)
figure(3)
% Scale values in B by 255 so max value is 1.
imshow(B/255)
figure(4)
% Or, explicitly indicate what black and white are
imshow(B,[0,255])


MATLAB uses two color representations:

• r, g, b -> The amount of red, green, and blue is represented by three numbers between 0 and 1 (inclusive). The functions IMSHOW and IMAGESC with inputs of a 3-D matrix containing double precision numbers use this representation.
• R,G,B -> The amount of each color is represented by an integer from 0 through 255. The function IMSHOW uses this convention (as does IMAGESHOW and IMAGESC when the input is a 3-D matrix of uint8 and uint16 values). The list of possible colors are the 0,0,0 through 255,255,255 corresponding to 2563 = 224 ~ 16 million colors.

5.2. IMAGESC with 2-D input

The function IMAGESC takes an input of a 2-D matrix, computes the maximum and minimum values in the matrix, and splits this range into N colors, where N is the number of colors in the colormap. The SC stands for "scaled".

Imagine displaying the numbers in the matrix and then overlaying a tile by looking at the value of the number.

Example:

 C = [1,0,0;0,1,0]; % Allowed colors are red and green

% Matrix with max of 1 and min of 12.
% Low numbers will be associated with first row in colormap C.
% High numbers will be associated with last row in colormap C.
% By default, the matrix M will have elements that are double precision
% (even though we entered integers and not, for example, <code>1.0</code>, <code>2.0</code>, etc.
M = [ 1 2 3 4 ; 5 6 7 8 ; 9 10 11 12 ]

imagesc(M); % Imput is a 2-D matrix
colormap(C); % Matrix with three columns and any number of rows.
colorbar;
print -dpng imagesc2D.png


5.3. IMAGESC with 3-D input

Example:

clear;clf;
M(:,:,1) = [255 0 0;0 255 0;0 0 255]; % Red channel
M(:,:,2) = [0 255 0;255 0 255;0 0 0]; % Green channel
M(:,:,3) = [0 0 0;0 0 0;255 0 255];   % Blue channel

% Note how the class of the matrix is "double".  Even though
% numbers that looked like unit8 values were used, MATLAB
% assumes that the matrix has double precision values.
whos M

% imagesc(M)
% This above command imagesc(M) would result in the error:
% TrueColor CData contains element out of range 0.0 <= value <= 1.0
% Because M contains doubles, IMAGESC expect them to be in range of 0-1.

M = uint8(M); % Convert values of M to be of integer type

% Note how the total size of the matrix is now only 27 bytes.
whos M

% When the matrix is 3-D and the matrix has a class of uint8, IMAGESC
% assumes that the values represent the R,G,B intensities (and no colormap
% is needed).
imagesc(M);

print -dpng imagesc3Dint.png


5.4. IMSHOW with 2-D Input

For a greyscale image, IMSHOW takes an input of a 2-D matrix, where each element of the matrix is the amount of black in each position in the matrix.

I = imread('cameraman.tif'); % Greyscale image
whos I
%  Name        Size             Bytes  Class    Attributes
%
%  I         256x256            65536  uint8
imshow(I)
print -dpng cameraman.png


Experiment: Place a zero along all diagonal values of the array and display the image.

clear;
I = imread('cameraman.tif'); % Greyscale image
for i = 1:256
I(i,i) = 0;
end
imshow(I)
print -dpng cameraman_black_diagonal.png


5.5. IMSHOW with 3-D Input

For a color image, IMSHOW takes an input of a 3-D matrix. The first layer of the matrix specifies the amount of red at each position in the matrix. The second layer specifies the amount of blue at each position in the matrix. The third layer specifies the amount of green at each position in the matrix.

clear;
whos A
%  Name        Size                Bytes  Class    Attributes
%
%  A         648x306x3            594864  uint8
imshow(A)
print -dpng board.png


Experiment: Set the green intensity to zero.

clear;
figure(1)
% Show the original image
imshow(A);
% Save the original image as a PNG file
print -dpng board_original.png
% Display the red layer numbers
A(:,:,1)
% Display the green layer numbers
A(:,:,2)
% Set all values in green layer to zero
for i = [1:648] % Better: use 1:size(A,1)
for j = [1:306] % Better: use 1:size(A,2)
A(i,j,2) = 0;
end
end
% Display the new green layer numbers (should all be zero)
A(:,:,2)
figure(1)
% Show the modified image
imshow(A);
% Save the modified image as a PNG file
print -dpng board_modified.png

 Image:board.png board.png Image:board after.png board_after.png

5.6. IMAGEINFO

You may display the information about an image (its metadata) using the function IMAGEINFO.

>> imfinfo('cameraman_printed.tif')

ans =

Filename: 'cameraman_printed.tif'
FileModDate: '02-Sep-2014 17:19:20'
FileSize: 324148
Format: 'tif'
FormatVersion: []
Width: 1200
Height: 900
BitDepth: 24
ColorType: 'truecolor'
FormatSignature: [73 73 42 0]
ByteOrder: 'little-endian'
NewSubFileType: 0
BitsPerSample: [8 8 8]
Compression: 'PackBits'
PhotometricInterpretation: 'RGB'
StripOffsets: [450x1 double]
SamplesPerPixel: 3
RowsPerStrip: 2
StripByteCounts: [450x1 double]
XResolution: 150
YResolution: 150
ResolutionUnit: 'Inch'
Colormap: []
PlanarConfiguration: 'Chunky'
TileWidth: []
TileLength: []
TileOffsets: []
TileByteCounts: []
Orientation: 1
FillOrder: 1
GrayResponseUnit: 0.0100
MaxSampleValue: [255 255 255]
MinSampleValue: 0
Thresholding: 1
Offset: 320316
ImageDescription: 'MATLAB Handle Graphics'



Note that more recently, information such as the geographic location where the image was take is stored in the image. I took this photo from my office. Take a look at the GPS information using

% The ~ is a shortcut for my home directory name (e.g., /Users/weigel on my Mac and /home/weigel on my Linux machine).

Info =

FileModDate: '02-Sep-2014 18:08:57'
FileSize: 2009480
Format: 'jpg'
FormatVersion: ''
Width: 2592
Height: 1936
BitDepth: 24
ColorType: 'truecolor'
FormatSignature: ''
NumberOfSamples: 3
CodingMethod: 'Huffman'
CodingProcess: 'Sequential'
Comment: {}
Make: 'Apple '
Model: 'iPhone 4 '
Orientation: 6
XResolution: 72
YResolution: 72
ResolutionUnit: 'Inch'
Software: '7.1.2 '
DateTime: '2014:09:02 18:07:25 '
YCbCrPositioning: 'Centered'
DigitalCamera: [1x1 struct]
GPSInfo: [1x1 struct]

>> Info.GPSInfo

ans =

GPSLatitudeRef: 'N '
GPSLatitude: [3x1 double]
GPSLongitudeRef: 'W '
GPSLongitude: [3x1 double]
GPSAltitudeRef: 0
GPSAltitude: 133.0002
GPSTimeStamp: [3x1 double]
GPSImgDirectionRef: 'T '
GPSImgDirection: 174.9081

>> Info.GPSInfo.GPSLatitude

ans =

38.0000
49.0000
47.2000

>> Info.GPSInfo.GPSLongitude

ans =

77.0000
18.0000
23.9100



Latitudes are reported in terms of degrees, arcminutes, and arcseconds. Using the conversions 60 arcminute = 1 degree and 3600 arcseconds = 1 degree, gives

Latitude: 38+49/60+47.2/3600 = 38.829777777777778

Longitude: 77+18/60+23.91/3600 = 77.306641666666664

Covered here:

7. Averaging

Algorithm for averaging an array using a for loop:

clear;
A = [0 0 0 10 0 0 0];
s = 0;
for i = 1:length(A)
s = s + A(i);
end
a = s/length(A)


Alternatively, one can use the functions mean and sum:

clear
A = [0 0 0 10 0 0 0];
s = sum(A)
a = mean(A)


Algorithm for averaging a matrix using a for loop:

clear
M = [0 0 0 10 0 0 0;0 0 0 10 0 0 0];
for i = 1:size(M,1)
for j = 1:size(M,2)
s = s + M(i,j);
end
end
s
a = s/(size(M,1)*size(M,2))


Alternatively, one can use the functions mean and sum:

 s = sum( sum(M) )
a = mean( mean(M) )
% or
a = mean2(M)
% or (the notation M(:) and A(:) creates an array out of a matrix)
s = sum(M(:))
a = mean(A(:))


Note that sum(M) returns the sum of the columns and sum( sum(M) ) sums this result.

8. Smoothing

Smoothing refers to making transitions in an array or matrix less sharp. Consider the array

A = [0 0 0 10 0 0 0]


The most basic smoothing operation is to create a new array based on the original array by averaging nearest points: Element i of the new matrix is the average of the elements i-1, i, and i+1. Because the first and last element of the array does not have two nearest points, we set the first and last element of the new array to be equal to that of the old array. In this case

Anew(1) = A(1)
Anew(2) = (A(1) + A(2) + A(3))/3
Anew(3) = (A(2) + A(3) + A(4))/3
...
Anew(7) = A(7)


Written using a for, the code is

 A = [0 0 0 10 0 0 0]
Anew(1) = A(1);
Anew(7) = A(7);
% Could also do Anew = A;
% Note that the first i value is 2 and the last is length(A)-1.
% if i = 1:length(A) was used, the first iteration would result in
% an attempt to evaluate
% Anew(i) = (A(0) + A(1) + A(2))/3
% but A(0) is not defined, so the program will not run.
for i = 2:length(A)-1
Anew(i) = (A(i-1) + A(i) + A(i+1))/3;
end
Anew


The result is Anew =

        0  0  3.3333  3.3333  3.3333 0  0


The algorithm given above for an array can be extended for a matrix.

tmp = [0 0 0 10 0 0 0];
% M = repmat(M,5,1)
% or
M = [tmp;tmp;tmp;tmp;tmp]
% M is the original matrix.

% Create a copy of the original matrix.
Mnew = M;

% Here we use the function call
% size(M,1), which gives the number of rows
% and size(M,2) gives the number of columns.
for i = 2:size(M,1)-1
for j = 2:size(M,2)-1
Mnew(i,j) = (M(i-1,j) + M(i+1,j) + M(i,j-1) + M(i,j+1) + M(i,j))/5;
end
end
Mnew


Executing the code above displays

M =

0     0     0    10     0     0     0
0     0     0    10     0     0     0
0     0     0    10     0     0     0
0     0     0    10     0     0     0
0     0     0    10     0     0     0

Mnew =

0     0     0    10     0     0     0
0     0     2     6     2     0     0
0     0     2     6     2     0     0
0     0     2     6     2     0     0
0     0     0    10     0     0     0


9. Transforming

A black and white image is created from a 2-D matrix with values between 0 and 255 - each element in the matrix corresponds to a pixel and the value corresponds to the amount of white, with 0 corresponding to black and 255 corresponding to white.

To transform the image, one must just re-arrange the numbers in the matrix. MATLAB provides a few matrix re-arrangement functions, namely flipud (flip up down), fliplr (flip left right), and transpose.

A = [0,128,255;255,128,0;0,0,0]
A =
0    128   255
255   128    0
0     0     0

B = flipud(A) % First row becomes the last, and last row the first.
B =
0     0     0
255   128    0
0    128   255

C = fliplr(A) % First column becomes the last, and last column the first.
C =
255  128    0
0   128   255
0    0     0

D = transpose(A) % Matrix transpose.  Think rotating the matrix 90 degrees about its center.
D =
0   255   0
128   128   0
255    0    0


Although MATLAB and many other programming languages provide functions that allow you to do a matrix transform with only a function call, it is important to know how to implement basic transforms on your own. Most often, the transformation code will contain the following:

for i = 1:3
for j = 1:3
B(br,bc) = A(ar,ac);
end
end


and the task is to determine what to use in place of br,bc and ar,ac. If one wanted to simply make a copy of the matrix A, br,bc and ar,ac would both be i,j.

clear;
A = [1,2,3;4,5,6;7,8,9];
% Make B be a copy of A.
for i = 1:3
for j = 1:3
B(i,j) = A(i,j);
end
end


The process of determining br,bc and ar,ac for an operation like flipping a matrix upside-down starts with comparing the starting matrix with the ending matrix and making observations. After you have done this a number of times, you may find that you can make guesses for br,bc and ar,ac in the above code. Until then, here is an approach for determining their values.

Inspect the two matrices

A =
0    128   255
255   128    0
0     0     0
B =
0     0     0
255   128    0
0    128   255


First, let's look at the first row of B and note how its elements are related to A:

B(1,1) = A(3,1) % First row, first column of B equals third row, first column of A
B(1,2) = A(3,2) % First row, second column of B equals third row, second column of A
B(1,3) = A(3,3) % First row, third column of B equals third row, third column of A


The goal is to write this with a for loop. It appears that the second argument (corresponding to the column) for each is identical - the only difference is the first argument (corresponding to the row). To implement the above three statements in a for loop, we could write

for j = 1:3
B(1,j) = A(3,j);
end


The result is that we have figured out how to create the first row of B.

Next, consider the second row of B

B(2,1) = A(2,1)
B(2,2) = A(2,2)
B(2,3) = A(2,3)


These three statements can be re-written with a for loop as:

for j = 1:3
B(2,j) = A(2,j);
end


The third row of B is

B(3,1) = A(1,1)
B(3,2) = A(1,2)
B(3,3) = A(1,3)


This can be re-written with a for loop as:

for j = 1:3
B(3,j) = A(1,j);
end


Putting everything together, we have

clear;
A = [1,2,3;4,5,6;7,8,9]
for j = 1:3
B(1,j) = A(3,j);
end

for j = 1:3
B(2,j) = A(2,j);
end

for j = 1:3
B(3,j) = A(1,j);
end
B


When the above program is executed, B will be an inverted version of A. However, we are not finished. We wanted to write our program in this form:

for i = 1:3
for j = 1:3
B(br,bc) = A(ar,ac);
end
end


with br,bc and ar,ac depending on i and j. Here is the answer:

for i = 1:3
for j = 1:3
B(i,j) = A(4-i,j);
end
end


To see that this gives the same result, write out the nested for loop in long-hand notation to see that it results in the following:

B(1,1) = A(3,1)
B(1,2) = A(3,2)
B(1,3) = A(3,3)
B(2,1) = A(2,1)
B(2,2) = A(2,2)
B(2,3) = A(2,3)
B(3,1) = A(1,1)
B(3,2) = A(1,2)
B(3,3) = A(1,3)


10. Antialiasing

Antialiasing is used to make sharp transitions in colors appear smoother. For a black-and-white image, one algorithm is

If a black pixel has a white pixel above, below, to the left, or to the right, replace the black pixel with a grey pixel.
 Example: Which looks better? Top: Antialias and Hinting Middle: Antialias only Bottom: No antialias and no hinting

10.1. The problem

• The sharp edges make letters look "unpleasant" when zoomed out.

----- zoom out -----

From [1]

10.2. Eye trick

• Fill in places where black is near white with gray.
• Translated to a mathematical statment:
If a 0 is next to a 1, replace the 0 with a 0.5.

From [2].

10.3. Eye trick cont.

• Fill in places where black is near white with gray.
• The small antialiased letter is considered "more pleasant" by most readers.

---- zoom out -----

10.4. 1D Example

1D Antialias logic

If a black pixel has a white pixel to the left or to the right, replace the black pixel with a grey pixel.

Original (assume black = 0)

1  0   1   1   0   0   0   1   1   1


Antialiased

1 0.5  1   1  0.5  0  0.5  1   1   1


Code:

clear
A = [1  0   1   1   0   0   0   1   1   1];
for i = 2:length(A)-1
fprintf('Inspecting array value %d\n',i);
if (A(i) == 0) && (A(i-1) == 1 || A(i+1) == 1)
A(i) = 0.5;
end
end
A


10.5. 2D Example

If a black pixel has a white pixel above, below, to the left, or to the right, replace the black pixel with a grey pixel.

11. Dithering

Dithering is used to reduce the number of colors used in an image while keeping the image visually similar to the original image.

Consider a gray scale image with 256 shades of gray (corresponding to values 0-255). Suppose that you wanted to print or display the image using only black and white. How would you re-assign the color of each pixel? One approach would be to set all values of gray below 255/2 = 127.5 to 0 and all values above 127.5 to 255. It turns out that instead of using 127.5 as the threshold, using the average gray intensity value is better. (To understand why this is better, consider an image with all values below 127.5 - after dithering with this algorithm, all values will be zero.)

From [3],

The Average Dithering is a basic two-level algorithm for [creating a] halftone image. It consists in choosing a certain constant gray level, in particular the average value of image pixels, and using it as a global threshold in deciding whether a pixel should be quantized to 0 or to 1. All pixels whose intensity level lies above the average value (the threshold) are quantized to 1; all others get a value of 0.

See [4] for sample images that have been dithered.

1-D B/W Example of Average Dithering:

A black-and-white image can be thought of as a matrix with values of 0 through 255, where each value represents the darkness level of a square pixel. Before attempting to dither a 2-D image (or matrix), we will dither a 1-D array.

Consider the array

A = [0,128,144,222,255,19,22];


The first step in the dithering algorithm involves determining the "certain constant gray level". If a pixel has a gray level above this constant ("threshold") gray level, the pixel color is set to black. If the pixel has a gray level below this constant gray level, the pixel color is set to black.

We can find the average gray level (corresponding to the "certain constant gray level") of the matrix using either the function mean or using a for loop.

clear;
A = [0,128,144,222,255,19,22];
Amean = mean(A)


or

clear;
A = [0,128,144,222,255,19,22];
s = 0; % The sum variable.
for i = 1:length(A)
s = A(i) + s; % Add to s the previous value of s + value of element i of A.
end
Amean = s/length(A) % The average of A is the sum of its values


Next, we iterate over each element in array A and modify its value.

for i = 1:length(A)

if (A(i) < Amean)
A(i) = 0; % If the element i of array A is less than Amean, change it to 0.
else
A(i) = 255; % If the element i of array A is greather than or equal to Amean, change it to 255.
end

end
A % Display modified matrix


2-D (matrix) B/W Example of Average Dithering:

clear;

% Compute the mean of matrix A
Am = mean(A(:)); % or Am = mean(mean(A)) or Am = mean2(A)

figure(1);
imshow(A);                % Display the original image.
print -dpng cameraman.png % Save original image.

for i = 1:size(A,1)     % Iterate over rows
for j = 1:size(A,2) % Iterate over columns

if (A(i,j) < Am) % If row i and column j of matrix A is less than Amean
A(i,j) = 0;      % Set row i and column j of matrix A to 0.
else
A(i,j) = 255;    % Set row i and column j of matrix A to 0.
end

end
end

figure(2);
imshow(A); % Display the modified image.
print -dpng cameraman_dithered.png % Save the modified image.


Note that MATLAB's Image Toolbox contains the function dither. The following program does the same as the above:

A = imread('cameraman.tif');
figure(3);


2-D Color Dithering in General

The two-dimension example above serves as a simple example of how to do the most simple type of dithering. I a real application, there are two complications: (1) more colors are used, and (2) the image may contain colors. The following page contains a discussion of dithering in general using MATLAB: [5].

12. Histogram Equalization

The histogram of an image represents the number of pixels that have a certain color. For the black-and-white image cameraman.tif, the histogram can be viewed using

 I = imread('cameraman.tif'); % hist function requires elements that % are doubles, not uint8. I = double(I); figure(1)  % I(:) flattens the matrix I into  % an array, which is required by  % hist function. hist(I(:),[0:255]); xlabel('Pixel Value'); ylabel('Number of pixels in bin'); set(gca,'XLim',[0,255]) 

Histogram equalization involves a two-step process. First, the number levels of color, Nl are chosen. Then, pixel bin boundaries are selected such that there are an equal number of pixels in each bin on the histogram. The new pixel values are selected to start at 0 and go to 255 in increments of 255/(Nl-1). Any non-integer pixel value is rounded to the nearest integer.

For example, if the number of levels of colors was selected to be Nl = 2, we would need to find the pixel value for which an equal number of pixels in the histogram fell below and above (which is the median). The new pixel values start at 0 and the increment is 255/(2-1) = 255, so the list of new colors is [0, 255]. Any element below the median would be set to zero and any element above the median would be set to 255.

For three levels, The new pixel values start 0 and the increment is 255/(3-1) = 255/2, so the list of new colors is [0, 128, 255]. Any pixel with a value in the lower 1/3 of the distribution would be set to 0. Any pixel with a value in the next 1/3 of the distribution would be set to 128. Any pixel value in the upper 2/3 of the distribution would be set to 255.

13. Compression

There are two ways of compressing an image. The first is to represent the numbers in an alternative way. The second is to identify patterns in the sequence of numbers that represent the image.

13.1. Alternative Representation

Consider the list of numbers 3,2,3,3,2. To store these numbers as 7-bit ASCII, you would need to represent each number as a list of five 7-bit sequences:

0110011 0110010 0110011 0110011 0110010


One way of storing these numbers with fewer bits is to use the rule "a 3 corresponds to binary 1 and a 2 corresponds to a binary 0". Now the bit pattern to be stored is simply five bits:

10110


This is an example of lossless compression. Given the shorter bit pattern and encoding rule, I can exactly recover the original sequence.

Now consider the list of numbers 3,2,3,3,2.01. I could say that 2.01 is close enough to 2 and write the compressed version in the same way as above. However, given the pattern 10110 and the rule, I have no way of knowing that the last value was 2.01 or 2.02 or 1.99, etc. In this way, the compression was lossy - information about the original pattern was lost.

13.2. Using Patterns

Consider the list of numbers 0,0,0,0,0,0,0,1,1,1,1,1,1,1. They could be represented in 7-bit ASCII as

0110000 0110000 0110000 0110000 0110000 0110000 0110000 0110001 0110001 0110001 0110001 0110001 0110001 0110001


However, because there are only two numbers, 7-bit ASCII encoding is not needed. The numbers could be stored as a (14-bit) binary sequence (one could call this uint1) as

00000001111111


This is another example of an alternative representation. The encoding rule is that a binary zero means a decimal zero and a binary 1 means a decimal 1.

The above 14-bit sequence could be compressed more using an encoding rule. To communicate these numbers in words, you would say "7 0s and 7 1s" instead of stating every number in the sequence - in this case you are using a compression encoding rule. This rule can be implemented on a computer using the rule that the first bit corresponds to the number that is to be repeated, and the next three bits correspond to how many times that number is repeated. The new 8-bit binary pattern (spaces added for readability)

0 111 1 111


would then be interpreted as a 0 repeated 7 times and a 1 repeated 7 times. This new pattern was expressed with only 8 bits while the previous pattern required 14 bits.

Images tend to be highly compressible because they typically contain repeated patterns (large areas of black or blue, for example). The lossy binary image formats use special encoding rules to allow the images to be compressed without too much loss of information (when interpreted by the human eye).

In contrast to the examples above, the image compression rules that are used in practice are highly complex.

14. Problems

14.1. Formats

In the notes I mentioned that the commonly used image formats+ require special software for them to be read - you can't just open the file in a text editor or browser and see the numbers.

+ We call such complex formats "binary" and formats that you can read with a simple text editor or browser "ASCII". Technically, ASCII is stored in binary, so this is an unfortunate convention to distinguish between formats. More specifically, you would say "PNG file" and "ASCII file". The difference is how the numbers are encoded in binary. A PNG has a set of encoding rules that is much more complicated than the ASCII encoding rule.

Question:

List reasons why a more complex format ("binary") is preferred over a format which can be read without special software ("ASCII").

14.2. Formats

A file encoded in 8-bit ASCII contains the numbers

1.000000
2.000000
3.000000
4.000000

1. What is the size of the file? Check your answer using a text editor program like Notepad (Windows), TextWrangler (on a Mac), or GEdit (Linux).
2. If you copy these numbers into a word processing program (such as Microsoft Word) and save, why is the file size larger than that created by a text editor?
3. If you encoded these numbers as 64-bit doubles, what would the size of the file be? Would there be any loss of information when the file is saved?
4. If you encoded these numbers a unsigned 8-bit unsigned integers, what would the size of the file be? Would there be any loss of information when the file is saved?

14.3. Color Maps

Open the MATLAB editor and copy and paste the following commands (You do not need to delete the lines (comments) that start with a % character. MATLAB will ignore these lines).

% Specify the allowed colors
C = [1.0, 0.0, 0.0 ; 0.0, 1.0, 0.0 ; 0.0, 0.0, 1.0 ; 0.5, 0.5, 0.5 ; 0.0, 0.0, 0.0];

% Tell MATLAB to uses these colors
colormap(C);

% Create a matrix with two rows
M = [1 2 3 4; 1 2 3 4; 1 2 3 4; 1 2 3 4];

% Tell MATLAB to render the matrix M as an image
imagesc(M);

% Tell MATLAB to show a colorbar
colorbar;


1. Modify the colormap matrix C so that when you run the program, the only colors that are shown are red, green, blue and black. Enter your colormap matrix below.

2. Using the same colormap matrix used to answer the previous question, modify the values in the matrix M so that the middle two columns in the image are blue. Enter your matrix M below.

3. What is a colormap matrix, that when used with the matrix M to answer the previous question, would make the middle two columns red?

14.4. IMAGESC vs. IMSHOW

MATLAB has two functions that allow you to display an image, IMAGESC and IMSHOW. Write an example that highlights at least one difference and one commonality between the two functions.

14.5. Saving Images

Execute the following commands. Why is the size of matrix Ip different from that of I?

% Read an image
whos I % Display the image size.
imshow(I)
% "print" (write) the image to a file
print -dtiff cameraman_printed.tif

whos Ip % Display the image size.


14.6. Image Manipulation

In this problem, you will manipulate a black-and-white image.

The function size is used to obtain the number of rows and columns of a matrix.

The image camerman.tiff is an image that comes with MATLAB. (For future reference, to read other images,you would need to determine its location on your computer. For example, if you saved an image file named afile.tif to the directory /tmp, you would enter imread('/tmp/afile.tiff').)

The following program reads the contents of an image file into a matrix and then displays each matrix element.

clear;
whos I
size(I,1) % Will display 256
size(I,2) % Will display 256
for i = [1:size(I,1)] % Could have written [1:256]
for j = [1:size(I,2)] % Could have written [1:256]
I(i,j)
end
end
imshow(I)


Modify the above program so that the man's head is covered with a white square. Save your answers in a file named HW2_2.m.

14.7. Smoothing

Write a program that computes the average of the columns of a matrix and places the values in an array. For example, if the starting matrix is

1 2 3 4
1 2 3 4
1 2 3 4


the average array would be

1 2 3 4


14.8. Averaging an Array

Write a program that uses a for loop to determine

1. the sum of elements in A excluding values between -1 and 1, and
2. the average of A excluding values between -1 and 1.

Your code should work for an array A of any length and with any values.

Test your program using the following arrays

A = [1,-1,0,4,8,16,32]
A = [1.001, 2.0, 3.0, -1.000, 0, 3]


14.9. Smoothing an Array

Write a program that creates an array of 10 arbitrary values and then creates a new array for which the value of element i is the average of the values in the original array at i-1, i, and i+1. The first and last values of the new array should be equal to that in the original array.

14.10. Averaging a Matrix

Write a program using a double for loop that computes the sum and the average of all elements in the matrix M

M = [1,1,1,1 ; 2,2,2,2 ; 3,3,3,3 ; 4,4,4,4];


16*mean2(M)
% or
16*mean(mean(M))
% or
16*mean(M(:))


mean2(M)
% or
mean(mean(M))
% or
mean(M(:))


14.11. Averaging a Matrix

The following program creates a matrix and then replaces each matrix element with the average of itself and the elements to its left and right. If the element is in the first or last column, it is unchanged.

clear;
% Create a matrix
for i = 1:128
for j = 1:128
A(i,j) = rand(); % A random number between 0 and 1 inclusive.
end
end

figure(1); % Any plotting commands that follow will appear in a window named figure 1.
imshow(A)

As = A; % Copy original matrix

% Do averaging
for i = 1:128
for j = 2:127 % Can't start at j=1 because no element to left to use.
% When j = 2, calculation is As(i,2) = (A(i,1)+A(i,2)+A(i,3))/3
As(i,j) = 0.33*A(i,j-1) + 0.33*A(i,j) + 0.33*A(i,j+1);
end
end
% Visually inspect row 1, columns 1 through 4
A(1,1:4)
As(1,1:4)

figure(2);  % Any plotting commands that follow will appear in a window named figure 2.
imshow(As)


Write a program that applies this algorithm to the matrix created using A = imread('cameraman.tif') (Instead of creating a matrix using a for loop as in the example). Save your answer in a file named HW2_3.m.

14.12. Smoothing a Matrix

Write a MATLAB program using a for loop that computes the average of the columns of a matrix of arbitrary size and excludes values of 99999 from the computation of the average only if the value appears in an odd-numbered column.

For example,

1. The average of [0,99999,1;1 1 1] will be [0.5,50000,1].
2. The average of [99999,1,1;1 1 1] will be [1,1,1].

14.13. Manipulating a Matrix I

Create a color version of cameraman.tif using the algorithm:

• Any pixel with an original value that is less than 85 is colored red,
• Any pixel with an original value that is in range 85-171 is green, and
• Any pixel with an original value that is above 171 is blue.

Save your program as HW2_EC1.m. When I copy and paste the code into MATLAB, I should see the new image.

14.14. Histogram Equalization

Read Images#Histogram_Equalization and then try to answer the following question. This will be the extra credit question on the HW.

In the following program, the median of the matrix associated with cameraman.tif is calculated using the function quantile.

I = imread('cameraman.tif');
b = quantile(double(I(:)),0.5)


to compute the quartile boundaries, one could use

b = quantile(double(I(:)),[0.25 0.5 0.75])


Apply histogram equalization to cameraman.tif using four levels and a double for loop with one or more if statements. Use the four boundaries given above.

Compare your result to using MATLAB's implementation of histogram equalization using four levels:

histeq(I,4)


14.15. Manipulating an Image

The following program displays the first two elements of matrix I and then their sum.

I = imread('cameraman.tif');
I(1,1)
I(2,1)
I(1,1)+I(2,1)


Explain why the sum of these two elements is not 316. Save your answer in a file named HW2_EC2.m.

14.16. Manipulating a Matrix I

Write a program that creates a 10x10 matrix with arbitrary values and then creates a new matrix for which the value of element i, j is the average of the values in the original array at i-1, j, i, j, i+1, j, i, j+1, and i, j-1. The border values of the new matrix should be equal to that in the original matrix.

14.17. Transforming a Matrix I

Suppose that you wanted to create flip an image from left-to-right without using one of MATLAB's functions. That is, you want a program that will convert the matrix

1  2  3
4  5  6
7  8  9


to

3  2  1
6  5  4
9  8  7


Write a program that does this using one or more for loops. Follow the steps used in Images#Transforming for fliping an image up-side-down.

14.18. Transforming a Matrix II

Write a program that rotates a matrix 90 degrees clockwise.

14.19. Transforming a Matrix III

14.19.1. Part I

Write a program using one or more for loops that creates a matrix with the row order reversed from a starting matrix. (Note that one could do this without a loop using flipud, but you should not use this function.)

For example, if the original matrix is

1  2  3
4  5  6
7  8  9
0  0  1


the new matrix would be

0  0  1
7  8  9
4  5  6
1  2  3


Save your program as Midterm_2I.m. When I run your code, I should see the original matrix displayed and then the new matrix. Note that your code should work for a starting matrix of any size; make sure that your program gives the correct answer if I add or subtract rows and columns from your original matrix.

14.19.2. Part II

Write a program using one or more for loops that create a matrix that with the column order reversed from a starting matrix. (Note that one could do this without a loop using fliplr, but you should not use this function.)

For example, if the original matrix is

1  2  3  4
4  5  6  7
7  8  9  2
0  0  1  1


the new matrix would be

4  3  2  1
7  6  5  4
2  9  8  7
1  1  0  0


Save your program as Midterm_2II.m. When I run your code, I should see the original matrix displayed and then the new matrix. Note that your code should work for a starting matrix of any size; make sure that your program gives the correct answer if I add or subtract rows and columns from your original matrix.

14.20. Transforming a Matrix IV

Write a program that reverses the elements along the diagonal connecting the lower left corner of a matrix to the upper right corner of a matrix using one or more for loops. The program should work for any matrix with equal number of columns as rows with arbitrary values for the matrix elements. For example, if the starting matrix is

1   2  3  4
5   6  7  8
9  10 11 12
13 14 15 16


The elements along the specified diagonal are 13,10,7,4.

the transformed matrix will be

1   2  3 13
5   6 10  8
9   7 11 12
4  14 15 16


and the elements along the specified diagonal are now 4,7,10,13.

14.21. Transforming a Greyscale Image I

Read the greyscale image cameraman.tif using imread.

• Make every other row in the image white and display the image using imshow.
• Modify the image so that it appears "brighter" and display the image and display the image using imshow.
• Modify the image so that it appears "darker" and display the image and display the image using imshow.
• Plot the matrix created by imread using
I = imread('cameraman.tif')
C = colormap('jet');
colormap(C);
imagesc(I);


and explain what you see.

• Use one or more for loops to crop off the bottom half of the image.

14.22. Transforming a Greyscale Image II

MATLAB has several functions that allow you to rotate or mirror a matrix: FLIPLR, FLIPUD, and IMROTATE. Use these functions on cameraman.tif and display it rotated by 90 degrees and up-side-down. In addition, use one or more of these function to create this image.

Extra credit: Do this for a color image. You may use any color image, but it would be best to choose one without too much symmetry.

14.23. Transforming a Grayscale Image III

The following program reads the black-and-white image cameraman.tif into a 256x256 matrix and plots it.

clear;
% Read image cameraman.tif into a matrix I (cameraman.tif comes with MATLAB)
% Convert values to double precision (original are unit8)
% This is done so that math with matrix elements works as expected (e.g., 255+5 = 260)
% If we did not take this step, the result of adding elements with values of 255 and 5 would be 255.
I = double(I);
% Plot the matrix
colormap('gray');
image(I);
colorbar;


We want to reduce the number of pixels so that the new matrix is 128x128 using various approaches.

14.24. Approach I

Create a 128x128 matrix which contains every other row and every other column of the 256x256 matrix from cameraman.tif.

To test your code, use the 4x4 matrix

0 1 2 3
4 5 6 7
8 9 0 1
2 3 4 5


and verify that your code produces the reduced matrix

0 2
8 0


Save your program as Midterm_1I.m. When I execute this program, I should see a plot of the reduced cameraman image.

14.25. Approach II

Create a new matrix, Mnew, where column 1 is the average of the values in column 1 and 2 of the original matrix, column 2 is the average of columns 3 and 4 or the original matrix, etc.

Mnew will have 1/2 as many columns and the same number of rows as the original matrix.

To reduce the number of rows, create a new matrix, Mnew2, where row 1 is the average of rows 1 and 2 of Mnew, row 2 is the average of rows 3 and 4 of Mnew, etc.

To test your code, verify that if the original matrix is

0 1 2 3
4 5 6 7
8 9 0 1
2 3 4 5


then Mnew is

0.5 2.5
4.5 6.5
8.5 0.5
2.5 4.5


and Mnew2 is

2.5 4.5
5.5 2.5


Save your program as Midterm_1II.m. When I execute this program, I should see a plot of the reduced cameraman image.

14.26. Transforming a Color Image

Read the image http://bobweigel.net/cds301/images/Display_Image_2_01.png using imread.

Set all values corresponding to the level of green and blue to zero and display the image (using imshow). Explain why the border that was previously white is now red.

Using the answer to the previous question, show only pixels where the level of red is above 200 (out of 255).

Crop the image so that the (originally) white border no longer appears in the color image.

14.27. Antialiasing

One method of smoothing an image is called antialiasing (for a review, see [6]).

Copy the following program into a MATLAB script.

close all;
clear all;
n = 16;

for i = [1:n]
for j = [1:n]
if ( (i - n/2)^2 + (j - n/2)^2) <= (n/4)^2
M(i,j) = 0;
else
M(i,j) = 1;
end
end
end
C = [0,0,0;0.5,0.5,0.5;1,1,1];
figure(1);hold on;axis square;grid on;
imagesc(M);
colormap(C);
print -dpng circle_before.png

i = 8;
for j = [1:n-1]
if (M(i,j) == 0)
if (M(i,j+1) == 1)
M(i,j) = 0.5;
end
end
end
figure(2);hold on;axis square;grid on;
imagesc(M);
colormap(C);
print -dpng circle_after.png


By building on the above set of commands, write a program that converts the circle shown in Figure 1 to the circle shown in Figure 2 below.

Figure 1
Figure 2

14.28. Dithering I

1. Find the average of the matrix

A = [0,128,144,222,255,19,22;...
4,228,144,222,255,19,22;...
8,28,44,22,0,19,22;...
9,38,44,22,0,19,22;...
2,47,44,22,0,19,22;...
1,100,144,0,255,19,22;...
1,99,144,0,255,19,22;...
];


2. Dither this matrix using the "Average Dithering Algorithm".

14.29. Dithering II

Dither the image cameraman.tif using the following algorithm:

If any value is below 85, set it to zero. If any value is above 170, set it to 255. Set all other values to 127.

14.30. Dithering III

Read the greyscale image cameraman.tif using imread. Compute the mean of each column of the matrix. Dither the image according to the algorithm: "If a pixel in a given column is below the mean of the pixels in that column, set the value of the pixel to 0; otherwise set the value of the pixel to 255".

14.31. Histogram Equalization

Consider the median matrix

M = [0,128,129,127,0;...
0,124,123,122,0;...
0,128,129,127,0;...
0,125,127,127,0;...
0,128,128,127,0];

1. Compute the mean of the matrix.
2. Apply histogram equalization to the matrix using two levels.

14.32. Histogram Equalization

In the following program, the median of the matrix associated with cameraman.tif is calculated using the function quantile.

I = imread('cameraman.tif');
b = quantile(double(I(:)),0.5)


to compute the quartile boundaries, one could use

b = quantile(double(I(:)),[0.25 0.5 0.75])


Apply histogram equalization to cameraman.tif using four levels and a double for loop with one or more if statements.

Compare your result to using MATLAB's implementation of histogram equalization using four levels:

histeq(I,4)


14.33. Compression

Is verbal communication lossless or lossy? Give an example to justify your answer.

If verbal communication is lossy, is it generally more lossy than written communication?

14.34. Compression

M = imread('http://www.mathworks.com/matlabcentral/fileexchange/screenshots/6523/original.jpg');
whos M


Suppose that you want to store the mandrill image above on a disk, but the disk only has a capacity that is 1/2 of the size of the existing image.

In words, describe how you would modify the matrix associated with the image so that it required 1/2 the number of bits. Your goal is to modify the image in a way that it still looks similar to the original image. Is your method lossless or lossy?