Compute Variance Map of an Image

8 06 2009

Variance map of an image is calculated by taking a square window of a set size around a center pixel, and calculating the variance of the values of the pixels.

The variance within the window can be calculated from the following equation:
Variance Equation

Example

MonopolyLeftColorMonopoly-Left Image

MonopolyLeft Variance Map 5,9Monopoly-Left Image-Variance Map

(Pixels marked as black are low-texture regions)

MATLAB Code

% *************************************************************************
% Title: Function-Compute Variance map of the image
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Input Image (var: inputImage), Window Size (var: windowSize),
% Threshold (var: thresh) Typical value is 140
% Outputs: Variance Map (var: varianceImg) , Time taken (var: timeTaken)
% Example Usage of Function: [varianceImg, timeTaken]=funcVarianceMap('MonopolyLeftColor.png', 5, 9);
% *************************************************************************
function [varianceImg, timeTaken]=funcVarianceMap(inputImage, windowSize, thresh)
try 
    % Grab the image information (metadata) of input image using the function imfinfo
    inputImageInfo=imfinfo(inputImage);
    if(getfield(inputImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable inputImage
        inputImage=rgb2gray(imread(inputImage));
        % Convert the image from uint8 to double
        inputImage=double(inputImage);
    else if(getfield(inputImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            inputImage=imread(inputImage);
            % Convert the image from uint8 to double
            inputImage=double(inputImage);
        else
            error('The Color Type of Left Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    inputImage=inputImage;
end
% Find the size (columns and rows) of the input image and assign the rows to
% variable nr, and columns to variable nc
[nr,nc] = size(inputImage);
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Create an image of size nr and nc, fill it with zeros and assign
% it to variable meanImg
meanImg=zeros(nr, nc);
% Create an image of size nr and nc, fill it with zeros and assign
% it to variable varianceImg
varianceImg=zeros(nr, nc);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
win=(windowSize-1)/2;
tic; % Initialize the timer to calculate the time consumed.
% Compute a map of mean values
for(i=1+win:1:nr-win)
    for(j=1+win:1:nc-win)
        sum=0.0;
        for(a=-win:1:win)
            for(b=-win:1:win)
                sum=sum+inputImage(i+a,j+b);
            end
        end
        meanImg(i,j)=sum/(windowSize*windowSize);
    end
end
% Compute a map of variance values
for(i=1+win:1:nr-win)
    for(j=1+win:1:nc-win)
        sum=0.0;
        for(a=-win:1:win)
            for(b=-win:1:win)
                sum=sum+((inputImage(i+a,j+b)-meanImg(i,j))^2);
            end
        end         
        var=sum/((windowSize*windowSize)-1);
        % Apply threshold to produce a binarized variance map
        if (var > thresh)
            varianceImg(i,j) = 255;
        else
            varianceImg(i,j) = 0;
        end
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;





Correlation based similarity measure-Sum of Hamming Distances (SHD)

30 05 2009

Example 1: Tsukuba

TsukubaLeftLeft Image TsukubaRightRight Image
TsukubaSHD9x9DispRange=0-16ColorSHD Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

Example 2: Stereogram

StereogramLeftLeft Image StereogramRightRight Image
StereogramSHD9x9DispRange=0-16ColorSHD Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

MATLAB Code-SHD Right to Left Matching

% *************************************************************************
% Title: Function-Compute Correlation between two images using the 
% similarity measure of Sum of Hamming Distances (SHD) with Right Image 
% as reference.
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Left Image (var: leftImage), Right Image (var: rightImage),
% Window Size (var: windowSize), Minimum Disparity (dispMin), Maximum
% Disparity (dispMax)
% Outputs: Disparity Map (var: dispMap), Time taken (var: timeTaken)
% Example Usage of Function: [dispMap, timeTaken]=funcSHDR2L('TsukubaLeft.jpg', 'TsukubaRight.jpg', 9, 0, 16);
% *************************************************************************
function [dispMap, timeTaken]=funcSHDR2L(leftImage, rightImage, windowSize, dispMin, dispMax)
try 
    % Grab the image information (metadata) of left image using the function imfinfo
    leftImageInfo=imfinfo(leftImage);
    % Since SHDR2L is applied on a grayscale image, determine if the
    % input left image is already in grayscale or color
    if(getfield(leftImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable leftImage
        leftImage=rgb2gray(imread(leftImage));
    else if(getfield(leftImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            leftImage=imread(leftImage);
        else
            error('The Color Type of Left Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    leftImage=leftImage;
end
try
    % Grab the image information (metadata) of right image using the function imfinfo
    rightImageInfo=imfinfo(rightImage);
    % Since SHDR2L is applied on a grayscale image, determine if the
    % input right image is already in grayscale or color
    if(getfield(rightImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable rightImage
        rightImage=rgb2gray(imread(rightImage));
    else if(getfield(rightImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            rightImage=imread(rightImage);
        else
            error('The Color Type of Right Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    rightImage=rightImage;
end
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrLeft, and columns to variable ncLeft
[nrLeft,ncLeft] = size(leftImage);
% Find the size (columns and rows) of the right image and assign the rows to
% variable nrRight, and columns to variable ncRight
[nrRight,ncRight] = size(rightImage);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrLeft==nrRight && ncLeft==ncRight)
else
    error('Both left and right images should have the same number of rows and columns');
end
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Check whether minimum disparity is less than the maximum disparity.
if (dispMin>dispMax)
    error('Minimum Disparity must be less than the Maximum disparity.');
end
% Create an image of size nrLeft and ncLeft, fill it with zeros and assign
% it to variable dispMap
dispMap=zeros(nrLeft, ncLeft);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
win=(windowSize-1)/2;
numberOfBits=8;
tic; % Initialize the timer to calculate the time consumed.
for(i=1+win:1:nrLeft-win)
    for(j=1+win:1:ncLeft-win-dispMax)
        min=0;
        position=0;
        rightWindow=rightImage(i-win:i+win, j-win:j+win);
        for(dispRange=dispMin:1:dispMax)
            if (j+win+dispRange <= ncLeft)
                leftWindow=leftImage(i-win:i+win, j-win+dispRange:j+win+dispRange);
                bloc3=bitxor(rightWindow,leftWindow);
                distance=uint8(zeros(windowSize,windowSize));
                for (k=1:1:numberOfBits)
                    distance=distance+bitget(bloc3,k);
                end
                dif=sum(sum(distance));
                if (dispRange==0)
                    min=dif;
                elseif (min>dif)
                    min=dif;
                    position=dispRange;
                end
            end 
        end
        dispMap(i,j) = position;
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;





Correlation based similarity measure-Normalized Cross Correlation (NCC)

20 05 2009

Example 1: Tsukuba

TsukubaLeftLeft Image TsukubaRightRight Image
TsukubaNCC9x9DispRange=0-16ColorNCC Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

Example 2: Stereogram

StereogramLeftLeft Image StereogramRightRight Image
StereogramNCC9x9DispRange=0-16ColorNCC Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

MATLAB Code-NCC Right to Left Matching

% *************************************************************************
% Title: Function-Compute Correlation between two images using the 
% similarity measure of Normalized Cross Correlation (NCC) with Right Image 
% as reference.
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Left Image (var: leftImage), Right Image (var: rightImage),
% Window Size (var: windowSize), Minimum Disparity (dispMin), Maximum
% Disparity (dispMax)
% Outputs: Disparity Map (var: dispMap), Time taken (var: timeTaken)
% Example Usage of Function: [dispMap, timeTaken]=funcNCCR2L('StereogramLeft.jpg', 'StereogramRight.jpg', 9, 0, 16);
% *************************************************************************
function [dispMap, timeTaken]=funcNCCR2L(leftImage, rightImage, windowSize, dispMin, dispMax)
try 
    % Grab the image information (metadata) of left image using the function imfinfo
    leftImageInfo=imfinfo(leftImage);
    % Since NCCR2L is applied on a grayscale image, determine if the
    % input left image is already in grayscale or color
    if(getfield(leftImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable leftImage
        leftImage=rgb2gray(imread(leftImage));
        % Convert the image from uint8 to double
        leftImage=double(leftImage);
    else if(getfield(leftImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            leftImage=imread(leftImage);
            % Convert the image from uint8 to double
            leftImage=double(leftImage);
        else
            error('The Color Type of Left Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    leftImage=leftImage;
end
try
    % Grab the image information (metadata) of right image using the function imfinfo
    rightImageInfo=imfinfo(rightImage);
    % Since NCCR2L is applied on a grayscale image, determine if the
    % input right image is already in grayscale or color
    if(getfield(rightImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable rightImage
        rightImage=rgb2gray(imread(rightImage));
        % Convert the image from uint8 to double
        rightImage=double(rightImage);
    else if(getfield(rightImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            rightImage=imread(rightImage);
            % Convert the image from uint8 to double
            rightImage=double(rightImage);
        else
            error('The Color Type of Right Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    rightImage=rightImage;
end
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrLeft, and columns to variable ncLeft
[nrLeft,ncLeft] = size(leftImage);
% Find the size (columns and rows) of the right image and assign the rows to
% variable nrRight, and columns to variable ncRight
[nrRight,ncRight] = size(rightImage);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrLeft==nrRight && ncLeft==ncRight)
else
    error('Both left and right images should have the same number of rows and columns');
end
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Check whether minimum disparity is less than the maximum disparity.
if (dispMin>dispMax)
    error('Minimum Disparity must be less than the Maximum disparity.');
end
% Create an image of size nrLeft and ncLeft, fill it with zeros and assign
% it to variable dispMap
dispMap=zeros(nrLeft, ncLeft);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
win=(windowSize-1)/2;
tic; % Initialize the timer to calculate the time consumed.
for(i=1+win:1:nrLeft-win)
    for(j=1+win:1:ncLeft-win-dispMax)
        prevNCC = 0.0;
        bestMatchSoFar = dispMin;
        for(dispRange=dispMin:1:dispMax)
            ncc=0.0;
            nccNumerator=0.0;
            nccDenominator=0.0;
            nccDenominatorRightWindow=0.0;
            nccDenominatorLeftWindow=0.0;
            for(a=-win:1:win)
                for(b=-win:1:win)
                   nccNumerator=nccNumerator+(rightImage(i+a,j+b)*leftImage(i+a,j+b+dispRange));
                   nccDenominatorRightWindow=nccDenominatorRightWindow+(rightImage(i+a,j+b)*rightImage(i+a,j+b));
                   nccDenominatorLeftWindow=nccDenominatorLeftWindow+(leftImage(i+a,j+b+dispRange)*leftImage(i+a,j+b+dispRange));
                end
            end
            nccDenominator=sqrt(nccDenominatorRightWindow*nccDenominatorLeftWindow);
            ncc=nccNumerator/nccDenominator;
            if (prevNCC < ncc)
                prevNCC = ncc;
                bestMatchSoFar = dispRange;
            end
        end
        dispMap(i,j) = bestMatchSoFar;
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;





Correlation based similarity measure-Sum of Squared Differences (SSD)

20 05 2009

Example 1: Tsukuba

TsukubaLeftLeft Image TsukubaRightRight Image
TsukubaSSD9x9DispRange=0-16ColorSSD Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

Example 2: Stereogram

StereogramLeftLeft Image StereogramRightRight Image
StereogramSSD9x9DispRange=0-16ColorSSD Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

MATLAB Code-SSD Right to Left Matching

% *************************************************************************
% Title: Function-Compute Correlation between two images using the 
% similarity measure of Sum of Squared Differences (SSD) with Right Image 
% as reference.
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Left Image (var: leftImage), Right Image (var: rightImage),
% Window Size (var: windowSize), Minimum Disparity (dispMin), Maximum
% Disparity (dispMax)
% Outputs: Disparity Map (var: dispMap), Time taken (var: timeTaken)
% Example Usage of Function: [dispMap, timeTaken]=funcSSDR2L('StereogramLeft.jpg', 'StereogramRight.jpg', 9, 0, 16);
% *************************************************************************
function [dispMap, timeTaken]=funcSSDR2L(leftImage, rightImage, windowSize, dispMin, dispMax)
try 
    % Grab the image information (metadata) of left image using the function imfinfo
    leftImageInfo=imfinfo(leftImage);
    % Since SSDR2L is applied on a grayscale image, determine if the
    % input left image is already in grayscale or color
    if(getfield(leftImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable leftImage
        leftImage=rgb2gray(imread(leftImage));
        % Convert the image from uint8 to double
        leftImage=im2double(leftImage);
    else if(getfield(leftImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            leftImage=imread(leftImage);
            % Convert the image from uint8 to double
            leftImage=im2double(leftImage);
        else
            error('The Color Type of Left Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    leftImage=leftImage;
end
try
    % Grab the image information (metadata) of right image using the function imfinfo
    rightImageInfo=imfinfo(rightImage);
    % Since SSDR2L is applied on a grayscale image, determine if the
    % input right image is already in grayscale or color
    if(getfield(rightImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable rightImage
        rightImage=rgb2gray(imread(rightImage));
        % Convert the image from uint8 to double
        rightImage=im2double(rightImage);
    else if(getfield(rightImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            rightImage=imread(rightImage);
            % Convert the image from uint8 to double
            rightImage=im2double(rightImage);
        else
            error('The Color Type of Right Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    rightImage=rightImage;
end
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrLeft, and columns to variable ncLeft
[nrLeft,ncLeft] = size(leftImage);
% Find the size (columns and rows) of the right image and assign the rows to
% variable nrRight, and columns to variable ncRight
[nrRight,ncRight] = size(rightImage);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrLeft==nrRight && ncLeft==ncRight)
else
    error('Both left and right images should have the same number of rows and columns');
end
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Check whether minimum disparity is less than the maximum disparity.
if (dispMin>dispMax)
    error('Minimum Disparity must be less than the Maximum disparity.');
end
% Create an image of size nrLeft and ncLeft, fill it with zeros and assign
% it to variable dispMap
dispMap=zeros(nrLeft, ncLeft);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
win=(windowSize-1)/2;
tic; % Initialize the timer to calculate the time consumed.
for(i=1+win:1:nrLeft-win)
    for(j=1+win:1:ncLeft-win-dispMax)
        prevSSD = 65532;
        temp=0.0;
        bestMatchSoFar = dispMin;
        for(dispRange=dispMin:1:dispMax)
            ssd=0.0;
            for(a=-win:1:win)
                for(b=-win:1:win)
                    if (j+b+dispRange <= ncLeft)
                        temp=rightImage(i+a,j+b)-leftImage(i+a,j+b+dispRange);
                        temp=temp*temp;
                        ssd=ssd+temp;
                    end
                end
            end
            if (prevSSD > ssd)
                prevSSD = ssd;
                bestMatchSoFar = dispRange;
            end
        end
        dispMap(i,j) = bestMatchSoFar;
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;





Quality metrics in Stereo-Vision

12 05 2009

Root-Mean-Squared (RMS) Error

This measure is computed by the following formula:

RMS Error

where, N is the total number of pixels in the image, dc is the computed disparity map, and dt is the ground truth disparity map.

MATLAB Code

% *************************************************************************
% Title: Function-Compute Root Mean Squared (RMS) Error
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Computed Disparity Map (var: computedDisparityMap), Ground Truth Disparity Map (var: groundTruthDisparityMap),
% How many pixels to ignore at the border (var: borderPixelsToIgnore). For
% a 9x9 windowSize used, border pixels to ignore should be (9-1)/2 or 4
% pixels, Disparity range (var: dispRange), Scale factor for groundtruth
% (var: scale)
% Outputs: RMS Error (var: rmsError), Time taken (var: timeTaken)
% Example Usage of Function: [rmsError, timeTaken]= RMSError('TsukubaSAD9x9DispRange=0-16.png','TsukubaGroundTruth.png',4,16,8);
% *************************************************************************
function [rmsError, timeTaken]= funcRMSError(computedDisparityMap,groundTruthDisparityMap,borderPixelsToIgnore,dispRange, scale)
% Read an image using imread function, and assign it to variable
% computedDisparityMap
try 
    computedDisparityMap=imread(computedDisparityMap);
catch
    % if it is not an image but a variable
    computedDisparityMap=computedDisparityMap;
end
% Convert the image from uint8 to double
computedDisparityMap=double(computedDisparityMap);
% Read an image using imread function, and assign it to variable
% groundTruthDisparityMap
groundTruthDisparityMap=imread(groundTruthDisparityMap);
% Convert the image from uint8 to double
groundTruthDisparityMap=double(groundTruthDisparityMap);
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrComputedDisparityMap, and columns to variable ncComputedDisparityMap
[nrComputedDisparityMap,ncComputedDisparityMap] = size(computedDisparityMap);
% Find the size (columns and rows) of the image and assign the rows to
% variable nrGroundTruthDisparityMap, and columns to variable ncGroundTruthDisparityMap
[nrGroundTruthDisparityMap,ncGroundTruthDisparityMap] = size(groundTruthDisparityMap);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrComputedDisparityMap==nrGroundTruthDisparityMap && ncComputedDisparityMap==ncGroundTruthDisparityMap)
else
    error('Both Computed Disparity Map and Groundtruth Disparity Map images should have the same number of rows and columns');
end
numPixels=0;
rmsError=0.0;
tic; % Initialize the timer to calculate the time consumed.
% Calculate rms error
for (i=borderPixelsToIgnore:1:nrComputedDisparityMap-borderPixelsToIgnore)
    for(j=borderPixelsToIgnore:1:ncComputedDisparityMap-borderPixelsToIgnore-dispRange)
        if(groundTruthDisparityMap(i,j)~=0.0) % Ignore Pixels with unknown disparity in the groundTruthDisparityMap
            rmsError= rmsError+(abs((computedDisparityMap(i,j)*scale)-groundTruthDisparityMap(i,j))^2);
            numPixels=numPixels+1;
        end
    end
end
rmsError=sqrt(rmsError/numPixels);
% Stop the timer to calculate the time consumed.
timeTaken=toc;

Percentage of Bad Matching Pixels

This measure is computed by the following formula:

Percentage Bad Matching

where, N is the total number of pixels in an image, dc is the computed disparity map, and dt is the ground truth disparity map, and δ_thresh is the threshold for evaluating bad matched pixels (usually attains the value of 1.0).

MATLAB Code

% *************************************************************************
% Title: Function-Compute Percentage of bad matching pixels
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Computed Disparity Map (var: computedDisparityMap), Ground Truth Disparity Map (var: groundTruthDisparityMap),
% How many pixels to ignore at the border (var: borderPixelsToIgnore). For
% a 9x9 windowSize used, border pixels to ignore should be (9-1)/2 or 4
% pixels, Disparity range (var: dispRange), Threshold (var: thresh), Scale
% factor for groundtruth (var: scale)
% Outputs: Percentage Bad matching pixels (var: perBADMatch), Time taken
% (var: timeTaken)
% Example Usage of Function: [perBADMatch, timeTaken]=
% funcPercentBadMatchingPixels(dispMap,'VenusGroundTruthL2R.png',4,16,1,8)
% *************************************************************************
function [perBADMatch, timeTaken]= funcPercentBadMatchingPixels(computedDisparityMap,groundTruthDisparityMap,borderPixelsToIgnore,dispRange,thresh, scale)
% Read an image using imread function, and assign it to variable
% computedDisparityMap
try 
    computedDisparityMap=imread(computedDisparityMap);
catch
    % if it is not an image but a variable
    computedDisparityMap=computedDisparityMap;
end
% Convert the image from uint8 to double
computedDisparityMap=double(computedDisparityMap);
% Read an image using imread function, and assign it to variable
% groundTruthDisparityMap
groundTruthDisparityMap=imread(groundTruthDisparityMap);
% Convert the image from uint8 to double
groundTruthDisparityMap=double(groundTruthDisparityMap);
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrComputedDisparityMap, and columns to variable ncComputedDisparityMap
[nrComputedDisparityMap,ncComputedDisparityMap] = size(computedDisparityMap);
% Find the size (columns and rows) of the image and assign the rows to
% variable nrGroundTruthDisparityMap, and columns to variable ncGroundTruthDisparityMap
[nrGroundTruthDisparityMap,ncGroundTruthDisparityMap] = size(groundTruthDisparityMap);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrComputedDisparityMap==nrGroundTruthDisparityMap && ncComputedDisparityMap==ncGroundTruthDisparityMap)
else
    error('Both Computed Disparity Map and Groundtruth Disparity Map images should have the same number of rows and columns');
end
numPixels=0;
perBADMatch=0.0;
tic; % Initialize the timer to calculate the time consumed.
% Calculate Percentage Bad Matching Pixels
for (i=borderPixelsToIgnore:1:nrComputedDisparityMap-borderPixelsToIgnore)
    for(j=borderPixelsToIgnore:1:ncComputedDisparityMap-borderPixelsToIgnore-dispRange)
        if(groundTruthDisparityMap(i,j)~=0.0) % Ignore Pixels with unknown disparity in the groundTruthDisparityMap
            if(abs((computedDisparityMap(i,j)*scale)-groundTruthDisparityMap(i,j))>thresh)
                perBADMatch=perBADMatch+1;
            end
            numPixels=numPixels+1;
        end
    end
end
perBADMatch=perBADMatch/numPixels;
% Stop the timer to calculate the time consumed.
timeTaken=toc;





Correlation based similarity measures-Sum of Absolute Differences (SAD)

11 05 2009

Example 1: Tsukuba

TsukubaLeftLeft Image TsukubaRightRight Image
TsukubaSAD9x9DispRange=0-16ColorSAD Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

Example 2: Stereogram

StereogramLeftLeft Image StereogramRightRight Image
StereogramSAD9x9DispRange=0-16ColorSAD Disparity Map
Disparity Range: 0-16
Window Size: 9×9
The hotter the color, the closer it is; the cooler the color the farther it is.

MATLAB Code-SAD Left to Right Matching-Implementation 1

% *************************************************************************
% Title: Function-Compute Correlation between two images using the 
% similarity measure of Sum of Absolute Differences (SAD) with Left Image 
% as reference.
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Left Image (var: leftImage), Right Image (var: rightImage),
% Window Size (var: windowSize), Minimum Disparity (dispMin), Maximum
% Disparity (dispMax)
% Outputs: Disparity Map (var: dispMap), Time taken (var: timeTaken)
% Example Usage of Function: [dispMap, timeTaken]=funcSADL2R('StereogramLeft.jpg', 'StereogramRight.jpg', 9, 0, 16);
% *************************************************************************
function [dispMap, timeTaken]=funcSADL2R(leftImage, rightImage, windowSize, dispMin, dispMax)
try 
    % Grab the image information (metadata) of left image using the function imfinfo
    leftImageInfo=imfinfo(leftImage);
    % Since SADL2R is applied on a grayscale image, determine if the
    % input left image is already in grayscale or color
    if(getfield(leftImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable leftImage
        leftImage=rgb2gray(imread(leftImage));
        % Convert the image from uint8 to double
        leftImage=double(leftImage);
    else if(getfield(leftImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            leftImage=imread(leftImage);
            % Convert the image from uint8 to double
            leftImage=double(leftImage);
        else
            error('The Color Type of Left Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    leftImage=leftImage;
end
try
    % Grab the image information (metadata) of right image using the function imfinfo
    rightImageInfo=imfinfo(rightImage);
    % Since SADL2R is applied on a grayscale image, determine if the
    % input right image is already in grayscale or color
    if(getfield(rightImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable rightImage
        rightImage=rgb2gray(imread(rightImage));
        % Convert the image from uint8 to double
        rightImage=double(rightImage);
    else if(getfield(rightImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            rightImage=imread(rightImage);
            % Convert the image from uint8 to double
            rightImage=double(rightImage);
        else
            error('The Color Type of Right Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    rightImage=rightImage;
end
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrLeft, and columns to variable ncLeft
[nrLeft,ncLeft] = size(leftImage);
% Find the size (columns and rows) of the right image and assign the rows to
% variable nrRight, and columns to variable ncRight
[nrRight,ncRight] = size(rightImage);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrLeft==nrRight && ncLeft==ncRight)
else
    error('Both left and right images should have the same number of rows and columns');
end
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Check whether minimum disparity is less than the maximum disparity.
if (dispMin>dispMax)
    error('Minimum Disparity must be less than the Maximum disparity.');
end
% Create an image of size nrLeft and ncLeft, fill it with zeros and assign
% it to variable dispMap
dispMap=zeros(nrLeft, ncLeft);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
win=(windowSize-1)/2;
tic; % Initialize the timer to calculate the time consumed.
for(i=1+win:1:nrLeft-win)
    for(j=1+win+dispMax:1:ncLeft-win)
        prevSAD = 65532;
        temp=0.0;
        bestMatchSoFar = dispMin;
        for(dispRange=-dispMin:-1:-dispMax)
            sad=0.0;
            for(a=-win:1:win)
                for(b=-win:1:win)
                    if (j-win+dispRange > 0)
                        temp=leftImage(i+a,j+b)-rightImage(i+a,j+b+dispRange);
                        if(temp<0.0)
                            temp=temp*-1.0;
                        end
                        sad=sad+temp;
                    end
                end
            end
            if (prevSAD > sad)
                prevSAD = sad;
                bestMatchSoFar = dispRange;
            end
        end
        dispMap(i,j) = -bestMatchSoFar;
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;

MATLAB Code-SAD Right to Left Matching-Implementation 1

% *************************************************************************
% Title: Function-Compute Correlation between two images using the 
% similarity measure of Sum of Absolute Differences (SAD) with Right Image 
% as reference.
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Left Image (var: leftImage), Right Image (var: rightImage),
% Window Size (var: windowSize), Minimum Disparity (dispMin), Maximum
% Disparity (dispMax)
% Outputs: Disparity Map (var: dispMap), Time taken (var: timeTaken)
% Example Usage of Function: [dispMap, timeTaken]=funcSADR2L('StereogramLeft.jpg', 'StereogramRight.jpg', 9, 0, 16);
% *************************************************************************
function [dispMap, timeTaken]=funcSADR2L(leftImage, rightImage, windowSize, dispMin, dispMax)
try 
    % Grab the image information (metadata) of left image using the function imfinfo
    leftImageInfo=imfinfo(leftImage);
    % Since SADR2L is applied on a grayscale image, determine if the
    % input left image is already in grayscale or color
    if(getfield(leftImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable leftImage
        leftImage=rgb2gray(imread(leftImage));
        % Convert the image from uint8 to double
        leftImage=double(leftImage);
    else if(getfield(leftImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            leftImage=imread(leftImage);
            % Convert the image from uint8 to double
            leftImage=double(leftImage);
        else
            error('The Color Type of Left Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    leftImage=leftImage;
end
try
    % Grab the image information (metadata) of right image using the function imfinfo
    rightImageInfo=imfinfo(rightImage);
    % Since SADR2L is applied on a grayscale image, determine if the
    % input right image is already in grayscale or color
    if(getfield(rightImageInfo,'ColorType')=='truecolor')
    % Read an image using imread function, convert from RGB color space to
    % grayscale using rgb2gray function and assign it to variable rightImage
        rightImage=rgb2gray(imread(rightImage));
        % Convert the image from uint8 to double
        rightImage=double(rightImage);
    else if(getfield(rightImageInfo,'ColorType')=='grayscale')
    % If the image is already in grayscale, then just read it.        
            rightImage=imread(rightImage);
            % Convert the image from uint8 to double
            rightImage=double(rightImage);
        else
            error('The Color Type of Right Image is not acceptable. Acceptable color types are truecolor or grayscale.');
        end
    end
catch
    % if it is not an image but a variable
    rightImage=rightImage;
end
% Find the size (columns and rows) of the left image and assign the rows to
% variable nrLeft, and columns to variable ncLeft
[nrLeft,ncLeft] = size(leftImage);
% Find the size (columns and rows) of the right image and assign the rows to
% variable nrRight, and columns to variable ncRight
[nrRight,ncRight] = size(rightImage);
% Check to see if both the left and right images have same number of rows
% and columns
if(nrLeft==nrRight && ncLeft==ncRight)
else
    error('Both left and right images should have the same number of rows and columns');
end
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Check whether minimum disparity is less than the maximum disparity.
if (dispMin>dispMax)
    error('Minimum Disparity must be less than the Maximum disparity.');
end
% Create an image of size nrLeft and ncLeft, fill it with zeros and assign
% it to variable dispMap
dispMap=zeros(nrLeft, ncLeft);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
win=(windowSize-1)/2;
tic; % Initialize the timer to calculate the time consumed.
for(i=1+win:1:nrLeft-win)
    for(j=1+win:1:ncLeft-win-dispMax)
        prevSAD = 65532;
        temp=0.0;
        bestMatchSoFar = dispMin;
        for(dispRange=dispMin:1:dispMax)
            sad=0.0;
            for(a=-win:1:win)
                for(b=-win:1:win)
                    if (j+b+dispRange <= ncLeft)
                        temp=rightImage(i+a,j+b)-leftImage(i+a,j+b+dispRange);
                        if(temp<0.0)
                            temp=temp*-1.0;
                        end
                        sad=sad+temp;
                    end
                end
            end
            if (prevSAD > sad)
                prevSAD = sad;
                bestMatchSoFar = dispRange;
            end
        end
        dispMap(i,j) = bestMatchSoFar;
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;

After running the above code (Implementation 1-Left to Right matching) on Tsukuba images (384×288 image resolution, disparity range [0,16], window size: 9×9), on my machine-Dell Studio XPS 435T, Intel Core i7-920 CPU, 2.67GHz, 9.00GB RAM, Windows Vista OS, MATLAB R2008a, i get an average run time of about 6.587 seconds.





Census Transform in Image Processing

10 05 2009

Census Transform is a form of non-parametric local transform (i.e. relies on the relative ordering of local intensity values, and not on the intensity values themselves) used in image processing to map the intensity values of the pixels within a square window to a bit string, thereby capturing the image structure [1]. The centre pixel’s intensity value is replaced by the bit string composed of set of boolean comparisons such that in a square window, moving left to right,

If (CurrentPixelIntensity<CentrePixelIntensity) boolean bit=0
else boolean bit=1

For each comparison the bit is shifted to the left, forming an 8 bit string for a census window of size 3×3 and a 32 bit string for a census window of size 5×5.

Advantages

  1. Reduces effects of variations caused by camera’s gain and bias.
  2. Increase in robustness to outliers near depth-discontinuities.
  3. Encodes local spatial structure.
  4. Tolerant to factionalism (If a minority of pixels in a local neighborhood has a very different intensity distribution than the majority, only comparisons involving a member of the minority are affected).
  5. It can distinguish between rotations and reflections

Disadvantages

  1. Loss of information associated with the pixel.
  2. As the size of the window increases, so does the requirement of the variable’s size. For a census window of 3×3, the variable used to store the census value would be of size 23, or 8 bits; for a census of window size 5×5 the number of bits required to store census value would be 25 or 32 bits.

Example

Cones Sample Image-Cones

censusCones3x3Census Transform (3×3) censusCones5x5Census Transform (5×5)

MATLAB Code

% *************************************************************************
% Title: Function-Census Transform of a given Image
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Image (var: inputImage), Window size assuming square window (var:
% windowSize) of 3x3 or 5x5 only.
% Outputs: Census Tranformed Image (var: censusTransformedImage), 
% Time taken (var: timeTaken)
% Example Usage of Function: [a,b]=funcCensusOneImage('Img.png', 3)
% *************************************************************************
function [censusTransformedImage, timeTaken] = funcCensusOneImage(inputImage, windowSize)
% Grab the image information (metadata) using the function imfinfo
try
imageInfo=imfinfo(inputImage);
% Since Census Transform is applied on a grayscale image, determine if the
% input image is already in grayscale or color
if(getfield(imageInfo,'ColorType')=='truecolor')
% Read an image using imread function, convert from RGB color space to
% grayscale using rgb2gray function and assign it to variable inputImage
inputImage=rgb2gray(imread(inputImage));
else if(getfield(imageInfo,'ColorType')=='grayscale')
% If the image is already in grayscale, then just read it.        
inputImage=imread(inputImage);
else
error('The Color Type of Input Image is not acceptable. Acceptable color types are truecolor or grayscale.');
end
end
catch
inputImage=inputImage;
end
% Find the size (columns and rows) of the image and assign the rows to
% variable nr, and columns to variable nc
[nr,nc] = size(inputImage);
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
error('The window size must be an odd number.');
end
if (windowSize==3)
bits=uint8(0);
% Create an image of size nr and nc, fill it with zeros and assign
% it to variable censusTransformedImage of type uint8
censusTransformedImage=uint8(zeros(nr,nc));
else if (windowSize==5)
bits=uint32(0);
% Create an image of size nr and nc, fill it with zeros and assign
% it to variable censusTransformedImage of type uint32        
censusTransformedImage=uint32(zeros(nr,nc));
else
error('The size of the window is not acceptable. Just 3x3 and 5x5 windows are acceptable.');
end
end
% Initialize the timer to calculate the time consumed.
tic;
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel
C= (windowSize-1)/2;
for(j=C+1:1:nc-C) % Go through all the columns in an image (minus C at the borders)
for(i=C+1:1:nr-C) % Go through all the rows in an image (minus C at the borders)  
census = 0; % Initialize default census to 0
for (a=-C:1:C) % Within the square window, go through all the rows
for (b=-C:1:C) % Within the square window, go through all the columns
if (~(a==0 && b==0)) % Exclude the centre pixel from the calculation
census=bitshift(census,1); %Shift the bits to the left  by 1
% If the intensity of the neighboring pixel is less than
% that of the central pixel, then add one to the bit
% string
if (inputImage(i+a,j+b) < inputImage(i,j))
census=census+1;
end
end
end
end
% Assign the census bit string value to the pixel in imgTemp
censusTransformedImage(i,j) = census;
end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;

In order to run the above code, you should have the image processing toolbox installed in MATLAB.

References

[1] R. Zabih and J. Woodfill, “Non-parametric Local Transforms for Computing Visual Correspondence”, Proceedings of the third European conference on Computer Vision (Vol. II)





Rank Transform in Image Processing

8 05 2009

Rank Transform is a form of non-parametric local transform (i.e. relies on the relative ordering of local intensity values, and not on the intensity values themselves) used in image processing to rank the intensity values of the pixels within a square window. The centre pixel’s intensity value is replaced by its rank amongst the neighboring pixels [1].

Advantages

  1. Reduces effects of variations caused by camera’s gain and bias.
  2. Increase in robustness to outliers near depth-discontinuities.

Disadvantages

  1. Loss of information associated with the pixel.
  2. Rank transform cannot distinguish between rotations and reflections, and has been shown to produce the same rank for variety of patterns [2].

Example

Cones  Sample Image-Cones

RankCones3x3Rank Transform (3×3)

RankCones5x5Rank Transform (5×5)
RankCones7x7Rank Transform (7×7) RankCones9x9Rank Transform (9×9) RankCones15x15Rank Transform (15×15)

MATLAB Code

% *************************************************************************
% Title: Function-Rank Transform of a given Image
% Author: Siddhant Ahuja
% Created: May 2008
% Copyright Siddhant Ahuja, 2008
% Inputs: Image (var: inputImage), Window size assuming square window (var:
% windowSize)
% Outputs: Rank Tranformed Image (var: rankTransformedImage), 
% Time taken (var: timeTaken)
% Example Usage of Function: [a,b]=funcRankOneImage('Img.png', 3)
% *************************************************************************
function [rankTransformedImage, timeTaken] = funcRankOneImage(inputImage, windowSize)
% Grab the image information (metadata) using the function imfinfo
imageInfo=imfinfo(inputImage);
% Since Rank Transform is applied on a grayscale image, determine if the
% input image is already in grayscale or color
if(getfield(imageInfo,'ColorType')=='truecolor')
% Read an image using imread function, convert from RGB color space to
% grayscale using rgb2gray function and assign it to variable inputImage
    inputImage=rgb2gray(imread(inputImage));
else if(getfield(imageInfo,'ColorType')=='grayscale')
% If the image is already in grayscale, then just read it.        
        inputImage=imread(inputImage);
    else
        error('The Color Type of Input Image is not acceptable. Acceptable color types are truecolor or grayscale.');
    end
end
% Check the size of window to see if it is an odd number.
if (mod(windowSize,2)==0)
    error('The window size must be an odd number.');
end
% Initialize the timer to calculate the time consumed.
tic;
% Find the size (columns and rows) of the image and assign the rows to
% variable nr, and columns to variable nc
[nr,nc] = size(inputImage);
% Create an image of size nr and nc, fill it with zeros and assign
% it to variable rankTransformedImage
rankTransformedImage = zeros(nr,nc);
% Find out how many rows and columns are to the left/right/up/down of the
% central pixel based on the window size
R= (windowSize-1)/2;
for (i=R+1:1:nr-R) % Go through all the rows in an image (minus R at the borders)
    for (j=R+1:1:nc-R) % Go through all the columns in an image (minus R at the borders)     
        rank = 0; % Initialize default rank to 0
        for (a=-R:1:R) % Within the square window, go through all the rows
            for (b=-R:1:R) % Within the square window, go through all the columns
                % If the intensity of the neighboring pixel is less than
                % that of the central pixel, then increase the rank
                if (inputImage(i+a,j+b) < inputImage(i,j))
                    rank=rank+1;
                end     
            end
        end
        % Assign the rank value to the pixel in imgTemp
        rankTransformedImage(i,j) = rank;
    end
end
% Stop the timer to calculate the time consumed.
timeTaken=toc;

In order to run the above code, you should have the image processing toolbox installed in MATLAB.

Experiment with Varying Patterns

 

TestPattern1

Test Pattern 1

 

TestPattern2

Test Pattern 2

 

TestPattern3

Test Pattern 3

 

TestPattern4

Test Pattern 4

 

TestPattern5

Test Pattern 5

 

TestPattern6

Test Pattern 6

 All of the above patterns yield  the following rank result:

TestPatternRankResult

with a rank of 4.

Thus proving that Rank transform cannot distinguish between rotations and reflections, and has been shown to produce the same rank for variety of patterns [2]. More of these patterns can be generated to emphasize the point.

References

[1] R. Zabih and J. Woodfill, “Non-parametric Local Transforms for Computing Visual Correspondence”, ECCV 2004, pp. 151-158

[2] J. Banks, P. Corke, “Quantitative Evaluation of Matching Methods and Validity Measures for Stereo Vision”, IJRR, 2001; 20; 512

R. Zabih and J. Woodfill, “Non-parametric Local
Transforms for Computing Visual Correspondence”, ECCV
2004, pp. 151-158







Follow

Get every new post delivered to your Inbox.

Join 180 other followers