Image

  1. Introduction to Digital Imaging
    • Image Sensing and Acquisition
    • Describe the operational principles of CCD and CMOS sensors.
    • List the major differences between CCD and CMOS image sensors. - CCD: Charged Coupled Device
    • Charge accumulates during exposure
    • Collected charges shifted out by horizontal and then vertical shift registers
    • Each pixel is converted to the voltage
    • Voltage is amplified by an amplifier
    • Blooming: The charge collected by a pixel leaks to other pixels. Electrons move more easily in the vertical direction, resulting in a vertical streak. - CMOS: Complementary Metal Oxide Semiconductor
    • In a CMOS sensor, each pixel has its own charge-to-voltage conversion, amplifiers and digitization circuits -> No blooming, faster operation.
    • Digital values are read in a line-by-line fashion -> Rolling shutter artifact
    • CMOS circuitry dissipates less power. | CCD | CMOS | |—————————————————-|——————————————-| | Broadcast cameras (still industry standard for TV) | Phone, Web, and digital SLR cameras | | Better image quality | Cheaper and faster (data transfer) | | Susceptible to the blooming effect | Susceptible to the rolling shutter effect |
    • Know how colour images are captured by CCD and CMOS image sensors.
      • 3 CCDs capturing red, green and blue, A prism
      • CMOS: Arranging colour filters on a square grid of photosensors, 50% green, 25% red and blue
        • Separated channel -> interpolated channel -> summing three channels
  • Digital Imaging Fundamentals (P30)
    • Know the definitions of the digital image (pixel, gray level) and the colour map.
    • Know the amount of storage required for an image.
      • i, j -> B = 8 -> L = 2^8 = 256 -> 0-255
      • 3 * M * N * B/8
    • Know the following terms relating to digitalization: continuous-tone image -> digital
      • Sampling: its brightness is measured and sampled at discrete locations to form an element in a rectangular matrix.
      • Quantization: Each element in the sampled image is quantized
      • Dynamic range: The range of grey levels
      • Spatial resolution: Physical size of a pixel.
        • Pixelation
      • Brightness resolution: how accurate the greyscale value can represent
        • Brightness contouring: less bits -> single grey level
    • Know how nearest neighbor and bilinear interpolation work.
      • Gray level of (i,j) is assigned to be that of its nearest neighbour.
      • Weighted average of 4 closest neighbours.
      • p(s, t) = (1-s)(1-t)p(00) + (1-s)tp(01) + (1-t)sp(10) + stp(11)
  • Colour Fundamentals
    • Know different colour models introduced and what they are good for.
      • 65, 33, 2
      • Grayscale along the diagonal
      • Quantize each component into 6 levels (6^3)
      • [C, M, Y] = [1-R, 1-G, 1-B] - P73
      • HSV space we can process only brightness without altering the color - P79
  1. Point Operations
    • Define point operations.
    • Common Point Operations
    • Describe the following point operations:
      • Linear point operation
        • b affects brightness of image by shifting grey levels.
        • a affects contrast and brightness.
      • Logarithmic
        • reveal more detail, extremely large range of values
      • Power law
        • (<1) Map narrow range of dark input to wider range of output
        • (>1) Map wide range of dark input to narrow range of output
      • Window and level
    • Know when these point operations are used. - Histogram Processing
    • Know how to derive the expression for output histogram given input histogram and point operation in continuous gray-level domain.
    • Know how to apply the expression.
    • H[i] & D[i]
    • P30 H(D) = HA(f-1(D))/f’(f-1(D)) - Histogram Equalization
    • Histogram equalization for images with continuous or discrete gray levels. Expected to know formula. - P42 - Histogram Specification (Matching) [P69]
    • Obtain point operations achieving histogram specification in continuous and discrete domains.
  2. Spatial Filtering
    • Need to know how to perform convolution and correlation for 2D discrete images. Know the difference between convolution kernel and computational molecule.
      • filter/convolution kernel.
      • Flip the kernel g in horizontal and vertical directions -> computational molecule.
      • Weighted filter
    • Know how borders are handled in the correlation operation.
    • Smoothing spatial filters (Remove noise)
      • Boxcar filter (avg), Gaussian filter (involves convolution)
    • P18 - Order-statistic filters (do not involve convolution)
    • Median filter.
    • remove salt-and-pepper noise - Sharpening spatial filters (Feature enhancement) - First derivative filters (e.g., Prewitt filter, Sobel filter – need to know how they are defined). Need to know how to apply these filters in x and y directions and calculate gradient magnitude. - Second derivative filters (Laplacian). - Use Laplacian for image enhancement. Need to know how one correlation operation can be used to compute I - \delta^2*I. Which property of linear filtering is used? - P39
  3. Transform Domain Filtering [4.1]
    • Fourier Transform in 1D and 2D
    • Know Fourier transform for continuous signals and discrete signal.
    • Describe sampling theorem in terms of the Fourier transforms of pulse samples and discrete samples.
    • Know discrete Fourier transform
      • Know how to perform circular convolution
      • Describe why wraparound error occurs. - Need zero padding
      • Know how to perform linear convolution via circular convolution.
        • Frequency Domain Filtering [4.2]
    • Why do we filter in frequency domain instead of spatial domain?
    • Know the effect of circular convolution in 2D filtering and how to mitigate the effect.
    • Know ideal, Gaussian and Butterworth lowpass filters.
    • Design lowpass filter using ideal, Gaussian and Butterworth filters.
    • Know ideal, Gaussian, Butterworth bandpass and bandreject filters.
    • Know how to construct a notch filters based on ideal, Gaussian and Butterworth filters.
  4. Image Restoration and Reconstruction
    • Know noise removal filters:
    • Arithmetic mean filter - mean(sum)
    • geometric mean filter - (mul)^(1/n)
    • median filter - median
    • alpha-trimmed mean filter - region(mean(sum))
    • adaptive median filter - - Know three ways for estimating impulse response:
    • observation
    • experimentation
    • modelling. - Know inverse filtering
    • What issue does it have? enhance the noise, prominent at regions where is close to 0
    • How to mitigate this issue? Apply the inverse filter only at the low frequencies - Know Wiener filtering - Mean Squared Error (MSE) Filtering
    • What does it optimize?
    • What issue does it have? - We do not know noise and the spectrum of the undegraded image
    • How to mitigate this issue? - Approximate an average signal-to-noise ratio, denoted by SNR, used across all (u,v) - Image Reconstruction from Projections
    • Know Radon Transform
    • Define “sinogram”. Why is this representation called sinogram? - The Radon transform data is often called a sinogram because the Radon transform of an off-center point source is a sinusoid.
    • Describe backprojection. Describe mathematically why backprojection results in a blurred image.
    • Know the Fourier-slice theorem
    • Define “filtered backprojection”. Describe the filter that should be used mathematically. List the steps involved in filtered backprojection. P92
  5. Geometric operations
    • Linear transformation: expected to know formulae to relate coordinates in the input image with those in the output image
    • Translation
    • Scaling about origin or an arbitrary point
    • Rotation about origin or an arbitrary point
      • [cos, -sin] [sin, cos]:
    • Composite transformation. Order of operation does matter. - For all transformation described in (a), know how to define the affine matrix to implement the transformation using Python built-in tool [i.e., need to be able to determine T passed to cv2.warpAffine(im, T, (width, height)) for the transformation described in (a).] P 39 - Define forward mapping and backward mappings. Why are the advantages of using backward mapping? Need to be able to perform the backward mapping given the forward mapping [i.e., need to be able to express (i,j) in terms of (i’,j’).] - Grey level interpolation
    • Nearest neighbour
    • Bilinear - Landmark registration: expected to know how the four transformation parameters, a, b, ti and tj, are derived.
  6. Morphological Image Processing
    • Binary morphological operations
    • Know how to perform erosion and dilation.
    • Know the applications of erosion and dilation.
      • FIT - erosion -> ALL of the ‘1’s of SE are covered
        • Shrinking objects
        • Removing small objects or noise
        • Removing bridges and branches
        • Removing protrusions
        • Enlarge holes
      • HIT - dilation -> any ONE of the ‘1’s of SE is covered
        • Growing objects
        • Repairing intrustions
        • Filling gaps
        • Filling holes
    • Know how to perform opening and closing.
      • (A-B)+B
      • (A+B)-B
    • Know the geometric interpretations of opening and closing. - Connected components and labelling
    • Identify connected components based on the 4-connectedness and 8-connectedness definitions.
    • Understand the two-pass labelling algorithm. - Morphological algorithms
    • Hit-Or-Miss transform
    • Boundary extraction
    • Region filling
    • Skeletonization (no need to know detailed algorithm) - Grayscale Morphology
    • Know how to perform erosion and dilation.
    • Know how to perform opening and closing.
    • Know the geometric interpretations of opening and closing.
    • Know the following grayscale morphology algorithms: morphological smoothing, morphological gradient, top-hat and bottom-hat transformations
  7. Image segmentation
    • Edge detection
    • Know how to get 1st and 2nd derivatives of an image using convolution/correlation.
      • backward f(x) - f(x-1)
      • forward f(x+1) - f(x)
      • center f(x+1) - f(x-1)
      • f(x+1)+f(x-1)-2f(x)
    • Know how to compute gradient vector, its magnitude and angle at each pixel.
      • P11
    • Know how to use the Laplacian of Gaussian (LoG) filter
      • P24
    • Know how to generate an edge map of an image by finding the zero-crossing in an LoG filtered image.
    • Know the three criteria for an optimal edge detected as defined by Canny.
      • Good detection
      • Good localization
      • Single response
    • Know how to generate an edge map of an image using the Canny edge detector. - Discrete dynamic contour (DDC)
    • Know what is a deformable model for segmentation.
    • Expected to be able to calculate the following items: di, ti, ri, ci, and len(ci).
      • P50/P62/P63/P65
    • Expected to know how internal force is computed to avoid collapsing of a closed contour to a point.
    • Given an external force, expected to calculate the actual external force applied to DDC to avoid clustering of vertices.
Written on December 8, 2020