[CS5670] Lecture 3: Image Resampling & Interpolation

    728x90

    | Today's topic is Sampling & interpolation.

     

    | Key points

    • Subsampling an image can cause aliasing. Better is to blur (“pre-filter”) to remote high frequencies then downsample
    • If you repeatedly blur and downsample by 2x, you get a Gaussian pyramid
    • Upsampling an image requires interpolation. This can be posed as convolution with a “reconstruction kernel”

     

    Image scaling

     

     

    This image is too big to fit on the screen. How can we generate a half-sized version?

     

    Image scaling

     

     

    Throw away every other row and column to create a 1/2 size image

    • called image sub-sampling

     

     

    Why does this look so crufty?

     

    Image sub-sampling - another example

     

     

    Even worse for synthetic images

     

     

     

     

     

    Aliasing

     

     

    • Occurs when your sampling rate is not high enough to capture the amount of detail in your image
    • Can give you the wrong signal/image - an alias
    • To do sampling right, need to understand the structure of your signal/image
    • Enter Monsieur Fourier…
    • To avoid aliasing:
      • sampling rate ≥ 2 * max frequency in the image
        • said another way: ≥ two samples per cycle
      • This minimum sampling rate is called the Nyquist rate

     

    Wagon-wheel effect

     

    • Imagine a spoked wheel moving to the right (rotating clockwise).
    • Mark wheel with dot so we can see what's happening.
    • If camera shutter is only open for a fraction of a frame time (frame time = 1/30 sec. for video, 1/24 sec. for film):

     

     

    Without dot, wheel appears to be rotating slowly backwards! (counterclockwise)

     

    Nyquist limit - 2D example

     

     

     

     

     

     

    Aliasing

     

    • When downsampling by a factor of two
      • Original image has frequencies that are too high
    • How can we fix this?

     

     

    Gaussian pre-filtering

     

     

    • Solution: filter the image, then subsample

     

     

    Subsampling with Gaussian pre-filtering

     

     

    • Solution: filter the image, then subsample

     

    Compare with...

     

     

    Gaussian pre-filtering

     

     

    • Solution: filter the image, then subsample

     

     

     

    Gaussian pyramids [Burt and Adelson, 1983]

     

     

    • In computer graphics, a mip map [Williams, 1983]
    • A precursor to wavelet transform Gaussian Pyramids have all sorts of applications in computer vision

     

    Gaussian Pyramids have all sorts of applications in computer vision.

     

    • How much space does a Gaussian pyramid take compared to the original image?

     

    Gaussian pyramid

     

     

    Question❓

     

    How much space (number of pixels) does a Gaussian pyramid of an image take compared to the original image?

     

    Back to the checkerboard

     

    • What should happen when you make the checkerboard smaller and smaller?

     

     

    Upsampling

     

    • This image is too small for this screen: 
    • How can we make it 10 times as big?
    • Simplest approach: repeat each row and column 10 times
    • ("Nearest neighbor interpolation")

     

     

    Image interpolation

     

     

    Recall that a digital images is formed as follows:

     

    $$F[x, y] = quantize \left\{ f(xd, yd) \right\} $$

     

    • It is a discrete point-sampling of a continuous function
    • If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

     

     

     

    What if we don't know $f$?

    • Guess an approximation: $\tilde{f}$
    • Can be done in a principled way: filtering
    • Convert $F$ to a continuous function:
      • $f_F (x) = F (\frac{x}{d})$ when $\frac{x}{d}$ is an integer, 0 otherwise
    • Reconstruct by convolution with a reconstruction filter, $h$

     

     

     

    Reconstruction filters

    What does the 2D version of this hat function look like?

     

     

     

     

    Image interpolation

     

     

     

    Raster-to-vector graphics

     

     

     

    Depixelating Pixel Art

     

     

    Modern methods

     

     

    Super-resolution with multiple images

     

    • Can do better upsampling if you have multiple images of the scene taken with small (subpixel) shifts
    • Some cellphone cameras (like the Google Pixel line) capture a burst of photos
    • Can we use that burst for upsampling?

     

    Google Pixel 3 Super Res Zoom

     

     

    Effect of hand tremor as seen in a cropped burst of photos, after global aligbnment

     

     

    Example photo with and without super res zoom (smart burst align and merge)

     

     

    Reference: Lecture 1: Images and image filtering

    728x90

    댓글