Tentative Schedule

8/31
Class Canceled (Instructor Travel)
9/2
Preliminaries
9/7
Buzzwords: Affine transformation, transformation matrix, scaling, rotation, identity matrix, translation, camera projection, matrix decomposition, singular value decomposition, unitary matrix, orthogonal vector
9/9
Buzzwords: Inner product, projection, Fourier Series, Fourier Transform, Discrete Fourier Transform, spectrum, low-pass filter, high-pass filter
9/14
Buzzwords: Nyquist-Shannon sampling theorem, signal sampling, signal reconstruction, convolution, convolution theorem, compressive sensing
    Suggested Reading:
  • Image Filtering Tutorial. Nice illustrations of filtering/convolutions. Has more filter types than what we discussed in the class.
Human Visual System
9/16 9/21
Buzzwords: Eyes, retina, fovea, foveated rendering, photoreceptors, cones, rods, blind spot, light spectrum, spectral power distribution, photopic vision, scotopic vision, luminous efficiency function, lumen, cone fundamentals, color blindness, dichromat, unilateral dichromat, isochrome, tetrachromacy, invisible light, visual cortex, metamerism, metamer, color matching experiment, color matching function, trichromatic theory of color, color space
9/23
Buzzwords: Colorimetry, color cube, gamut, quantization, chromaticity, chromaticity plot, chromaticity gamut, XYZ color space, reference white, black-body radiation, Planckian locus, color temperature, correlated color temperature, sRGB, additive color model, subtractive color model, perceptual color space, HSV, HSL, HSB, hue, saturation, value, lightness, brightness, CMYK, gamma, MacAdam ellipses
Color in Nature, Arts, and Technologies
9/28 9/30
Buzzwords: Thermal radiation, bioluminescence, electroluminescence, biofluorescence, absorption, scattering, emission, transparent, translucent, opaque, reflectance, glazing, vanish, incomplete hiding, spectroradiometer, spectrophotometer, fluorescent light, phosphor, color rendering index, emissive display, light-filtering display, LED, LCD, liquid crystal, polarizer, white LED, phosphor-converted LED, field sequential color, FSC, color wheel, DLP projector, digital micromirror device, DMD, color breakup, three-chip projector, motion blur, solid angle, irradiance, radiance, nits, lux, brightness, dynamic range
    Required Reading:
10/5 10/7
Buzzwords: absorption coefficient, absorption efficiency, cross section area, scattering coefficient, scattering efficiency, pigment, binder, refractive index, exponential decay, exponential growth, phase function, multiple scattering, Kubelka-Munk model, sub-surface scattering, glossy, specular, diffuse, mass tone, tint, saturation
    Required Reading:
  • Chapter 5.7, Digital Modeling of Material Appearance. It has a nice discussion radiometry metrics and emission/absorption/scattering. Physically-based as far as rendering is concerned.
Digital Camera Imaging
10/12
Buzzwords: Pinhole camera, motion blur, convex lens, concave lens, Snell's law, refraction index, gravitational lens, waveguide, thin lens approximation, paraxial approximation, gaussian lens equation, circle of confusion, depth of field, F-number, magnification factor, field-of-view, think lens, optical aberration, chromatic aberration, monochromatic aberration, spherical aberration, motion from blur, depth from blur
    Required Reading:
  • Chapter 4.2 -- 4.5, Virtual Reality. Talks about cameras, lenses, and optical aberrations at a high level; a pleasant read.
10/14
Buzzwords: IR filter, UV filter, microlens, photoelectric effect, photodiodes, quantum efficiency, light-matter interaction, energy-harvesting camera, pixel well, full-well capacity, saturation, dynamic range, high dynamic range, NIR photography, thermographic camera, IR camera, depth sensing, fill factor, charge-to-voltage conversion, voltage gain, CCD sensor, CMOS sensor, rolling shutter, global shutter, analog-to-digital conversion, ADC, resolution
10/19
Buzzwords: noise, temporal noise, fixed pattern noise, photon shot noise, Poisson distribution, dark current noise, read noise, reset noise, hot pixels, bias/offset voltage, noise amplification, signal to noise ratio, signal gain, ISO, ADC gain, ADC noise, ADC quantization error, correlated double sampling, recover noise-free images, dark-frame subtraction, bias frame, dark frame, flat-field correction, dynamic range, stop, dB, three chip sensor, foveon sensor, bayer pattern, spectral sensitivity function, SSF
    Suggested Reading:
  • ARVR Sensing. Has good slides on image sensor hardware; includes some discussions on Digital Pixel Sensor (DPS), Correlated Double Sampling (CDS), and Time-to-Saturation for high dynamic range imaging.
  • Noise and modelling SNR. A great technical discussion on noise.
  • CCD Noise Sources and Signal-to-Noise Ratio. Another narrative of sensor noise. Talks about CCD, but the general idea is the same.
10/21
Buzzwords: image signal processing, ISP, denoising, filtering, Gaussian filter, low-pass filter, bilateral filter, demosaic, color correction, regression, ColorChecker
    Suggested Reading:
  • Andrew Rowlands' Color conversion matrices in digital cameras: a tutorial is the definitive to camera color correction; give it a read if you are serious about it, which also talks about the connection between color correction and white balance.
  • These three articles in a series (1, 2, 3) has a good explanation of color correction too.
10/26
Buzzwords: HDR imaging, tone mapping, graduated neutral density filter, GND, multiple exposures, radiance map, linear tone mapping, global tone mapping, local tone mapping, contrast, local contrast, HDR look
    Required Reading:
  • Tones and Contrast and Tone Curve in Photoshop. They talk about the notion of tone and contrast as well as how changing image tones on a standard DR image changes the visual effects. An interesting thing to note is that tone mapping in SDR images usually uses an S-shape curve that increases the contrast in midtones and compress contrasts in highlights and shadows, whereas tone mapping curves for HDR images tend to be inverse-S shape to bring out details in highlights and shadows.
10/28
Buzzwords: Feedback-driven optimization, color constancy, chromatic adaptation, neutral point, white point, illuminant, von Kries adaptation model, auto white balancing, AWB, active AF, passive AF, phase detection, contrast detection, PDAF, CDAF, cross-correlation, autofocus point, dual pixel AF, time-of-flight, depth from stereo, structured light, metering
11/2
11/4
Buzzwords: Encoder, decoder, codec, lossless, lossy, luma, chroma, chroma subsampling, Y'CbCr, JPEG, DCT, basis function, quantization matrix, entropy encoding, run-length encoding, RLE, Motion-JPEG, M-JPEG, MPEG, H.262, H.264, H.265, AVC, HEVC, container former, .mp4, intra-frame prediction, inter-frame prediction, motion estimation, motion compensated interpolation, three step search, residual, macroblock, motion vector, reference frame, I-frame, P-frame, B-frame, GoP, sub-pixel motion, transcoding
11/9 11/11
Buzzwords: Panorama, field of view, FOV, telephoto lens, perspective projection, ghosting, cylindrical projection, spherical projection, distortion, equi-rectangular projection, cubemap projection, spherical projection, stereo 360, omni-direction stereo, light field, extended depth of field, plenoptic camera, NeRF, photogrammetry, structure from motion, bundle adjustment, ego-motion estimation
Modeling and Rendering
11/16
Buzzwords: Photo-realism, computer-generated imagery, parametric surface, parametric curve, polygon mesh, triangle mesh, point cloud, point set, normal, polygon triangulation, bezier curve, bezier surface, basis functions, Bernstein polynomials, LiDAR, point set triangulation, Delaunay triangulation
11/18
Buzzwords: rasterization, ray tracing, image-centric, object-centric, visibility, shading, local frame, world frame, camera space, raster space, perspective projection, orthographic projection, environmental camera, near-clipping plane, far-clipping plane, barycentric coordinates, depth-buffer, z-buffer, supersampling, GPU, shader, SIMD
    Required Reading:
  • Read the first four chapters on this Scratchapixel lession on Rasterization: a Practical Implementation.
  • Both this Scratchapixel article and Chapter 7.2 and 7.3 of Fundamentals of Computer Graphics (4e) talk about how to calculate the perspective transformation matrix. They use different conventions but the principle is the same. Go over them to reinforce your understanding. Pay attention to details.
11/23
Thanksgiving Holiday
11/25
Thanksgiving Holiday
11/30
Buzzwords: ray-triangle intersection, acceleration structures, space partition, object partition, uniform grid, octree, k-d tree, binary space partitining tree, BSP, bounding volume hierarchy, BVH, recursive ray tracing, whitted-style ray tracing
12/2
Buzzwords: Phong model, global illumination, ambient shading, rendering equation, bidirectional reflectance distribution function, BRDF, Monte Carlo integration, Monte Carlo estimator, variance, bias, specular lobe, caustic, bidirectional subsurface scattering function, BSSDF, texture mapping, texel, UV unwrapping, UV map, UV island, texture sampling, texture filtering
Applications
12/7
Medical Imaging (Guest Lecture by Xiao Wang)
Buzzwords: Herculaneum Papyri, Archimedes Palimpsest, Hipparchus’s Star Catalogue, Scheimpflug imaging, pentimento, The Old Guitarist, The Martellus Map, The Hunt-Lenox Globe, multispectral imaging, structure from motion
12/9
Buzzwords: Eye piece, eye relief, virtual display, motion to photon latency, motion tracking, DoF, IMU, gyroscope, accelerometer, inside-out/outside-in tracking, teleportation, stereopsis, vergence-accommodation conflict, light field display, gaze tracking, foveated rendering, ventral metarism, foveated ray tracing, lens distortion, optical see-through, video see-through, waveguide
12/14