Tentative Schedule

8/26
Wed.
Assignment: Written assignment 0
8/28
Fri.
Buzzwords: Linear transformation, affine transformation, transformation matrix, scaling, rotation, identity matrix, translation, matrix decomposition, singular value decomposition, orthogonal matrix, orthogonal vector, rotation around arbitrary axis
Assignment: Written assignment 1
9/2
Wed.
Buzzwords: Eyes, retina, fovea, foveated rendering, photoreceptors, cones, rods, blind spot, light spectrum, spectral power distribution, standard illuminant, luminosity efficiency function, luminance, lumen, spectral reflectance, color blindness, invisible light, visual cortex, metamerism, metamer, color matching experiment, color matching function, trichromatic theory of color, color space
9/4
Fri.
Buzzwords: Colorimetry, equal-energy white, color cube, gamut, quantization, chromaticity, chromaticity plot, chromaticity gamut, XYZ color space, reference white, Planckian locus, sRGB, additive color model, subtractive color model, perceptual color space, HSV, HSL, HSB, hue, saturation, value, lightness, brightness, CMYK, gamma, gamma encoding, gamma correction
    Suggested Reading:
  • Representing Colors as Three Numbers (IEEE Computer Graphics and Applications, 2005).
  • Chapter 19.2, Fundamentals of Computer Graphics (4e). In the lecture we didn't talk about CIELAB in Chapter 19.2.4, but skim it if you want.
  • Chapter 19.2.1 in the book above talks about constructing transformation matrices between XYZ and RGB, which is mathematically rigorous but might not be intuitive to read. For a detailed derivation of the transformation, read these two articles.
  • The last a few slides in the lecture desk have a few more useful references.
Assignment: Written assignment 2
9/9
Wed.
Buzzwords: Pinhole camera, aperture stop, field stop, convex lens, concave lens, Snell's law, refraction index, waveguide, thin lens approximation, paraxial approximation, gaussian lens equation, circle of confusion, depth of field, F-number, magnification factor, think lens, optical aberration, chromatic aberration, monochromatic aberration, spherical aberration, coma, astigmatism, field curvature, distortion
    Required Reading:
  • Chapter 5.1, 5.2, and 5.3, Optics (5e). Skip the detailed math if you want. Try to focus on understanding the concepts.
  • Play with this DoF applet.
    Suggested Reading:
  • Chapter 6.3, Optics (5e). Again, skip the detailed math if you want. Try to focus on understanding the concept of different aberrations.
9/11
Fri.
Buzzwords: perceptual distortion, point spread function, PSF, Bokeh effect, convolution, deconvolution, coded aperture, aperture plane, Fourier plane, depth estimation, digital refocusing, depth from defocus, phase mask, phase-coded aperture
9/16
Wed.
Buzzwords: IR filter, UV filter, microlens, photoelectric effect, photodiodes, quantum efficiency, light-matter interaction, energy-harvesting camera, pixel well, full-well capacity, saturation, dynamic range, high dynamic range, NIR photography, thermographic camera, IR camera, depth sensing, fill factor, charge-to-voltage conversion, voltage gain, CCD sensor, CMOS sensor, rolling shutter, global shutter, analog-to-digital conversion, ADC, resolution, step size, spectral sensitivity function, spectral transmittance function, color correction, color compensation, linear least square, three-shot camera, vertically-stacked sensor, three-chip camera, color filtering array
    Required Reading:
  • Image Sensor (Computer Vision: A Reference Guide, 2nd Edition). Very brief, but gives you an overall idea. Skip the noise part, which we will get to in next lecture.
  • These three articles in a series (1, 2, 3) has a detailed description of color correction.
  • Study Table 2 and its related texts in A Review of RGB Color Space...from xyY to R'G'B'. It shows how to obtain the "ground truth" of a color patch, which can then be used to construct the color correction matrix (the paper itself doesn't use it for that purpose).
    Suggested Reading:
  • Chapter 3 of Image Sensors and Signal Processing for Digital Still Cameras. It has an in-depth discussion of the various aspects of camera sensors. What's relevant for today's lecture is Chapter 3.1 and 3.2. You can safely skip the math there, but going through it gives you a nice technical understanding of the topic.
  • The last a few slides in the lecture desk have a few more useful references.
9/18
Fri.
Buzzwords: bayer filter, CYGM filter, aliasing, anti-aliasing filter, Nyquist-Shannon sampling theorem, demosaic, noise, fixed pattern noise, photon shot noise, dark current noise, read noise, Poisson distribution, signal to noise ratio, signal gain, ISO, dynamic range, decibel, dB, stop, radiant power, radiant intensity, radiance, luminous power, luminous intensity, luminance, candela, tone mapping, gamut mapping
9/23
Wed.
Buzzwords: demosaic, white balancing, image signal processing, ISP, camera serial interface, systems-on-a-chip, SoC, edge, spatial correlation, spectral correlation, color constancy, chromatic adaptation, neutral point, white point, illuminant, von Kries adaptation model, auto white balancing, AWB
9/25
Fri.
Buzzwords: Denoising, spatial denoising, temporal denoising, filtering, convolution, mean filter, moving average filter, Gaussian filter, low-pass filter, bilateral filter, luminance denoising, chroma denoising, HDR imaging, tone mapping, graduated neutral density filter, GND, multiple exposures, radiance map, linear tone mapping, global tone mapping, local tone mapping, contrast, local contrast, HDR look
    Required Reading:
  • Image Filtering Tutorial. If you've never head of image filtering or convolution, read this tutorial, which also talks about how filtering can be used for sharpening and edge detection.
  • Tones and Contrast and Tone Curve in Photoshop. They talk about the notion of tone and contrast as well as how changing image tones on a standard DR image changes the visual effects. An interesting thing to note is that tone mapping in SDR images usually uses an S-shape curve that increases the contrast in midtones and compress contrasts in highlights and shadows, whereas tone mapping curves for HDR images tend to be inverse-S shape to bring out details in highlights and shadows.
9/30
Wed.
Buzzwords: Feedback-driven optimization, active AF, passive AF, phase detection, contrast detection, PDAF, CDAF, cross-correlation, autofocus point, dual pixel AF, time-of-flight, depth from stereo, structured light, metering, exposure value
10/2
Fri.
Buzzwords: Encoder, decoder, codec, lossless, lossy, luma, chroma, chroma subsampling, Y'CbCr, JPEG, DCT, basis function, quantization matrix, entropy encoding, run-length encoding, RLE, Motion-JPEG, M-JPEG, MPEG, H.262, H.264, H.265, AVC, HEVC, container former, .mp4, intra-frame prediction, inter-frame prediction, motion estimation, motion compensated interpolation, three step search, residual, macroblock, motion vector, reference frame, I-frame, P-frame, B-frame, GoP, sub-pixel motion, transcoding
10/7
Wed.
Buzzwords: Light field, plenoptic function, extended depth of field, optical flow, camera rig, omnidirection stereo
10/9
Fri.
Buzzwords: Panorama, field of view, FOV, telephoto lens, long-focus lens, perspective projection, ghosting, cylindrical projection, spherical projection, distortion, equirectangular projection, cubemap projection, VR tax, semantic-aware streaming
10/14
Wed.
Buzzwords: Photo-realism, computer-generated imagery, parametric surface, parametric curve, polygon mesh, triangle mesh, point cloud, point set, normal, polygon triangulation, bezier curve, bezier surface, basis functions, Bernstein polynomials, LiDAR, point set triangulation, Delaunay triangulation
10/16
Fri.
Buzzwords: rasterization, ray tracing, image-centric, object-centric, visibility, shading, local frame, world frame, camera space, raster space, perspective projection, orthographic projection, environmental camera, near-clipping plane, far-clipping plane, barycentric coordinates, depth-buffer, z-buffer
    Suggested Reading:
  • Read the first four chapters on this Scratchapixel lession on Rasterization: a Practical Implementation.
  • Both this Scratchapixel article and Chapter 7.2 and 7.3 of Fundamentals of Computer Graphics (4e) talk about how to calculate the perspective transformation matrix. They use different conventions but the principle is the same. Go over them to reinforce your understanding. Pay attention to details.
10/21
Wed.
Buzzwords: recursive ray tracing, ray-triangle intersection, acceleration structures, space-partitioning trees, octree, kd-tree, bounding volume hierarchy, BVH
10/23
Fri.
Buzzwords: reflection, refraction, glossy, diffusion, translucent, subsurface scattering, specular, fresnel equation, Phong shading, Lambert's cosine law, Lambertian diffusion surface, albedo, reflectance, directional-hemispherical reflectance, hemespherical-directional reflectance
10/28
Wed.
Canceled (Instructor Virtual Traveling)
10/30
Fri.
Buzzwords: Rendering equation, bidirectional reflectance distribution function, BRDF, emitted radiance, path tracing, Monte-Carlo integration, variance, bias, Monte-Carlo path tracing, specular lobe, caustic, bidirectional path tracing, texture map, texture mapping, texel, UV unwrapping, UV map, UV island, texture sampling, texture filtering, normal map
11/4
Wed.
Buzzwords: Volumetric data, volumetric rendering, surface reconstruction, splat, splatting, reconstruction kernel, ε z-buffering, level of details, LOD, qsplat, bounding sphere hierarchy, point cloud compression
    Required Reading:
  • Chapter 1, Point-based Graphics. The chapter overviews the book, but is a nice overview of the field.
11/6
Fri.
Abstract: Cultural heritage preservation is vital to making informed policy that shapes our future. I will present three projects illustrating different ways computing can support this effort. A project in medieval manuscripts, DESMM, shows the use of sensing and machine learning for documentation and interpretation. A project in collecting diverse data types associated with heritage artifacts and sites, CHER-Ob, shows the use of software systems design and new algorithms for enabling study and communication. Finally, a project in using mobile devices to generate personalized explanatory videos, AniCode, shows the potential for leveraging advanced technology to enhance visitor experiences at heritage sites without the burden of maintaining expensive advanced systems.
    Suggested Reading:
  • See the various sources from the slides!
11/11
Wed.
Buzzwords: Color rendering index, luminous efficacy, luminous flux, fluorescent light, phosphor, emissive display, light-filtering display, LED, LCD, liquid crystal, polarizer, pointillism, white LED, phosphor-converted LED, field sequential color, FSC, color wheel, DLP projector, digital micromirror device, DMD, color breakup, three-chip projector, brightness, DCI-P3
11/13
Fri.
Buzzwords: Neural networks, receptive field, loss function, convolutional neural network, max pooling, training, universal approximation theorem, photonics, PSF
11/18
Wed.
Abstract: Real-time 3D graphics systems are everywhere, but the tools that software engineers use to build them are from the dark ages. OpenGL was invented in a time when GPUs were fixed-function pipelines, and it has grown haphazardly into a flexible system that resembles an inconvenient, confusing programming language. Essentially, it attempts to be both a convenient programming model and a portable hardware abstraction-and it does poorly at both. Modern APIs like Vulkan and Metal address the problem by acting *only* as low-level system abstractions, relegating the programming model to higher levels in the system stack. This shift creates an opportunity for research on designing programming models for real-time graphics that make it more expressive, faster, and more correct. In this talk, I will complain endlessly about OpenGL and then talk about two projects in my lab that aim to alleviate some of the misery it induces.
11/20
Fri.
Buzzwords: Foveated rendering, foveated ray tracing, vergence-accommodation conflict, gaze tracking, multi/vari-focal display, defocus blur
11/25
Thanksgiving Holiday
11/27
Thanksgiving Holiday
12/2
Wed.
Abstract: Getting past the acronyms of advanced displays -- HDR, WCG, LCD, QLED, OLED, AR/VR, HMD, etc. -- the overall goal is to deliver a predictable perceptual experience to the viewer. This lecture will cover some of the technology and perception details that make this a complicated process, and it will provide some tools and models to help structure solutions.
12/4
Fri.
Abstract: The physical world, in all its complexity, has visual information that presents itself at different levels of spatiotemporal resolution. Mobile systems, including smartphones, tablets, and headsets, could benefit from the ability to situationally sacrifice image resolution to save system energy when imaging detail is unnecessary for computer vision and augmented reality tasks, but boost resolution when needed for precision. This talk will discuss challenges and opportunities of mobile operating systems and vision sensing pipeline architectures to flexibly support dynamic multi-resolution workloads with rapid reconfigurability at low latencies, and expressiveness to meet computational needs with minimal developer burden.
12/9
Wed.
Abstract: We report on development progress of our particle-based hybrid fluid-kinetic multiphysics framework named TriForce. The code recovers results from both radiation-magnetohydrodynamic and fully kinetic codes, and is being designed to operate in between where both descriptions may co-exist and interact. The hybrid method enables capabilities beyond either of the individual modeling methods alone, and is being used to investigate a range of topics in fields such as inertial confinement fusion, magneto-inertial fusion, magnetic confinement fusion, high- energy-density physics, and high-intensity lasers. The goal of the TriForce Center for Multiphysics Modeling is to provide better predictive capability and access to advanced models for the benefit of the academic community. The current status of the project and its applications will be surveyed.