Real-Time Computer Graphics

Also known as 15-472/672/772. Tuesdays and Thursdays from 14:00-15:20 in Posner Hall 151 during Spring 2024. Taught by Jim McCann. Office hours after class in Smith 229 or by appointment.

Real-time computer graphics is about building systems that leverage modern CPUs and GPUs to produce detailed, interactive, immersive, and high-frame-rate imagery. Students will build a state-of-the-art renderer using C++ and the Vulkan API. Topics explored will include efficient data handling strategies; culling and scene traversal; multi-threaded rendering; post-processing, depth of field, screen-space reflections; volumetric rendering; sample distribution, spatial and temporal sharing, and anti-aliasing; stereo view synthesis; physical simulation and collision detection; dynamic lights and shadows; global illumination, accelerated raytracing; dynamic resolution, "AI" upsampling; compute shaders; parallax occlusion mapping; tessellation, displacement; skinning, transform feedback; and debugging, profiling, and accelerating graphics algorithms.

Principles

This course is based on four principles:

• Do Things for Reasons -- if you don't understand why you are doing something, look deeper. (Corr: if you don't understand why someone else is doing something, ask.)
• No Magic -- (as much as possible) avoid black boxes and needless helper libraries.
• Test and Improve -- make things efficient through real-world testing. (Corr: asymptotic complexity is not the only factor; constants matter in the real world.)
• Go Big -- push systems to their limits.

Work

(Note: subject to revision based on class timing.)

Your grade in this course will be determined by four assignments that cumulatively build a high-performance real-time renderer (60%); a project that adds a state-of-the-art feature to that renderer (30%); and participation in in-class discussions and activities (10%).

Don't Copy and Don't Steal

You will write a lot of code in this class, and I require that you actually type this code. (It is easy to copy-paste faster than you can think.)

You may use code from the internet, but make sure the code is not covered by a license that prohibits copying (copyright violation -- illegal!), be sure to include a comment citing your source (plagiarism -- immoral!), and make sure it passes through your brain on its way to your fingers (e.g., "fix" the coding style, re-work awkward parts, look up and document parts you don't understand).

You may work together, but take extra care to not copy code. Rule of thumb: if you are writing code, you should not be doing it where you can see your classmates' code.

Examples:

• GOOD: re-writing a documentation example in your own coding style (including a comment + URL citing your source).
• BAD: copy-pasting a stackoverflow answer or code snippet from the documentation, then re-working it in place. (You must type the code you use.)
• GOOD: working with a classmate to develop an design, then writing the code separately (and including a comment crediting the discussion in your code).
• GOOD: helping a classmate to debug their code, being inspired by their approach, and -- with your classmate's permission and a citation comment -- using a similar idea in your code.
• BAD: using copilot. (You aren't typing and you aren't thinking.)
• BAD: ChatGPT and similar. (Just bad in general. A copyright [or at least plagiarism] laundry.)
• GOOD: copy-pasting code you already wrote for this class.
• DISCOURAGED (but not forbidden): using autocomplete / codesense.

Resources

This course does not have a textbook; but you will likely find the following web resources helpful:

• C++ Reference
• Compiler Explorer -- for playing with how code might get optimized
• MDN's JavaScript reference
• Node.js documentation
• Blender Python API
• Vulkan
  • Vulkan Specification
  • Vulkan Guide (easier reading than the spec!)
  • Vulkan Tutorial (you will work through this for A0)
  • (something about SPIR-V?)
• Inspiration
  • Pouët -- a demoscene archive
  • Shadertoy -- fun fragment shader hacks
• Advances in Real-Time Rendering -- notes from SIGGRAPH courses
• GPU Gems -- the classic GPU programming books (now free!)

See also: the "extra readings" links in each section below.

Schedule

Real-Time Graphics

Tuesday, January 16th

What it means to be real-time and general techniques for getting there.

• T Jan 16
• Course Overview; Core Principles
• Interactive, Real-Time, and Beyond
• Big Ideas
• CPU Benchmarking and Fast Code Intuition
• A0a Timing, Math, and Intrinsics

The Graphics Pipeline

Thursday, January 18th
-Thursday, January 25th

How we get pixels onto the screen quickly. From attributes to framebuffers.

• R Jan 18
• Vulkan Intro
• A Picture of Vulkan
• Uniforms, Attributes, Vertices, Primitives, Framebuffers
• Opening a Window
• A0b Vulkan Tutorial
• T Jan 23
• Framework Style
• Vulkan Debugging Session
• A0a Scoreboard Check-In
• R Jan 25
• Javascript
• Build systems

Data Wrangling

Tuesday, January 30th

Getting data into our renderer.

• T Jan 30
• Scene graphs
• Instancing
• Data inflation: subdivision surfaces
• Sub-polygon detail: textures, normals, displacements
• Mesh compression?
• Texture compression
• least squares vertex baking(?)
• Creating (and finding) Bottlenecks
• A1 statement walkthrough
• Export scripts
• A1 Scene Viewer out

Culling and Occlusion

Thursday, February 1st
-Thursday, February 8th

What you don't render can't slow you down.

• R Feb 1
• Frustum culling
• Portal rendering
• Level-of-detail
• Occlusion queries
• Geometry images
• T Feb 6
• Bottlenecks
• R Feb 8: No Class / A1 Work Time

Materials

Tuesday, February 13th
-Thursday, February 22nd

Matching the real world, or doing better.

• T Feb 13
• detail with textures
• R Feb 15
• physical materials
• procedural materials
• approximations and ground truth
• material capture
• antialiasing
• glossy surfaces
• precomputation
• A2 Materials out
• T Feb 20
• Interesting Mesh Ideas
• A1 Scene Viewer due (before class)
• R Feb 22
• A1 debrief + Code Share

Color

Tuesday, February 27th

Primaries, white points, and encodings oh my.

• T Feb 27
• color spaces
• demo: P3 vs sRGB primaries
• demo: HDR
• vulkan formats and color spaces

Going Big

Thursday, February 29th
-Saturday, March 9th

Moving beyond what can fit in GPU memory.

• R Feb 29
• Mega-textures
• Terrains
• LOD streaming
• T Mar 5: Spring Break
• R Mar 7: Spring Break
• S Mar 9
• A2 Materials due (end of day)

Direct Lighting

Tuesday, March 12th

The first bounce of lighting is always the brightest.

• T Mar 12
• sphere, spot, and sun lights
• lambertian contribution from a spherical area light
• the "representative point" approximation
• light loops

Shadows

Thursday, March 14th
-Friday, March 15th

Solving the visibility problem.

• R Mar 14
• shadow maps
• cubic shadow maps
• percentage-closer filtering
• soft shadows with PCSS
• F Mar 15
• A3 Lights out

Real-time Raytracing

Tuesday, March 19th

Just brute-forcing the lighting.

• T Mar 19
• raytracing extensions
• sample sharing

Rendering for VR

Thursday, March 21st

The upsides and downsides of virtual reality.

• R Mar 21
• OpenXR APIs
• lens correction
• reprojection

Post-processing

Tuesday, March 26th

Fixing the pixels later.

• T Mar 26
• bokeh
• lens flares
• motion blur
• SSAO
• screen-space reflections

Volume Rendering

Thursday, March 28th

Clouds, medical data, and fire.

• R Mar 28
• raymarching smoke and clouds
• screen-space surfacing for liquids
• Nubus3

Simulation

Tuesday, April 2nd
-Friday, April 5th

Making things move without keyframes.

• T Apr 2
• Eulerian vs Lagrangian
• shallow-wave equations
• grid-based smoke
• F Final Project topic selection
• R Apr 4
• particle-based smoke
• particle-based fluid
• particle-based solids
• F Apr 5
• A3 Lights due (end of day)
• F Final Project out

Skinning and Animation

Tuesday, April 9th
-Thursday, April 11th

• T Apr 9
• dual-quaternion skinning
• Skinned Mesh Animation
• R Apr 11: Carnival

Miscellaneous

Tuesday, April 16th
-Friday, April 26th

Topics that didn't fit elsewhere (thematically or temporally) and were interesting enough to bring up.

• T Apr 16
• sparse voxel octrees and sparse voxel directed acyclic graphs
• hierarchical ordered grids
• R Apr 18: Jim Travelling
• T Apr 23
• R Apr 25
• F Apr 26
• F Final Project due (end of day)