Course Description
Modern video
games employ a variety of sophisticated algorithms to produce groundbreaking 3D
rendering pushing the visual boundaries and interactive experience of rich
environments. This course brings state-of-the-art and production-proven
rendering techniques for fast, interactive rendering of complex and engaging
virtual worlds of video games.
This year
the course includes speakers from the makers of several innovative games and
game engines, such as Sucker Punch Productions, Epic Games, Activision, EA |
SEED and Unity Technologies. The course will cover a variety of topics relevant
to the practitioners of real-time rendering in games and other real-time 3D
applications. The topics will cover diverse subjects such as real-time global
illumination, atmospheric rendering and dynamic time of day management,
advances in physically-based rendering, novel skylight
model, improvements for spatial upscaling, and several approaches for handling
large geometric complexities in real-time scenarios.
This is the course to attend if you are in the game development
industry or want to learn the latest and greatest techniques in the real-time
rendering domain!
Previous years’
Advances course slides: go here
Syllabus
Advances in Real-Time Rendering in Games: Part I
Tuesday, 10 August 2021 9 am - 12 pm PDT | Virtual Conference
Advances in Real-Time Rendering in Games: Part II
Wednesday, 11
August 2021 9 am - 12 pm PDT | Virtual Conference
Prerequisites
Working
knowledge of modern real-time graphics APIs like DirectX or Vulkan or Metal and
a solid basis in commonly used graphics algorithms. Familiarity with the
concepts of programmable shading and shading languages. Familiarity with
shipping gaming consoles hardware and software capabilities is a plus but not
required.
Intended Audience
Technical
practitioners and developers of graphics engines for visualization, games, or
effects rendering who are interested in interactive rendering.
Advances in Real-Time Rendering in Games: Part I
Tuesday, 10 August 2021 9 am - 12 pm PDT | Virtual Conference
9:00 am PDT
Welcome and Introduction
Natalya Tatarchuk (Unity Technologies)
9:15 am PDT
Improved Spatial Upscaling through FidelityFX Super Resolution for Real-Time Game Engines
Timothy Lottes (Unity Technologies)
Kleber Garcia (Unity Technologies)
9:40 am PDT
Experimenting with Concurrent Binary
Trees for Large Scale Terrain Rendering
Thomas Deliot (Unity
Technologies)
Jonathan Dupuy (Unity Technologies)
Kees Rijnen (Unity Technologies)
Xiaoling Yao (Unity Technologies)
10:10 am PDT
A Deep Dive into Nanite Virtualized Geometry
Brian Karis (Epic Games)
Rune Stubbe (Epic Games)
Graham Wihlidal (Epic
Games)
11:20 am PDT
Large-Scale
Global Illumination at Activision
Ari Silvennoinen (Activision Publishing)
12:15 am PDT
Part I
Closing Remarks
Natalya Tatarchuk (Unity Technologies)
12:30 pm PDT
Live Q&A
All Speakers
Advances in
Real-Time Rendering in Games: Part II
Wednesday, 11
August 2021 9 am - 12 pm PDT | Virtual Conference
9:00 am PDT
Welcome to Part II
Natalya Tatarchuk (Unity Technologies)
9:10 am PDT
Real-Time Samurai Cinema: Lighting,
Atmosphere, and Tone mapping in Ghost of
Tsushima
Jasmin Patry (Sucker Punch Productions)
10:10 am PDT
Radiance
Caching for Real-time Global Illumination
Daniel
Wright (Epic Games)
10:45 am PDT
Global Illumination Based on Surfels
Henrik Halen (SEED at Electronic Arts),
Andreas Brinck (Ripple Effect Studios at Electronic
Arts),
Kyle Hayward (Frostbite at Electronic Arts),
Xiangshun Bei (Ripple Effect Studios at Electronic
Arts)
11:35 am PDT
Part II Closing Remarks
Natalya Tatarchuk (Unity Technologies)
12:00 pm PDT
Live Q&A
All Speakers
Course
Organizer
|
|
Natalya Tatarchuk
(@mirror2mask) is a graphics engineer and
a rendering enthusiast at heart, currently focusing on driving the
state-of-the-art rendering technology and graphics performance for the Unity
engine as a Distinguished Technical Fellow and VP, AAA
and Graphics Innovation, and, prior to that, led the Graphics team at Unity.
Before that she was a AAA games developer, working on innovative
cross-platform rendering engine and game graphics for Bungie’s Destiny
franchise, as well the Halo series, such as Halo: ODST and Halo: Reach,
and AMD Graphics Products Group where she pushed parallel computing boundaries
investigating advanced real-time graphics techniques, and graphics hardware
design and APIs. Natalya has been encouraging sharing in the games graphics
community for several decades, largely by organizing a popular series of
courses such as Advances
in Real-time Rendering and Open Problems in Real-Time Rendering at SIGGRAPH, and convincing people to speak there. It seems to be
working. |
Improved Spatial Upscaling through FidelityFX
Super Resolution for Real-Time Game Engines
Abstract: A two-part talk, illustrated
with diagrams, images, and performance numbers. First part dives into the
details and inner workings of AMD's FSR1 scaling algorithm. With a focus on
teaching the optimization principles used in its design, and to leave the
viewer with some creative thoughts on image processing. Second part covers
integration into a modern physically-based deferred
and forward rendering pipeline.
Bios:
|
|
Timothy
Lottes specializes in GPU algorithms and optimization with a
background in photography. Currently focusing on new graphics technology at Unity
as part of the Graphics Innovation Group. Prior work included authoring FidelityFX shaders like CAS and LPM at AMD, working on
TAA and mobile post processing at Epic, and authoring FXAA/TXAA at NVIDIA. |
|
|
Kleber
Garcia has a M. Sc. in Computer Science from the Florida Institute
of Technology. He led the rendering team for the Madden franchise during the
first drop of Xbox One and PS4. He later joined the Frostbite rendering team
in Stockholm where he implemented shadow systems, character material and lighting
models, and post process effects. He architected and led the development of
an open world GPU probe system currently shipping in Battlefield 6 and Need
for Speed HEAT. Kleber has joined the frontlines at Unity Technologies where
he has delivered the DLSS integration. He is an avid coffee drinker, foodie,
and has a passion for all things GPU and work pranks. |
Materials (Updated August 10th, 2021): Slides (PPTX - 23 MB, PDF 3.5 MB)
Experimenting With Concurrent Binary Trees for Large-scale Terrain
Rendering
Abstract: In this talk, we share
results of our novel technique using concurrent binary trees for large-scale terrain
rendering. First, we will review the foundations of concurrent binary trees and
provide intuition about their benefits for computing adaptive tessellations for
large-scale terrain geometries. Next, the presentation will share the results
of the original 2020 paper, followed by the deep dive into latest efforts to
integrate the original technique into the Unity game engine. The speakers will
share further optimizations found over the original implementation and showcase
early integration results.
Bios:
|
|
Thomas
Deliot is a senior research engineer
working at Unity Technologies. His work focuses on computer graphics topics
and improving real-time rendering of 3D content, by bringing new papers into
rendering engines and bridging the gap from research to production. This
includes GPU/parallel programming, post-processing, level-of-detail,
materials, lighting and machine learning. |
|
|
Jonathan
Dupuy is a senior research scientist working at Unity Technologies.
His research interests are primarily oriented towards high quality real-time
rendering. This encompasses a wide range of topics including antialiasing,
level-of-detail, analytic models for both materials and lighting, and
GPU/parallel programming. |
|
|
Kees
Rijnen is an engineering manager working
at Unity Technologies. His work focuses on helping his team research and
develop new environment technologies and workflows. |
|
|
Xiaoling Yao is a senior graphics
programmer working at Unity Technologies. His work mainly focuses on
improving the terrain system, integrating with new artist workflows
and adding graphics features like instancing and virtual texturing. |
Materials (Updated August 10th, 2021): Slides (PDF 5 MB)
A Deep Dive into Nanite Virtualized Geometry
Abstract: Nanite, Unreal Engine 5's new virtual geometry system, enables the
rendering of trillion triangle scenes at real-time framerates. This lecture
will take a deep dive into how Nanite works, from mesh import all the way to
final rendered pixels. We will explain how the mesh-based data structure is
built, streamed, decompressed, culled, rasterized, and finally shaded.
Bios:
|
|
Brian Karis is an Engineering Fellow in graphics at Epic Games. Most recently he
has led the development of Nanite for
UE5. He is most known for work on physically based shading and temporal
anti-aliasing although has touched most areas of real-time computer graphics
throughout his career. Prior to Epic he worked at Human Head Studios. |
|
|
Rune Stubbe is a Principal Rendering Programmer at Epic Games, where he focuses
on Nanite development and optimization. He has previously worked on rendering
technology at IO Interactive and Unity. Rune has also been active in the demoscene (as Mentor/TBC), where he has contributed
widely used compression tools and several award-winning releases. |
|
|
Graham Wihlidal is a Principal Rendering Programmer at Epic Games, primarily working
on Nanite and other UE5
initiatives. Previously, Graham worked at Electronic Arts (SEED, Frostbite,
BioWare), implementing and supporting technology used in many hit games like Battlefield, Dragon Age: Inquisition, Plants
vs. Zombies, FIFA, Star Wars: Battlefront, and others.
While at BioWare, Graham shipped numerous titles including the Mass Effect and Dragon Age trilogies, and Star Wars: The Old Republic. Graham is
also a published author and has presented at several conferences. |
Materials (Updated: August 10th, 2021): Slides (PDF, 16 MB)
Large-Scale Global Illumination at Activision
Abstract: : In this talk, we’ll describe
the key techniques behind the large-scale global illumination system in Activision.
We present a new precomputed lighting compression technique that enables
high-performance and seamless reconstruction directly from the compressed
lighting data. In addition, we’ll discuss visibility-based sampling of
precomputed volumetric lighting and describe a practical method for computing
constrained spherical harmonics representations.
Bio:
|
|
Ari
Silvennoinen is a Fellow Software
Engineer at Activision, where he works on graphics technology research and
development. Prior to Activision, he obtained a master’s degree from the
University of Helsinki and worked on graphics technology at Umbra Software
and Remedy Entertainment. His main interests are in global illumination,
visibility algorithms and real-time rendering and he has contributions in
graphics conferences and journals, including SIGGRAPH, I3D, CGF and EGSR. |
Materials (Updated August 10th, 2021): Slides
(PPTX – 237
MB)
Real-Time Samurai Cinema: Lighting, Atmosphere, and Tonemapping
in Ghost of Tsushima
Abstract: In this talk, we describe some of the
graphics techniques used in the production of Ghost of Tsushima. Set in 13th
century Japan, Ghost of Tsushima pays homage to classic samurai cinema with
dramatic lighting, wind, clouds, haze, and fog, and features a beautiful
open-world version of the island of Tsushima during the first Mongol invasion.
In this talk, we will cover the diffuse and specular indirect lighting
techniques used in the game including methods of computing SH irradiance probes
from sky visibility data, including plausible sky and sun/moon bounce light.
Next, we present our atmospheric lighting and rendering techniques, including
how we lit our haze, clouds, and particles with multiple scattering. We show
how to improve the accuracy of Rayleigh scattering to approach that of spectral
rendering by using a custom color space for atmospheric lighting calculations.
Finally, we discuss some of the tone mapping techniques we used to recreate the
samurai cinema experience in-game.
Bio:
|
|
Jasmin
Patry is a Lead Rendering Engineer at
Sucker Punch Productions, where he has worked on Infamous 2, Infamous Second
Son, Infamous First Light, and Ghost of Tsushima. Prior to that, he was at
Radical Entertainment and contributed to their Hulk, Scarface, and Prototype
titles. As a graduate student in the Computer Graphics Lab at the University
of Waterloo, he created the popular Linux game Tux Racer, which was named
“Best Free Software” by PC Magazine and has downloads numbering in the
millions. His interests include physically based rendering, scientific
computing, and performance optimization — and anything that makes games look
better and run faster. |
Materials (Updated August 11th, 2021): Slides
(PDF - 66 MB, HTML)
Radiance Caching for Real-Time Global Illumination
Abstract: This talk will present an efficient and
high-quality Final Gather for fully dynamic Global Illumination with ray
tracing, targeted at next generation consoles and shipping in Unreal Engine
5. Hardware Ray Tracing provides a new
and powerful tool for real-time graphics, but current hardware can barely
afford 1 ray per pixel for diffuse indirect, while Global Illumination needs
hundreds of effective samples for high quality indoor lighting. Existing approaches that rely on Irradiance
Fields cannot scale up in quality, while approaches relying on a Screen Space
denoiser have exorbitant costs at high resolutions. This talk will present practical applications
of Radiance Caching along with effective techniques to reduce noise and
leaking.
Bio:
|
|
Daniel
Wright is an Engineering Fellow in graphics at Epic Games, and
Technical Director of the 'Lumen' dynamic Global Illumination and Reflections
system in Unreal Engine 5. Prior to
that, he developed lighting and shadowing techniques for Unreal Engine 3 and
4 which shipped in Gears of War, Fortnite and a multitude of games licensing
Unreal Engine technology. Daniel's
main passion is real-time Global Illumination. |
Materials
(Updated August 18th, 2021): Slides (PPTX
– 122 MB)
Global Illumination Based on Surfels
Abstract: Global Illumination Based on
Surfels (GIBS) is a solution for calculating indirect
diffuse illumination in real-time. The solution combines hardware ray tracing
with a discretization of scene geometry to cache and amortize lighting
calculations across time and space. It requires no pre-computation, no special
meshes, and no special UV sets, freeing artists from tedious and time-consuming
processes required by traditional solutions. GIBS enables new possibilities in
the runtime, allowing for high fidelity lighting in dynamic environments and for
user created content, while accommodating content of arbitrary scale. The
algorithm is part of the suite of tools available to developers and teams
throughout EA as part of the Frostbite engine.
This talk will detail the GIBS
algorithm and how surfels are used to enable
real-time ray traced global illumination. We will describe how the scene is
discretized into surfels on the fly, and why we think
this discretization is a good fit for caching lighting operations. The talk
will describe the acceleration structure used to enable efficient access to surfel data, and how this structure allows us to cover
environments of arbitrary size, while keeping a predictable performance and
memory footprint. We will detail how the algorithm handles dynamic objects,
skinned characters, and transparency. Several techniques have been developed to
efficiently integrate irradiance on surfels. We will
describe our use of ray guiding, ray binning, spatial filters, and how we
handle scenes with large numbers of lights.
Bios:
|
|
Henrik
Halen joined Electronic Art's SEED research division as a Senior
Rendering Engineer in 2017. His work at SEED is focused on real-time graphics
algorithms, lighting and characters. Henrik's
experience as a rendering engineer prior to joining SEED includes a decade of
contributions to franchises such as Gears of War, Battlefield, Medal of Honor
and Mirror's Edge. |
|
|
Andreas
Brinck has worked as a rendering engineer
for more than two decades. He joined Electronic Arts in 2011 to help start
Ghost Games and was later the rendering lead on NFS Rivals, NFS 2015, NFS
Payback, and NFS Heat. In 2019 he joined DICE LA where he is currently
working on the Battlefield franchise. |
|
|
Kyle
Hayward has worked as a rendering engineer since 2010. He has focused
on multiple areas in graphics, from animation compression to global
illumination, working on both offline and real-time solutions. He joined EA
in 2012, and later became the NBA rendering lead from 2014 onwards. In 2019
he joined Frostbite, where he has been working on global illumination and
raytracing. |
|
|
Xiangshun Bei has been a rendering
engineer within DICE LA at EA since 2019, focusing on real-time rendering and
ray tracing. He currently works on the Battlefield franchise. Prior to DICE,
he contributed to graphics drivers for Adreno GPU on Snapdragon SoC at
Qualcomm. He received his master’s degree in computer science from University
of Southern California in 2017. |
Materials (Updated August 11th, 2021): Slides
(PPTX – 315 MB,
PDF – 16 MB)
