Course Description

Modern video games employ a variety of sophisticated algorithms to produce groundbreaking 3D rendering pushing the visual boundaries and interactive experience of rich environments. This course brings state-of-the-art and production-proven rendering techniques for fast, interactive rendering of complex and engaging virtual worlds of video games.

 

This year the course includes speakers from the makers of several innovative games and game engines, such as Guerrilla Games, Unity Technologies, Epic Games, and Reality Research Labs. The course will cover a variety of topics relevant to the practitioners of real-time rendering in games and other real-time 3D applications. The topics will cover diverse subjects such as real-time global illumination, volumetric clouds for realistic skies rendering, water systems and water rendering, hair simulation and rendering, ray and path tracing in real-time, and real-time Catmull-Clark subdivision techniques, amongst some.

 

This is the course to attend if you are in the game development industry or want to learn the latest and greatest techniques in the real-time rendering domain!

 

Note: All parts of the Advances will be recorded, whether virtual or live, pending speaker permissions, and they will be posted on SIGGRAPH’s virtual on-demand program until EOM October, and then on the Advances in Real-Time Rendering YouTube channel

Previous years’ Advances course slides: go here

Syllabus

Advances in Real-Time Rendering in Games: Part I
Virtual program

Advances in Real-Time Rendering in Games: Part II
Tuesday, August 9, 10:45 am-12:15 pm
West Building, Ballroom C/D

Advances in Real-Time Rendering in Games: Part III
Thursday, August 11, 2:15 - 5:15pm
West Building, Room 211-214

 

Prerequisites:  

 

Working knowledge of modern real-time graphics APIs like DirectX or Vulkan or Metal and a solid basis in commonly used graphics algorithms. Familiarity with the concepts of programmable shading and shading languages. Familiarity with shipping gaming consoles hardware and software capabilities is a plus but not required.

 

Intended Audience

 

Technical practitioners and developers of graphics engines for visualization, games, or effects rendering who are interested in interactive rendering.

 

Advances in Real-Time Rendering in Games: Part I
Virtual program

Welcome and Introduction
Natalya Tatarchuk (Unity Technologies)

 

Bisection Based Triangulation of Catmull Clark Subdivision
Jonathan Dupuy (Unity Technologies)
Thomas Deliot (Unity Technologies)

Lumen: Real-time Global Illumination in Unreal Engine 5

Daniel Wright (Epic Games)

Krzysztof Narkowicz (Epic Games)

Patrick Kelly (Epic Games)

Ray Tracing Open Worlds in Unreal Engine 5

Aleksander Netzel (Epic Games)

Tiago Costa (Epic Games)

 

Closing Notes

Natalya Tatarchuk (Unity Technologies)

 

 

Advances in Real-Time Rendering in Games: Part II
Tuesday, August 9, 10:45 am-12:15 pm
West Building, Ballroom C/D

10:45am

Welcome and Introduction
Natalya Tatarchuk (Unity Technologies)

 

10:55am

Nubis, Evolved: Real-Time Volumetric Clouds for Skies, Environments, and VFX
Andrew Schneider (Guerrilla Games)

12:10pm

Closing Notes

Natalya Tatarchuk (Unity Technologies)

 

Advances in Real-Time Rendering in Games: Part III
Thursday, August 11, 2:15-5:15 pm
West Building, Room 211-214

2:15pm

Welcome and Introduction
Natalya Tatarchuk (Unity Technologies)

 

2:20pm

Probe-based lighting, strand-based hair system, and physical hair shading in Unity’s ‘Enemies’
Francesco Cifariello Ciardi (Unity Technologies),

Lasse Jon Fuglsang Pedersen (Unity Technologies),

John Parsaie (Unity Technologies)

 

3:25pm

Real-time Cluster Path Tracing for Remote Rendering
Feng Xie (Reality Labs Research)

 

4:20pm

Rendering Water in Horizon Forbidden West
Hugh Malan (Guerrilla Games)

5:10pm

Closing Notes

Natalya Tatarchuk (Unity Technologies)

 

 

 

Course Organizer

Natalya Tatarchuk (@mirror2mask) is a graphics engineer and a rendering enthusiast at heart, currently focusing on driving the state-of-the-art rendering technology and graphics performance for the Unity engine as a Distinguished Technical Fellow and Chief Architect, VP, Professional Artistry and RT3D Graphics Innovation, and, prior to that, led the Graphics team at Unity. Before that she was a AAA games developer, working on innovative cross-platform rendering engine and game graphics for Bungie’s Destiny franchise, as well the Halo series, such as Halo: ODST and Halo: Reach, and AMD Graphics Products Group where she pushed parallel computing boundaries investigating advanced real-time graphics techniques, and graphics hardware design and APIs. Natalya has been encouraging sharing in the games graphics community for several decades, largely by organizing a popular series of courses such as Advances in Real-time Rendering, Open Problems in Real-Time Rendering and Rendering Engine Architecture at SIGGRAPH, and convincing people to speak there. It seems to be working.

Bisection Based Triangulation of Catmull Clark Subdivision

 

Abstract: Concurrent binary trees are a recent GPU-friendly data-structure suitable for generating bisection-based terrain tessellations, i.e., adaptive triangulations over square domains. In this talk, we introduce simple mappings and algorithms that bring such adaptive triangulations to Catmull-Clark subdivision surfaces. Our resulting implementation is straightforward and allows us to render densely triangulated subdivision surfaces in a few milliseconds on a modern GPU. We showcase production scenes rendered in real time within the Unity game engine thanks to our method.

 

Speaker Bios: 

	Jonathan Dupuy is a senior research scientist working at Unity Technologies. His research interests are primarily oriented towards high quality real-time rendering. This encompasses a wide range of topics including antialiasing, level-of-detail, analytic models for both materials and lighting, and GPU/parallel programming.
	Thomas Deliot is a senior research engineer working at Unity Technologies. His work focuses on computer graphics topics and improving real-time rendering of 3D content, by bringing new papers into rendering engines and bridging the gap from research to production. This includes GPU/parallel programming, post-processing, level-of-detail, materials, lighting and machine learning.

Materials (Updated 9/19/2022):   PPTX (57MB), Paper (1MB)

 

 

 

Lumen: Real-time Global Illumination in Unreal Engine 5

 

Abstract: Lumen is Unreal Engine 5’s fully dynamic global illumination and reflection system, scaling from next generation console games to high-end PC visualizations. This talk will dive into how Lumen works, from software ray tracing with signed distance fields, to virtualized surface caching, hardware ray tracing, final gathering, and reflections. We’ll also show how Lumen solved indirect lighting in a large-scale procedural city in ‘The Matrix Awakens’, including an emissive-only night mode.

 

Speaker Bios: 

	Daniel Wright is an Engineering Fellow in graphics at Epic Games, and Technical Director of the Lumen dynamic global illumination and reflections system in Unreal Engine 5.  Prior to that, he developed lighting and shadowing techniques for Unreal Engine 3 and 4 which shipped in Gears of War, Fortnite and a multitude of games licensing Unreal Engine technology.  Daniel's main passion is real-time Global Illumination.
	Krzysztof Narkowicz is a Technical Director in graphics at Epic Games. Krzysztof worked on rendering architecture, virtual production, shadows and lighting before co-founding the Lumen initiative. Prior to that, he spent almost a decade working on smaller game titles at 11 Bit Studios and Flying Wild Hog. Outside of work, he may be known for his Shadertoy shaders. He loves working with artists, pretty pixels, and coding “close to the metal”.
	Patrick Kelly is a Senior Rendering Programmer at Epic Games. Patrick is most closely associated with the real-time ray tracing technology in Unreal Engine and has contributed to numerous demos along with ray tracing in Fortnite. Before arriving at Epic, he spent nearly a decade in production rendering at studios such as DreamWorks Animation, Weta Digital, and Walt Disney Animation Studios.

 

Materials (Updated 9/19/2022):   PPTX (257 MB), PDF (26 MB)

 

Ray Tracing Open Worlds in Unreal Engine 5

 

Abstract: In this talk, we will describe some of the techniques used in Unreal Engine 5 to support ray tracing in large open worlds, based on the development of "The Matrix Awakens: An Unreal Engine 5 Experience". We will focus on how we build a ray tracing scene representation of a world containing over 1.5M instances, and cover topics such as inline ray tracing and ray traversal optimization.

 

Speaker Bios: 

	Aleksander Netzel is a Senior Rendering Programmer at Epic Games, working on ray tracing. Previously he has worked at Apple and Rockstar as a Graphics Engineer.
	Tiago Costa is a Senior Rendering Programmer at Epic Games, working on ray tracing and rendering features. Previously he has worked at Rockstar North, Apple and Facebook Reality Labs.

Materials (Updated 9/19/2022):  PPTX (156 MB)

 

Nubis, Evolved: Real-Time Volumetric Clouds for Skies, Environments, and VFX

 

Abstract: Real-Time volumetric clouds in games have seen broadening adoption over the past several years. Many of the new applications use systems based on or similar to our approach, Nubis, which was introduced in 2015 in the Advances in Real-Time Rendering course. Nubis could produce a variety of cloud types in various lighting conditions for use in dynamic volumetric skyboxes that could render in under two milliseconds on the PlayStation 4 for the game, Horizon Zero Dawn. With the arrival of the new PlayStation 5 hardware and the sequel title, Horizon Forbidden West, we were able to push Nubis further into the game experience. In addition to improving our rendering approach for volumetric cloudscapes for use in skies, we started to use volumetric clouds to create environments such as clouds that the player could fly through. Additionally, we began to use them as VFX elements such as fast spinning superstorms with internal lighting flashes. In order to extend Nubis in these directions, several open problems in real-time volumetric cloud rendering had to be mitigated or solved. Rendering cloud environments up close, especially flying through them, can easily become computationally expensive. We will present a new cloud modeling and rendering approach that delivers performant and detailed results at 1080p resolution without the use of temporal upscaling. VisualFX are often synonymous with speed and lighting effects – two things that temporally upscaled clouds on a budget cannot do easily. We will present a way to mitigate temporal artifacts in fast-moving clouds and a near zero cost method to add internal lighting effects to clouds. Finally, we will explain how these three approaches are integrated and unified into the new Nubis Cloud System. As a bonus, we will offer a look at some future work.

 

Speaker Bio: 

Andrew Schneider is a Principal VFX Artist at Guerrilla in Amsterdam. He spends his time developing the tools and technology behind the Nubis real-time volumetric cloud system for the Horizon Franchise Games. Previously, he worked as a Senior FX Technical Director at Blue Sky Studios, where he developed the volumetrics and clouds pipelines for the Rio and Ice Age animated movies. His interests include visual effects, simulation, lighting, and volumetrics - but specifically how to engineer realistic and efficient solutions for these disciplines. He has previously given 5 Talks at SIGGRAPH from 2011 to 2017, one talk at Eurographics 2018, one GDC talk in 2022, and published a chapter about rendering real-time volumetric clouds for games in GPU Pro 7.

Materials (Updated 11/7/2022):   PDF (48MB), Full presentation (1.4GB)

Note: The Slides contain corrections and bonus material that were added after presentation day.

 

Probe-based lighting, strand-based hair system, and physical hair shading in Unity’s ‘Enemies’

 

Abstract: Enemies is the latest real-time short film produced by the Unity Demo Team. This talk will cover some of the key engine improvements informed by and built alongside the production and used to help realize the real-time film. We will discuss the choices made in building the adaptive probe-based system that powers the indirect diffuse lighting of the film’s detailed environment and character. Further, we will present the scalable strand-based hair system that is responsible for driving the tens of thousands of individual strands on the digital human. We will conclude in sharing our advances on physically based hair shading, where improvements in multiple scattering and area light support were made.

 

Bios: 

	Francesco Cifariello Ciardi is a Senior Graphics Engineer at Unity Technologies working on the HDRP team. His work spans a wide range of topics, including post-processing to lighting. Prior to Unity, he worked at Rare Ltd. where he owned the VFX and Lighting pipelines for Sea of Thieves. He received his Master's Degree from University College London with a thesis focused on Real-time Rendering of Human Skin.
	Lasse Jon Fuglsang Pedersen is a Senior Software Engineer at Unity Technologies and a member of the Unity Demo Team, where he helps identify and advance what is possible in a real-time context through production. His most recent work includes the strand-based hair system that is now featured and responsible for the simulated hair in "Enemies". Previously, he worked on the technical realization of the digital human face in "The Heretic", including the developer-facing release of that work. Prior to joining Unity, Lasse was at Playdead, where he helped build LIMBO for iOS, and the critically acclaimed INSIDE. He holds a Master's Degree in Computer Science from the University of Copenhagen.
	John Parsaie is a Senior Graphics Engineer at Unity and has been contributing to Unity's rendering architecture since 2016. His interests lie chiefly in physically-based materials, parallel algorithms, and cinematic rendering. Broadly speaking, John is passionate about any problem that reconciles our observations of the natural world with modern hardware constraints. Most recently, he has been helping to improve Unity's digital character rendering capabilities.

 

Materials (Updated 9/19/2022):   PPTX (302 MB), PDF (43 MB), Enemies Demo Video, Lion demo video

 

Real-time Cluster Path Tracing for Remote Rendering

 

Abstract: High-fidelity photorealistic rendering effects are commonplace in film rendering. At the same time, while we see many advances in this domain in real-time rendering in games and other 3D real-time applications; most of the real-time rendering still uses rasterization-based approaches to deliver rich visual experiences. Yet recent advances in computing hardware have seen increased application of real-time ray tracing in games and other real-time graphics applications, including hybrid rasterization rendering pipelines.

In this talk, we present the design choices for the architecture and implementation of the first production quality real-time cluster-based path tracing renderer that supports dynamic digital human characters with curve-based hair and path traced anisotropic sub-surface scattering for skin. We build our cluster path tracing system using the open-source Blender and its GPU-accelerated production quality renderer Cycles. Our system's rendering performance and quality scales linearly with the number of RTX GPUs and cluster nodes used. It can generate and deliver path traced images with global illumination effects to remote light-weight client systems at 15-30 frames per second for a variety of Blender scenes including virtual objects and animated digital human characters. In addition to design choices of path distribution and efficient support for dynamic geometry and BVH update on many GPUs, we will also discuss pragmatic considerations for designing and implementing remote cloud rendering systems for virtual reality and other real time graphics applications.

 

Speaker Bio: 

Feng Xie is a research scientist at Reality Labs where she leads cloud-based photorealistic rendering of digital humans for VR and AR.  Before Reality Labs, Feng was a senior principal engineer at PDI/DreamWorks where she worked on production rendering and shading systems.  Feng has credits on over 22 DreamWorks animation films and has presented her work at numerous SIGGRAPH and Eurographics conferences. After working in CG film production rendering for 16 years, Feng completed her PhD dissertation in physically based rendering with Professor Pat Hanrahan at Stanford University.

 

Materials (Updated 3/4/2023):   PDF (23 MB)

 

Rendering Water in Horizon Forbidden West

 

Abstract: In this talk we describe some of the techniques used to render the water in Horizon Forbidden West, focusing on the breaking waves.  We'll cover how Houdini sims were processed into localized water surface deformations, how the localized deformations were assembled into wavefronts, and the tools we built for the artists to control their shape and animation.  In addition, we'll talk about a few of the features that contributed the most to the quality.

 

Speaker Bio: 

Hugh Malan is a Senior Principal Programmer working at Guerrilla, in Amsterdam.  He led the development of the water for Horizon Forbidden West, and the indirect lighting for Horizon Zero Dawn.  Prior to that he worked on graphics for Dust 514 for CCP, and Crackdown for Realtime Worlds. Hugh is a graduate of Victoria University and Otago University, New Zealand.

Materials (Updated 9/19/2022):   PPTX (442 MB), PDF (4 MB)

Follow @mirror2mask	Tweet
Contact: