Course Description

Modern video games employ a variety of sophisticated algorithms to produce groundbreaking 3D rendering pushing the visual boundaries and interactive experience of rich environments. This course brings state-of-the-art and production-proven rendering techniques for fast, interactive rendering of complex and engaging virtual worlds of video games.

This year the course includes speakers from the makers of several innovative games and game engines, such as Activision, EA | Frostbite, Tencent Games, Intel, and even independent engine developers, for the first time in the Advances’ history! The course will cover a wide range of topics, this year diving deeply into improvements for GPU-driven pipeline rendering with visibility buffer and related techniques, and mobile-friendly cluster rendering; delve into the design of shader language for AAA games; and cover global illumination topics with practical innovations for runtime surfels-based GI, as well as a neural network-based GI, and advances in hemispherical lighting. We will also explore improvements for dense geometry rendering, and more.

This is the course to attend if you are in the game development industry or want to learn the latest and greatest techniques in the real-time rendering domain!

Previous years’ Advances course slides: go here

Syllabus

Advances in Real-Time Rendering in Games: Part I

Tuesday, 30 July 2024 9:00 am - 12:15 pm MDT

Location: Four Seasons 1

Welcome and Introduction to Part I
Natalya Tatarchuk (Activision)

Neural Light Grid: Modernizing Irradiance Volumes with Machine Learning

Michał Iwanicki (Activision)

Seamless Rendering on Mobile: The Magic of Adaptive LOD Pipeline

Shun Cao (Tencent Games)

Flexible and Extensible Shader Authoring in Frostbite with Serac

Simon Taylor (EA | Frostbite)

Closing Notes for Part I

Natalya Tatarchuk (Activision)

Advances in Real-Time Rendering in Games: Part II

Tuesday, 30 July 2024, 2:00 pm - 5:15 pm MDT

Location: Four Seasons 1

Welcome and Introduction to Part II
Natalya Tatarchuk (Activision)

Announcing The Call of Duty Open-Source USD Caldera Data Set

Michael Vance (Activision)

Variable Rate Shading with Visibility Buffer Rendering

John Hable (Visible Threshold)

Shipping Dynamic Global Illumination in Frostbite

Diede Apers (EA | Frostbite)

Hemispherical Lighting Insights from the Call of Duty Production Lessons

Thomas Roughton (Activision)

Achieving scalable performances for large scale components with CBTs

Anis Benyoub (Intel), Jonathan Dupuy (Intel)

Closing Notes for Advances in Real-Time Rendering in Games, 2024

Natalya Tatarchuk (Activision)

Prerequisites

Working knowledge of modern real-time graphics APIs like DirectX or Vulkan or Metal and a solid basis in commonly used graphics algorithms. Familiarity with the concepts of programmable shading and shading languages. Familiarity with shipping gaming consoles hardware and software capabilities is a plus but not required.

Intended Audience

Technical practitioners and developers of graphics engines for visualization, games, or effects rendering who are interested in interactive rendering.

Course Organizer

Natalya Tatarchuk (@mirror2mask) is a graphics engineer and a rendering enthusiast at heart, currently serving as Chief Technology Officer at Activision Publishing where she’s helping drive innovative technology powering award-winning franchises like Call of Duty at Activision / Microsoft. Prior to her current position, she drove the state-of-the-art rendering technology, graphics performance and character content creation in her role as a Distinguished Technical Fellow and Chief Architect, VP, Wētā Tools at Unity. Before leading the Graphics team at Unity as VP of Graphics for the Unity Editor and Engine, she was a AAA games developer, working on innovative cross-platform rendering engine and game graphics for Bungie’s Destiny franchise, as well the Halo series, such as Halo: ODST and Halo: Reach, and AMD Graphics Products Group where she pushed parallel computing boundaries investigating advanced real-time graphics techniques, and graphics hardware design and APIs. Natalya has been encouraging sharing in the games graphics community for several decades, largely by organizing a popular series of courses such as Advances in Real-time Rendering, Open Problems in Real-Time Rendering and Rendering Engine Architecture, and convincing people to speak there. It seems to be working.

Talks

Welcome and Introduction – Trends in Games and Rendering

Abstract: This talk provides the context behind the history of Advances in Real-Time Rendering in Games course since its inception, as well as explaining the goals of what this session aims to achieve. The speaker further delves into the analysis of current trends scene in gamers, player perspectives for video games, and then explores what that means for trends for gaming technology in rendering and related areas, as well as platforms, and beyond.

Speaker Bio:

Natalya Tatarchuk (@mirror2mask) is a graphics engineer and a rendering enthusiast at heart, currently serving as Chief Technology Officer at Activision Publishing where she’s helping drive innovative technology powering award-winning franchises like Call of Duty at Activision / Microsoft. Prior to her current position, she drove the state-of-the-art rendering technology, graphics performance and character content creation in her role as a Distinguished Technical Fellow and Chief Architect, VP, Wētā Tools at Unity. Before leading the Graphics team at Unity as VP of Graphics for the Unity Editor and Engine, she was a AAA games developer, working on innovative cross-platform rendering engine and game graphics for Bungie’s Destiny franchise, as well the Halo series, such as Halo: ODST and Halo: Reach, and AMD Graphics Products Group where she pushed parallel computing boundaries investigating advanced real-time graphics techniques, and graphics hardware design and APIs. Natalya has been encouraging sharing in the games graphics community for several decades, largely by organizing a popular series of courses such as Advances in Real-time Rendering, Open Problems in Real-Time Rendering and Rendering Engine Architecture, and convincing people to speak there. It seems to be working.

Materials (Updated August 5^th, 2024): PDF (Slides + Notes, 5MB)

Neural Light Grid: Modernizing Irradiance Volumes with Machine Learning

Abstract: Irradiance volumes and their variants have been used in video games for over two decades. While more advanced, real-time global illumination systems have been presented, they are only practical on most high-end hardware. The diversity of hardware used by players, particularly low-end mobile GPUs, still creates a need for a robust, high-quality solution that can be universally used across all supported platforms, without requiring to re-author the lighting setup. Despite the popularity, the basic problems associated with irradiance volumes have not been fully mitigated: to avoid leaking artifacts various hand-tuned heuristics are used, but they are often insufficient and require manual fixes.

This talk will present how modern machine learning methods can be used to eliminate these artifacts, while keeping the performance of the algorithm acceptable even on very constrained platforms. We will show how we took the idea of utilizing machine learning to improve precomputed lighting through multiple experiments, reshaped it, reframed it, reformulated it, to finally form a solution that shipped to multiple millions of players of the Call of Duty: Warzone and Call of Duty: Modern Warfare 3.

Speaker Bio:

Michał Iwanicki has been working in the game industry for over 20 years. He started in CD Projekt RED, working on the first installments of "The Witcher" series as a rendering and engine programmer. At Lionhead he contributed to the engine technology for "Milo&Kate" project. As a part of the rendering team at Naughty Dog, he worked on "Uncharted 3", "The Last of Us" and "Uncharted 4". Since 2014, with a brief break to help as the CTO of CD Projekt RED, he's been at Activision Central Technology team, focusing on precomputed lighting, low level systems, CPU and GPU optimizations and various hardware projects. He presented some of his previous work at GDC, SIGGRAPH, EGSR. He's one of the co-authors of the 4th edition of the "Real-Time Rendering" book.

Materials (Updated August 5^th, 2024): PPTX (151 MB), PDF (Slides + Notes, 6MB)

Seamless Rendering on Mobile: The Magic of Adaptive LOD Pipeline

Abstract: In this talk, the authors will share a seamless and adaptive rendering solution for GPU-driven rendering on mobile building on previous approaches, ranging from Nanite, to other algorithms. Their method delivers an integrated lightweight LOD generation with low I/O overhead and high-performance rendering on mobile platforms. The presenters’ approach gives fine granular control in all aspects of performance and quality. The LOD techniques provided as part of this system reduce the complexity of LOD management for game pipelines by removing the necessity of manual LOD setup.

Speaker Bio:

Shun Cao is an expert engineer and tech lead at the R&D center of Tencent Games. He has a broad range of research interests with a particular focus on global illumination and animation simulation systems. In the past few years, he has been leading the development of various GI solutions, such as the distributed offline light baking tool Dawn and real-time mobile GI solution SmartGI, which are widely used by commercial projects at Tencent Games.

Materials (Updated August 10^th, 2024): PDF (2.6 MB)

Flexible and Extensible Shader Authoring in Frostbite with Serac

Abstract: Frostbite, like any large game engine, includes a lot of complex features and systems, as well as mechanisms to customize and extend those systems. This poses a particular problem for shader code since modern shading languages prioritize performance over flexibility and extensibility. Traditional tools used to provide this flexibility, such as callbacks and virtual functions, are thus not available – and would likely not be performant if they were.

Serac is a domain-specific language “wrapper” around HLSL that we built to address this problem without sacrificing performance. We will give an overview of what Serac is, how it fits into the broader shader authoring workflows in Frostbite, and what differentiates it from other similar approaches. We will describe how the language itself is implemented and how code is combined to form a final shader. Finally, we will go over what we learnt rolling this system out across the engine and titles using it, what went well and what we would do differently.

Speaker Bio:

Simon Taylor is a Senior Software Engineer at EA Frostbite, where he currently focuses on shader authoring and workflows. He joined Frostbite in 2016 where he has helped ship multiple titles, including those from the Battlefield, Battlefront, Mass Effect and FIFA franchises. A rendering engineer since 2004, he previously worked on Enlighten at Geomerics, and APB at Real Time Worlds.

Materials (Updated August 2, 2024): PPTX (78.3MB)

Announcing The Call of Duty Open-Source USD Caldera Data Set

Abstract: We present the Call of Duty Caldera Open-Source USD Data Set, a new geometric data set release of production assets from Call of Duty: Warzone, licensed for academic research and non-commercial use. We will discuss the details of the data set and our hopes for how it will spur interesting solutions to novel problems in the environmental geometry problem space. This data set is provided in the OpenUSD format, and also represents one of the most geometrically complex data sets available in OpenUSD.

Speaker Bio:

Michael Vance (Mastodon) is a Senior Vice-President and Fellow Software Engineer at Activision. He received his B.Sc in Computer Science in 1999 from Pennsylvania State University. His first job in the industry was at the start-up Loki Software, and he later began working at Treyarch before its acquisition by Activision. As a technical director he has led engineering on the Spider-Man series and contributed to many other titles including all of the Call of Duty games from 2011’s Modern Warfare 3 to present. He resides in Falmouth, ME, where he plays at farming when not volunteering for the Falmouth Land Trust.

Materials (Updated August 10^th, 2024):

· PPTX (142 MB),

· Activision Releases Call of Duty®: Warzone Caldera Data Set for Academic Use blog,

· Download the Call of Duty Data Set Now

Variable Rate Shading with Visibility Buffer Rendering

Abstract: Visibility Buffer rendering is an alternative approach to real-time rendering, with some very different tradeoffs compared to GBuffer and Forward rendering. But the most interesting property of Visibility Buffer rendering is that the shading rate is completely decoupled from the native rendering resolution. This talk discusses a method of reducing the number of pixels shaded while still maintaining visual fidelity, as well as discussing the advantages and disadvantages of the approach.

Speaker Bio:

John Hable is a rendering programmer who has worked at Electronic Arts, Naughty Dog, Epic Games, and Unity.

Materials (Updated August 5^th, 2024): PPTX (35 MB)

Shipping Dynamic Global Illumination in Frostbite

Abstract: Global Illumination Based on Surfels (GIBS) is Frostbite's dynamic indirect diffuse lighting system, developed in collaboration with EA SEED. We presented a thorough technical description of GIBS at Advances in 2021. Since then, GIBS has shipped with College Football 25, and is being used in an upcoming open world action game. This talk will cover both game’s requirements and budgets imposed on GIBS, and discuss several practical learnings, improvements, and optimizations that were crucial to achieve truly dynamic global illumination at 60fps on current generation consoles.

Speaker Bio:

Diede Apers is a Rendering Engineer at Electronic Arts in Stockholm. After graduating from Breda University of Applied Sciences in 2016, he joined EA to work on the Frostbite engine. Since the introduction of hardware-accelerated ray tracing, he has been involved in all aspects of its adoption in Frostbite. His work spans from the foundational infrastructure to the development of multiple shipped ray tracing features. He co-authored an article in Ray Tracing Gems in 2019, and presented Ray Traced Ambient Occlusion at GTC in 2021. His contributions have directly impacted games such as: Plants vs. Zombies: Battle for Neighborville, Battlefield 2042, Dead Space (2023), College Football 25, and more.

Materials (Updated August 10^th, 2024): PDF (7 MB)

Hemispherical Lighting Insights from the Call of Duty Production Lessons

Abstract: Activision’s titles such as Call of Duty make heavy use of precomputed lighting in order to deliver great visual fidelity while maintaining jitter-free competitive first-person shooter framerates. Precomputed lighting is mainly applied to surface hemispheres, which require special reasoning to achieve the best quality for normal mapped surfaces. In this talk, we discuss the importance of hemispherical occlusion, and extend that to show how high-quality occlusion can be efficiently computed from visibility cones, significantly improving the appearance of runtime AO with normal mapped surfaces. We additionally introduce our new, efficient HHD (Hemisphere and Highlight Direction) model for lightmaps that enables higher-quality runtime blending than the Ambient and Highlight Direction (AHD) model and will touch on how to correctly and generally solve for lightmaps that preserve IrradZ, the irradiance in the hemisphere normal.

Speaker Bio:

Thomas Roughton is a graphics engineer at Activision Central Tech, working on rendering techniques and performance with a particular focus on the baked lighting pipeline. Previously, Thomas led development of the renderer at LiveSurface; this wide-ranging work included core engine design and development, rendering techniques to plausibly blend many layers of custom user artwork or heightmaps with real photos, and implementation of the content pipeline for internally authored 2.5D or 3D scenes.

Materials (Updated August 5^th, 2024): PDF (48 MB)

Achieving scalable performances for large scale components with CBTs

Abstract: A concurrent binary tree (CBT) is a GPU-friendly data-structure suitable for the generation of bisection-based terrain tessellations, i.e., adaptive triangulations over square domains. In this talk, we will expand the benefits of this data structure in two respects. First, we show how to bring bisection-based tessellations to arbitrary polygon meshes rather than just square heightfields. Second, we alleviate a limitation that restricted the triangulations to low subdivision levels. We do so by using the CBT as a memory pool manager rather than an implicit encoding of the triangulation as done originally. We demonstrate the benefits of our improvements by rendering planetary scale geometry out of very coarse meshes. Performance-wise, our triangulation method evaluates in less than 0.2ms on PlayStation®5-level hardware.

Speaker Bios:

Jonathan Dupuy is a senior research scientist working at Intel Corporation. His research interests are primarily oriented towards high quality real-time rendering. This encompasses a wide range of topics including antialiasing, level-of-detail, analytic models for both materials and lighting, and GPU/parallel programming.

Anis Benyoub is a graphics research engineer at Intel Corporation. He is passionate about Monte Carlo integration, physically based rendering, and real-time performance. Before Intel, he worked at Unity Technologies in the High-Definition Render Pipeline team. He holds an M. Sc. in Computer Science from Ecole Polytechnique de Montréal and M. Eng degrees in Computer Science from INSA Lyon.

Materials (Updated August 5^th, 2024): PDF (12.5 MB)

Direct contact:

Follow @mirror2mask
Mastodon @mirror2mask