Ray tracing support

[rntrtul]

There's been previous discussion of adding ray tracing support to Godot (in 2019 , and more recently).

Vulkan seem to have added full support in their SDK along with validation layers and other helpful info link to press release. So I think development or planning implementation could begin now.

Another issue that was brought up was lack of hardware supporting ray tracing, but looking at the steam hardware survey from july 2022 I think 30% of GPU's support some form of hardware RT.

• Ampere (RTX 30XX): 17.41%
• Turing (RTX 20XX): 11.66%
• RDNA2 (RX 6XXX): 1.32%

Now not all of these are using ray tracing or should be running ray tracing, but it's still a significant number. There is also the consoles that support ray tracing now. Accounting for the time it will take to implement ray tracing and then the time it will take for games to ship with RT, I think this total number will be higher.

some features for ray tracing are

• RT shadows
• RT reflections
• RT global illumination

Hopefully this can be used to discuss the RT implementation and plan going forward.

[Calinou]

So I think development or planning implementation could begin now.

The current goal to stabilize the existing Vulkan renderer for 4.0. Someone was recently interested in exploring raytracing support in Godot as a side project, but I don't think it should be the focus right now unless you're doing this as a learning exercise. Feature freeze has also started for 4.0, so any new feature development will be targeted for 4.1 unless it's critical for 4.0.

Don't worry, the current Vulkan renderer will have plenty of features to keep a RTX 3090 Ti (and soon a RTX 4090) busy 🙂

[Mercesa]

I am the person Calinou mentioned that's interested in adding RT support :). I have started evaluating Godot's structures on the best ways could be to support Raytracing within Godot in my spare time.

I'm currently in the progress of adding RT support for the RenderingDevice and calling the RenderingDevice from GdScript.

I am planning to make a proposal at some point, but first want a working prototype to discover all the different types of problems I could run into before starting the proposal. I think the best focus would be first on adding support for rendering engineers to be able to use raytracing where they please through inline tracing, you could easily do shadows with some basic RT features.

There are two ways to trace rays:

• RT pipelines: A bespoke pipeline fully focused on raytracing, this would be dispatched by using vkCmdTraceRaysKHR similar to a compute shader is using dispatches.
• Inline raytracing: It is possible to trace rays from any type of shader stage (Compute, Frag, Vertex, etc). This is much easier to implement and support.

Both RT pipelines and Inline RT require acceleration structure building, but inline RT is definitely less effort to implement.

Adding RT is non-trivial, there are several layers of Godot that would require slight/big reworks/additions, I will describe a few elements of the cuff in no order of importance and will probably edit this post or add more posts as I gain more info. I am new to Godot's source, so I will definitely not have the full picture yet :).

• Acceleration structure building (AS): This requires geometry information about the whole scene, and some type of geometry LOD'ing to make the acceleration structure building workable, since building all geometry as is, is not possible for runtime at runtime framerates with complex game scenes, so we would need some type of geometry simplification step. To make this engine-worthy, we would need to support BLAS/TLAS refitting and have dynamic updates within the acceleration structures so animations are also supported.
• Shader authoring/system: Godot has several layers of shading systems, users can make their own nodes, and raytracing shaders are quite complicated and generally are grouped together. At the moment I don't see an easy way to have users create their own RT shaders without having to either make their own renderpasses and integrate them with the current rendering systems.
• Material complexity: Engines that deal with lots of materials such as Godot, Unreal etc. can be quite difficult for Raytracing in terms of grouping. Every Material would need to have its own bespoke RT Hit shader, and this would in the end turn into a giant pipeline that can slow down PSO creation to a crawl.

[Ansraer]

iirc reduz' SDFGI already builds a simplified representation of the static geometry. You might want to look into whether its possible to extend that code to also build the BVH for static meshes at the same time.

[Calinou]

I'm currently in the progress of adding RT support for the RenderingDevice and calling the RenderingDevice from GdScript.

@Mercesa Did you find time to progress on this? If you didn't manage to complete it, that's OK, but please make sure to post the unfinished code somewhere so that someone else may be able to finish it.

[mrjustaguy]

2 RT Related References to add from Godot Contributors Chat:
https://chat.godotengine.org/channel/rendering?msg=rn2NZeMbxNJs5gW4K
https://chat.godotengine.org/channel/rendering?msg=gnYbLKXZFKc7NEwGB

[mrjustaguy]

Another relevant link for this discussion
MIT Unity Realtime Path tracer (Has Compute Fallback for no Hardware RT)
https://github.com/Pjbomb2/TrueTrace-Unity-Pathtracer

[Calinou]

Note for future readers: a pathtracer is designed to work with offline rendering/Movie Maker mode, as opposed to being used during gameplay. The tradeoff is that it offers more accurate rendering compared to hybrid RT, as (almost) everything you see on screen is the result of raytracing.

Ideally, we should support both a hybrid RT approach and pathtracing. While hybrid RT is most suited for gameplay, pathtracing is an useful development tool as it can be used to confirm your scene's lighting is physically accurate. It can also be used to create accurate renderings with Movie Maker mode, which is useful for things like archviz.

When coupled with a denoiser, path tracing is also starting to become viable for gaming (though only on high-end GPUs currently). Future generations of GPUs are expected to make this more viable, especially in laptops and mid-range GPUs.

[radiantgurl]

Bumping thread as there is now a new cinematic renderer being proposed and there are still people wanting full raytracing support inside Godot.

[HydrogenC]

Is the ciematic renderer planned to adopt a Path Tracing approach which is not viable for realtime, or a hybrid Rasterization-RT approarch? Where could I find more technical information about the proposed cinematic renderer (along with the proposed deferred renderer)? Thanks in advance.

[octanejohn]

the correct way is a new renderer with amd gi 1.1/path tracing but a hybrid way with the current renderer for shadows/ao/reflection maybe wanted

https://gist.github.com/reduz/c5769d0e705d8ab7ac187d63be0099b5

#8295

[HydrogenC]

the correct way is a new renderer with amd gi 1.1/path tracing but a hybrid way with the current renderer for shadows/ao/reflection maybe wanted

https://gist.github.com/reduz/c5769d0e705d8ab7ac187d63be0099b5

#8295

Thanks for your information. A fully raytraced pipeline is more cutting edge, but a hybrid way seems more mainstream, at least in present. May be out of performance concerns.

[radiantgurl]

STATUS UPDATE: Work started on implementing hardware raytracing support. godotengine/godot#99119

[zun1uwu]

I'm a beginner, but I recently came across SVOGI (Sparse Voxel Octree Global Illumination) through Crytek's Neon Noir Benchmark (YouTube) and ran the benchmark myself afterwards.

I run a Ryzen 5 3600 and a GTX 1070 Ti and was surprised by the results, because I had an average FPS of around 80 when running the Benchmark on Utra.

So this method of ray tracing has seemingly amazing performance and I think this could potentially be the route Godot could go when implementing something like ray tracing or a great addition to "normal" ray tracing, especially since it runs on machines without specialized RT cores.

But since everyone is talking about ray-tracing with DLSS and other AI upscaling methods to improve the performance of traditional polygon-based ray tracing and SVOGI seems to be kind of "forgotten" (note that the tech demo is 6 years old), there has to be a catch, right?

[Calinou]

The idea of SVOGI is pretty close to Godot's VoxelGI and SDFGI, which are methods to achieve global illumination without raytracing by using coarser approximation structures (voxels and signed distance fields respectively).

Hardware-accelerated raytracing can get much more precise because it works with triangles, not voxels or SDFs. Even though you need to build an acceleration structure (BVH) with simplified geometry and limited draw distance, it will basically always be more precise than voxels or SDFs.

[zun1uwu]

The voxel ray tracing simulation in the benchmark provided reflections that looked extremely detailed (just like the video shows) and very much comparable to traditional polygon ray tracing (which baffled me) in comparison to the ones generated by SDFGI or Godot's VoxelGI, hence I was wondering.

[mrjustaguy]

IIRC that's because at like a few meters of distance they're using short ray tracing for smooth surface reflections, on lower LOD versions of it.

Other engines typically just switch to SSR and use Voxel reflections as a fallback (like Lumen)

[zun1uwu]

Very interesting, thank you for explaining it. Does it also explain why it runs so well on my GTX 1070 Ti? I hope I'm not being bothersome, it's my first time writing in a thread like this (I still feel a little bit clueless about this).

[mrjustaguy]

Yeah, Voxel based GI and SSR are typically quite light, and Low levels of actual Ray Tracing can also be fairly light, it's just that such Low levels of Ray Tracing are typically unusable for most things outside of like basic non soft shadows and basic smooth surface reflections

[TheAenema]

Hi, I’ve developed a real-time path tracer for Godot 4 using latest NVIDIA path tracing technology, DLSS 4.0 (RR) and it doesn't require any changes to the engine. It has the same performance as publicly released NVIDIA path tracer and has zero blit and overhead.
Here's a video showcasing it in action.

My implementation is entirely developed in Jenova Framework and it works on Godot 4.3+ & Windows 10/11 x64, It's very flexible and can be used to render whole scene or just lights, shadows and reflections.

I'm still adding features but I'm pretty happy with the results.