Better lighting & rendering

[damro]

The lighting and rendering in the scenes needs to be improved.

Here are some recommendations from Micheal Mara, a graphics expert:

"There are three major things that is causing the discrepancy:

a) The material setup
b) The lighting setup
c) The rendering algorithm
These all effect each other in noticeable ways, but a first order approximation is that improving any one in isolation will improve the final render quality. I will discuss each in turn.

a) It seems like the glossy BRDF parameters are not being interpreted properly by Blender, the materials appear to be completely diffuse (no glossy component at all). In blender you need to make sure you are using a Specular Shader: https://www.blender.org/manual/render/blender_render/materials/properties/specular_shaders.html. In Unity you want to increase metalness and smoothness to get some reflections: https://docs.unity3d.com/Manual/Materials.html

b) The lighting in the blender example seems to be a single directional light from the left with no shadowing whatsoever. This is always going to look strange. It would look slightly less strange with shadows enabled (like they should be by default in Unity), but still won't look particularly good on shiny materials. You need some way of having light coming in from all directions. The Cycles render seems to have uniform white light coming in from all directions, the subtle gradations and interesting light is all coming from multiple scattering events (light bouncing multiple times before hitting the camera). This is going to be really hard to reproduce in realtime, I would recommend having some sort of spatially varying HDR environment lighting (https://hdrihaven.com/bundle.php?b=free_bundle) instead, which will produce decent results in real time in Unity (assign the HDR environment map to the skybox). I don't know if Blender has an option to use environment lighting in its non-Cycles renderer. You can make a (very rough) approximation to having full environment lighting by combining multiple point and spot lights together (https://wiki.blender.org/index.php/Doc:2.4/Tutorials/Lighting/Three_Point_Light). This is inferior to full environment lighting but should start to show some detail. If you aren't seeing bright highlights on your metallic rings doing any of these changes, your materials are still off.

c) Here's the biggest problem. No realtime solution is going to be able to offer you everything you want here. Accurate multibounce reflections (especially if not-quite-perfect-mirror) are completely out of the question for anything without raytracing, and raytracing will take too long (and isn't built in to Unity anyways). Better materials and lighting will go a long way to making your images appear plausible to humans without changing the rendering algorithm... whether that is sufficient for your purposes remains unknown. Pretty much all real-time renderers work by only calculating direct illumination (light that bounces off a single surface and comes directly into the camera), and only correctly for infinitesimal or infinitely far away lights, approximating all other light (indirect illumination) with one of a select few complete hacks (for example, manually placing "fake" lights that stand in for the first bounce of indirect light. For an overview of vanilla Unity rendering options, you can check out the official documentation: https://unity3d.com/learn/tutorials/topics/graphics/unity-5-lighting-and-rendering. These make many tradeoffs in terms of "bake" time, whether lighting can be changed at runtime, the accuracy of the approximation and the run time cost. For most games today, when it comes to approximating real-time reflections, they often use screen-space reflections (available for Unity here: https://www.assetstore.unity3d.com/en/#!/content/51515) augmented with artist-placed "reflection probes" (https://docs.unity3d.com/Manual/class-ReflectionProbe.html) for the many obvious failure cases of screen-space reflections. This is likely too labor intensive for your purposes and you'll want a more general, if inefficient and lower-quality solution. This (http://www.sonicether.com/segi/) is a commercialized version of some recent research into fast indirection illumination calculations, but I can't comment on its particular limitations or quality, having never used the plugin myself. I would be wary given its "beta" nature, but it is likely the most sophisticated technique you'll be able to find that works at realtime rates."

So I will work on improving the lighting by finding a combination of lighting techniques that works best by combining the following:

1. Global Illumination (GI):
   • HDR Tone mapping: https://hdrihaven.com/bundle.php?b=free_bundle
   • Directional Light / Sun light
   • Pre-bake lighting for static elements (mainly walls, windows etc.)
2. Direct Light Illumination Setup: https://unity3d.com/learn/tutorials/topics/graphics/unity-5-lighting-and-rendering
   • Point Lights & Spot Lights (both static and dynamic objects, be aware of light leaks and 6x rendering, spot lights as wall down lights or flash lights)
   • Area Lights & Emissive Materials (light static objects only, can be expensive, emissive lights can change whereas area lights are static)
   • Light probes (to light dynamic objects from global illumination, place denser where illumination changes)
   • The total light energy in the scene should be kept constant after finding a good value to avoid over exposing the scene with light.
   • Optional: Consider three-point-light setup: 1 key light at front (Spot) + 1 back light from back (Lamp) + 1 Fill light from camera perspective; if bright floors are used, an additional floor diffuse light is needed which is usually placed mirrored to the key light for floor diffusion. The color of this light should match the color of the floor (shadeless spot light).
3. Reflection probes (~ 1 per room): https://docs.unity3d.com/Manual/class-ReflectionProbe.html
4. Screen Space Reflections (free asset): https://www.assetstore.unity3d.com/en/#!/content/51515
5. Realtime Indirect Light / Global Illumination (80 $, needs DX11): http://www.sonicether.com/segi/ | https://www.assetstore.unity3d.com/en/#!/content/64763

[pbashivan]

Thanks for the comment!
How are these solutions different than changing the quality settings in unity project setting (Edit/Project Settings/Quality)?

[damro]

In graphics, in order to create realistic scenes you need to have the lights setup appropriately. Rendering on high quality will do poorly if the lights are setup wrongly. For instance I read, that area lights without light probes do not light dynamic objects in Unity, which are most of the objects in our scenes.
The goal should be to create a lighting setup, where light comes from all directions outdoors, and the windows and lights indoors respectively. Therefore, we need a global illumination setup additionally to the direct lights choosing the correct lights. Reflection probes are necessary to let reflective objects reflect more than the outside sky. Screen space reflections and realtime indirect lighting will add indirect lighting, which, without any extensions, can only be pre-baked in Unity.
So basically good lighting is a prerequisite for good rendering.

[yamins81]

@damro is mostly "finished" with item 1 above and parts of 2 involving total light control, point lights --> spot lights, and item 3 (reflection probes). Next he will test using scene generation code like in curiosity. then he will consider doing some of the other items (4 and 5 above and maybe light probes / area lights)

[damro]

More items:
6. Screen Space Ambient Obscurance: https://docs.unity3d.com/Manual/script-ScreenSpaceAmbientObscurance.html
7. Screen Space Indirect Illumination: - Michael will send link
8. Tone mapping: https://www.assetstore.unity3d.com/en/#!/content/51515
9. Dynamic Area Lights: - Michael will send link
10. Write Ns to Smoothness conversion function

[damro]

Progress update:

• Finished with item 1., partly 2., 3., 4., 6., and 8.
• Need further input from Michael Mara to complete items 7., 9. (same as remaining part of 2.).
• Item 5. seems no longer relevant as it only runs on Windows.
• Item 10. has been posted in GitHub issue Improving Materials and fixing the MaterialProcessor.cs #63.

It remains to be analyzed which of these are worth the additional computation time and how any of these items can be sped up.