Issue with misoriented Environment Map

[rlnxocta]

Bevy version

cargo version 0.12.0 and 0.12.1

Relevant system information

Cargo.toml file for building the project

[package]
name = "bevy-test"
version = "0.1.0"
authors = ["Robert Lang"]
edition = "2021"
description = ""

[[bin]]
name = "bevy-test"
path = "src/bin.rs"

[dependencies.bevy]
    version = "0.12.1"

`AdapterInfo { name: "NVIDIA GeForce RTX 3070 Laptop GPU", vendor: 4318, device: 9373, device_type: DiscreteGpu, driver: "NVIDIA", driver_info: "511.79", backend: Vulkan }`

What you did

I was recently messing around with Skyboxes and Environment Maps, built my own one in the Process.
Then i realized that the orientation of the reflections was wrong, and i had no idea why.
I have added some images of my own geometry and the default cube both with calculated tangents and a normal map from the examples with arrows, where the horizon is. You can see, where the horizon supposed to be based on the background, i use the same ktx2 file for it.

What went wrong

When using tangent based normal maps (the blue-ish purple ones), i was expecting the reflections being level, wherever the normal vector in the map would be (0.5, 0.5, 1.0) (which is the normal upright vector).

But the reflections have been rotated, even when using a completely flat normal map

• what were you expecting?
• what actually happened?

Additional information

To reproduce the issue, i have written an example program, that showcases the problem.

//src/bin.rs
use bevy::{prelude::*, core_pipeline::Skybox};


pub fn main() {

    App::new()
        .add_plugins(DefaultPlugins)
        .add_systems(Startup, init)
        .add_systems(Update, rotate_cube)
        .run();

}

#[derive(Component)]
pub struct RotatyCube;

pub fn init(
    mut commands: Commands,
    (mut meshes, mut materials, asset_server): (ResMut<Assets<bevy::prelude::Mesh>>, ResMut<Assets<StandardMaterial>>, Res<AssetServer>),
) {

    let mut mesh: Mesh = shape::Cube{
        size: 1.0,
    }.into();

    _ = mesh.generate_tangents();

    commands.spawn((
        PbrBundle {
            mesh: meshes.add(mesh),
            material: materials.add(StandardMaterial{
                perceptual_roughness: 0.0f32,
                metallic: 1.0f32,
                normal_map_texture: Some(asset_server.load("cube_normal.png")),
                ..Default::default()
            }),
            transform: Transform::from_translation(Vec3::ZERO),
            ..Default::default()
        },
        RotatyCube
    ));

    commands.spawn((
        Camera3dBundle{
            transform: Transform::from_translation(Vec3{ x: -2.0, y: 1.0, z: -1.0 }).looking_at(Vec3::ZERO, Vec3{ x: 0.0, y: 1.0, z: 0.0 }),
            ..Default::default()
        },
        Skybox(asset_server.load("Ryfjallet_cubemap_bc7.ktx2")),
        EnvironmentMapLight {
            diffuse_map: asset_server.load("Ryfjallet_cubemap_bc7.ktx2"),
            specular_map: asset_server.load("Ryfjallet_cubemap_bc7.ktx2"),
        },
    ));

}

pub fn rotate_cube(
    mut rotaty_cubes: Query<(&mut Transform, With<RotatyCube>)>
) {
    rotaty_cubes.for_each_mut(|mut cube| {
        cube.0.rotate_y(0.001);
    });
}

Resources used:

Ryfjallet_cubemap_bc7.ktx2
cube_normal.png

what i have tried:

• I have built a custom shader, in which i calculated the normals myself (though i have copied it from the standard shader impl). Since i suspected that would be the cause, i started disabling single inputs to check, what might be causing it. When i multiplied the world_tangents in the shader by 0.0 (effectively disabling it) the normals were working fine again (albeit without the normal map obviously), hinting it might be the tangents
• I have a custom mesh builder, that applies all kinds of stuff and calculates some things. So to circumvent it, i used a default Shape instead (here the cube), and the problem still persists.
• I use a custom built texture atlas, that also generates normals texture, and have suspected it might be that, so i removed the normal map temporarily completely and saw the normals working fine (but again without the normal map). Adding back in a normal map from the examples makes that issue return again. The problem persisted even, when i supplied a normal map consisting only out of RGBA(127, 127, 255, 255) which is the upright normal vector.

As i already mentioned on the Discord server a few days ago, help might be greatly appreciated, even if it's just a workaround for now. I got a bit frustrated, because i have no clue, what the issue could be and how to solve it. If it is a fault or misunderstanding on my part please point me in the right direction.

thanks a lot already for all the help :)

[rlnxocta]

A little update, i tried to replicate the issue within Blender, since it uses the same mikktspace tangents generation algo.
and i was able to replicate it. However, it was only happening, when i did load the image as sRGB Image and not NonColor Image, which obviously skewed the inputs.

this is with the normal map loaded with sRGB

this is with the normal map loaded with NonColor (and working as intended)

So it looks like the issue might actually lie in how the image is loaded.
I also found, that a potential workaround would be to set the texture type with

image.texture_descriptor.format = TextureFormat::Rgba8Unorm;

but other than in this specific example this is not documented anywhere.

The question is also: what would be the best way, to fix this?
At the very least i would suggest, that some documentation in the standard material normal_map_texture or the generate_tangents is added, to prevent that in the future

[772]

Another similar bug report: #16061 and here is another bug that demands a hint in the docs: #12123.

[Affinator]

Fixed by #20037
This can be closed.