Can't remove texture on state change. Is this a bug or a feature?

[BrendanBerkley]

https://codesandbox.io/s/smoosh-firefly-83x6zs?file=/src/App.js

In each of those instances, if you replace a texture with a different texture (uncomment flowerTextureProps), it works as expected. If you hover and change the material color, it works as expected. But if you swap out a map texture for null:

<meshStandardMaterial map={!hovered ? wallTextureProps : null} />

It doesn't work. It doesn't work if you try changing the map prop, or swapping out <meshStandardMaterial>, or swapping out the entire mesh.

If I add onUpdate={(material) => console.log("update!", material.map)} into that meshStandardMaterial, it looks like what I'd expect: logs null on hover, logs a texture when not hovered.

What is happening here? I can't tell if this is a bug or a feature.