Editing a texture between frames by swapping renderTargets

[ancientjpeg]

Hi there!

I'm brand new to r3f, and pretty green on React and JS in general. I'm trying to take an image and distort it every frame with a dedicated distortion fragment shader, and then render the result to the canvas. As reading from and writing to a single shader is generally undefined behavior in Open/WebGL, I'm trying to use the common tactic of reading from one render target, writing to a second, and then swapping them each frame. However, instead of getting the desired effect (slowly mutating the image over time), I'm just getting a rapidly flickering render of my image.

Here's my code:

import React, { useMemo, useRef } from "react";
import { Canvas, useFrame } from "@react-three/fiber";
import styles from "../../styles/Render.module.scss";
import * as THREE from "three";

const vertShaderString = `
precision mediump float;

varying vec2 texCoord;

void main() {
  gl_Position = vec4(uv, 0.0, 1.0);
  texCoord = uv;
}`;

const distortionFragShaderString = `
precision mediump float;

uniform sampler2D u_tex;

varying vec2 texCoord;

void main() {
    vec4 sample_col = texture2D(u_tex, texCoord);
    // MESS WITH THE SAMPLE COLOR HERE
    gl_FragColor = sample_col;
}
`;

const outputFragShaderString = `
precision mediump float;

uniform sampler2D u_tex;

varying vec2 texCoord;

void main() {
    vec4 sample_col = texture2D(u_tex, texCoord);
    gl_FragColor = sample_col;
}
`;

function ShaderRenderTest(props: any) {
  const gl_ref: any = useRef();

  var [dist_uniforms, output_uniforms, target_0, target_1] = useMemo(() => {
    let target_0 = new THREE.WebGLRenderTarget(1024, 1024);
    let target_1 = new THREE.WebGLRenderTarget(1024, 1024);
    const d_unif = {
      u_time: { value: 0.0 },
      u_tex: { value: new THREE.Texture() },
    };
    const o_unif = {
      u_tex: { value: new THREE.Texture() },
    };
    return [d_unif, o_unif, target_0, target_1];
  }, []);

  target_0.texture = new THREE.TextureLoader().load("/img/example_image.jpg");
  target_1.texture = target_0.texture.clone();

  const distortion_mat = new THREE.ShaderMaterial({
    uniforms: dist_uniforms,
    vertexShader: vertShaderString,
    fragmentShader: distortionFragShaderString,
  });

  // Subscribe this component to the render-loop, rotate the mesh every frame
  useFrame(({ gl, scene, camera }, delta) => {
    const mat = gl_ref.current.material;
    distortion_mat.uniforms.u_time.value += delta;

    mat.uniforms.u_tex.value = target_0.texture;

    gl.autoClear = false;
    scene.overrideMaterial = distortion_mat;
    gl.setRenderTarget(target_1);
    gl.render(scene, camera);

    mat.uniforms.u_tex.value = target_1.texture;

    const temp = target_0.texture;
    target_0.texture = target_1.texture;
    target_1.texture = temp;

    gl.autoClear = true;
    scene.overrideMaterial = null;
    gl.setRenderTarget(null);
    gl.render(scene, camera);
  });

  return (
    <mesh ref={gl_ref}>
      <planeGeometry />
      <shaderMaterial
        uniforms={output_uniforms}
        fragmentShader={outputFragShaderString}
        vertexShader={vertShaderString}
      />
    </mesh>
  );
}

const Render = () => {
  return (
    <div id={styles["render-container"]}>
      <Canvas>
        <ShaderRenderTest />
      </Canvas>
    </div>
  );
};

export default Render;

I apologize in advance for any rookie React mistakes. I come from a C/C++ background, so the very declarative nature of React and r3f is still something I'm getting used to! I'd appreciate any tips/guidance.

Thank you!

[l0x539]

Any updates regarding this issue?

[ancientjpeg]

Unfortunately, I have yet to get this working properly. I put this project on the backburner but I've been considering taking another crack at it, I'll post here if I manage to figure it out.