Data used as texture for a Plane

[iamterryclark]

How would I load a Uint8Array of data into a texture to use on a plane? I believe I need to use a DataTexture in order to do this but I seem to just get a black rectangle.

So far I am doing something like this.. am I missing something? I tried using lighting also but still just a black plane. I have checked the dataTexture and there is valid data inside.

dataTexture = new THREE.DataTexture( myUint8Array, 1280, 720, THREE.RGBAFormat );
dataTexture.needsUpdate = true;

<Canvas 
    camera={{ position: [0, 0, 8], fov: 20 }}
    width={1280}
    height={720} >            
    <mesh scale={[1, 1, 1]}>
        <planeGeometry attach="geometry" args={[1, 1, 1]} />
        <meshStandardMaterial attach="material" map={dataTexture } />
    </mesh>                    
</Canvas>

[drcmda]

i have never used them but if that's what the material takes, then yes, you do it in the exact way you'd do it in plain three. the only thing that's wrong here is that you create objects smack in the middle of the render function, so they will re-instanciate on every render. always memoize, or useState for statics:

// static
const [foo] = useState(() => new Foo())
// memoized constructor args
const bar = useMemo(() => new Bar(xyz), [xyz])

[iamterryclark]

Hey Again,

Just to extend this query further after running some tests.

I now have the Uint8Array video buffer being sent to the DataTexture and this now renders to a Plane object. My concern is that when I open the task manager I can see memory increases dramatically. This happens when the component unloads and reloads. I wondered if I have implemented this correctly using react.

Note: the canvas does resize as well, whether this makes a difference

import React, {useState, useContext, useEffect, useMemo} from "react";
import Context from "context";

import * as THREE from 'three'
import { Canvas } from 'react-three-fiber'

let canvas_x = 1920;
let canvas_y = 1080;

const VideoBufferDisplay = () => {
    const [buffer, setBuffer] = useState(()=>new Uint8Array(canvas_x * canvas_y));
    const texture = useMemo(() => new THREE.DataTexture(buffer, canvas_x, canvas_y, THREE.RGBAFormat,THREE.UnsignedByteType), [buffer, canvas_x, canvas_x]);

    const [{incomingBuffer}, dispatch] = useContext(Context);

    useEffect(() =>{
        texture.magFilter = THREE.NearestFilter;
        texture.minFilter = THREE.NearestFilter;
        texture.needsUpdate = true;
    },[])

     useEffect(() => {
        setInterval(() =>{
            setIpcBuffer(incomingBuffer.readBuffer(canvas_x, canvas_y));
        }, 40);
    }, [incomingBuffer])

    return (
        <Canvas id="videoCanvas"  }>
            <ambientLight />
                <mesh position={[0,0,0]} scale={[1, -1, 1]}>
                    <planeBufferGeometry attach="geometry" args={[16, 9, 1]} />
                    <meshBasicMaterial attach="material" map={texture} />
                </mesh>
        </Canvas>
    )
}

[drcmda]

you dont dispose these textures, better do this:

const canvas_x = 1920
const canvas_y = 1080

function VideoBufferDisplay() {
  const [buffer, setBuffer] = useState(() => new Uint8Array(canvas_x * canvas_y))
  const ref = useRef()
  useLayoutEffect(() => {
    const texture = new THREE.DataTexture(buffer, canvas_x, canvas_y, THREE.RGBAFormat, THREE.UnsignedByteType)
    texture.magFilter = THREE.NearestFilter
    texture.minFilter = THREE.NearestFilter
    texture.needsUpdate = true
    ref.current.map = texture
    return () => texture.dispose()
  }, [buffer])

  const [{ incomingBuffer }] = useContext(Context)
  useEffect(() => {
    setInterval(() => setBuffer(incomingBuffer.readBuffer(canvas_x, canvas_y)), 40)
  }, [incomingBuffer])

  return (
    <Canvas id="videoCanvas">
      <ambientLight />
      <mesh position={[0, 0, 0]} scale={[1, -1, 1]}>
        <meshBasicMaterial ref={ref} attach="material" />

but still, you're creating gigantic video textures every 40ms, that is a recipe for disaster. the GC won't be able to dump this much data at once, so it will accumulate and then fush in one giant frame skipping op, i dont understand why you don't just create one texture and then alter the buffer.

[iamterryclark]

Thank you for the reply!

Hmm I thought my example above was already altering the buffer and setting one texture? Unless you mean to initialise a texture outside of the component?

Also should I be able to set the buffer width and height like so?

texture.image.data = buffer;
texture.image.width = canvas_x;
texture.image.height = canvas_y;