Takes too much RAM loading a texture

[TeamBaconn]

Hi, i'm using react three fiber (the react native compactible version). What is the amount of memory needed to load/render a simple plane with texture? Everytime i spawn a plane with a 1MB image as the texture, it takes me over 200MB of memory storage. Which means if i have 10 plane like that, it would take over 2GB of RAM just to render a basic scene. Have anyone met this situation before? I'm using expo-three TextureLoader to load the texture from the internet since useLoader and THREE.TextureLoader do not work with mobile.

[drcmda]

i don't think that a texture generates 200mb, at least on the web, but that's something you would need to clear with expo and/or threejs. r3f just casts jsx into threejs function calls. maybe @CodyJasonBennett can help you, and i think he's made default threejs loaders work already in the new RN version (r3f v8).

[CodyJasonBennett]

How are you loading your texture? As a module, URL, data URL, etc.? The latter can eat up memory and block the main thread.

I'd let expo-three know about this, but R3F v8 has a working useLoader implementation (shipping in alpha-10). You can also use Drei's hook abstractions like useTexture from their respective native targets.

In the meantime, here's something that utilizes the filesystem using expo-asset and creates a data texture from it. This works for both remote and local assets.

import { Asset } from 'expo-asset'
import * as THREE from 'three'
import { useLoader } from '@react-three/fiber'

const getAsset = (input) => {
  if (input instanceof Asset) return input

  switch (typeof input) {
    case 'string':
      return Asset.fromURI(input)
    case 'number':
      return Asset.fromModule(input)
    default:
      throw 'Invalid asset! Must be a URI or module.'
  }
}

class ExpoTextureLoader extends THREE.Loader {
  async load(url, onLoad) {
    // Generate & cache asset
    const asset = await getAsset(url).downloadAsync()

    // There's no Image in native, so we create & pass a data texture instead
    const texture = new THREE.Texture()
    texture.isDataTexture = true
    texture.image = { data: asset, width: asset.width, height: asset.height }
    texture.needsUpdate = true

    return onLoad(texture)
  }
}

// Suspense is enabled by useLoader here, and the response is cached
const textureSrc = url || require('./path/to/asset')
const texture = useLoader(ExpoTextureLoader, textureSrc)

[TeamBaconn]

How are you loading your texture? As a module, URL, data URL, etc.? The latter can eat up memory and block the main thread.

I'd let expo-three know about this, but R3F v8 has a working useLoader implementation (shipping in alpha-10). You can also use Drei's hook abstractions like useTexture from their respective native targets.

In the meantime, here's something that utilizes the filesystem using expo-asset and creates a data texture from it. This works for both remote and local assets.

import { Asset } from 'expo-asset'
import * as THREE from 'three'
import { useLoader } from '@react-three/fiber'

const getAsset = (input) => {
  if (input instanceof Asset) return input

  switch (typeof input) {
    case 'string':
      return Asset.fromURI(input)
    case 'number':
      return Asset.fromModule(input)
    default:
      throw 'Invalid asset! Must be a URI or module.'
  }
}

class ExpoTextureLoader extends THREE.Loader {
  async load(url, onLoad) {
    // Generate & cache asset
    const asset = await getAsset(url).downloadAsync()

    // There's no Image in native, so we create & pass a data texture instead
    const texture = new THREE.Texture()
    texture.isDataTexture = true
    texture.image = { data: asset, width: asset.width, height: asset.height }
    texture.needsUpdate = true

    return onLoad(texture)
  }
}

// Suspense is enabled by useLoader here, and the response is cached
const textureSrc = url || require('./path/to/asset')
const texture = useLoader(ExpoTextureLoader, textureSrc)

Hi, thanks for helping me out, currently i'm using import {TextureLoader} from "expo-three" to load the URL asset from the internet. Everytime the object loaded & show in the screen, it took me over 200MB RAM until i dispose() it. I've tried your solution but some how I got this wierd error message: https://i.imgur.com/dVPu4Cq.png. I have try drei library too but I can not find the version of Drei to work with the react-three-fiber react native version (which is @v5).

I'm using:
"react-three-fiber": "^5.3.22",
"expo-three": "^5.7.0",
"three": "^0.130.1",

[TeamBaconn]

Hi everyone, I've combined your code and expo textureloader, its something like this
`export default class ExpoTextureLoader extends THREE.TextureLoader {
load(asset, onLoad, onProgress, onError) {
if (!asset) {
throw new Error('ExpoTHREE.TextureLoader.load(): Cannot parse a null asset');
}
let texture = new THREE.Texture();
(async () => {
const fixAsset = await getAsset(asset).downloadAsync()
function parseAsset(image) {
texture.image = image;
texture.needsUpdate = true;
}

        texture['isDataTexture'] = true; // Disable this, texture will not be shown but it wont take too much memory
        parseAsset({
            data: fixAsset,
            width: fixAsset.width,
            height: fixAsset.height,
        });
    })();
    return texture;
}

}`
I use load asset from expo-asset, the code works fine but still consume too much memory.
I think that there is something to do with the texture['isDataTexture']. If i disable this, well the texture will be downloaded and not be shown but it wont cause any memory spikes.

[CodyJasonBennett]

We might be able to recreate this with vanilla three then, but it's between expo-gl and three.

[TeamBaconn]

Hi @CodyJasonBennett, i figure it out. My texture is RGBA mode and have the resolution up to 3500x5000, it must be why the thing consume too much RAM (over 200MB) i supose?

[CodyJasonBennett]

Might be worth trying different modes and textures to confirm.

[TeamBaconn]

Confirmed, I've tried 320x320 texture and it consume maximum 1MB of memory. I thought that my image only weights 1.1MB should weight the same in the memory block but some how three.js takes the resolution into account to. Thanks for helping me.

[MeRahulAhire]

Choose .ktx2 format for best performance, its easy on gpu

Check this out : https://github.com/HacksawStudios/basisu