Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGPURenderer: wrong update of compressed textures #29786

Open
vlucendo opened this issue Nov 1, 2024 · 7 comments
Open

WebGPURenderer: wrong update of compressed textures #29786

vlucendo opened this issue Nov 1, 2024 · 7 comments

Comments

@vlucendo
Copy link
Contributor

vlucendo commented Nov 1, 2024

Description

Updating compressed textures with the contents of others of different formats by copying them fails sometimes and have visual errors, but the same functionality works fine in WebGL. This functionality is often used when creating an empty texture while the asset loads, and then copying the loaded data to it once it's available.

I created a couple of fiddles with webgpu and webgl to compare. Webgpu only updates 3 times instead of 4 and displays a wrong texture.

Reproduction steps

(see fiddles)

  • Create an empty compressed texture
  • Update it by copying others

Code

See fiddles

Live example

Screenshots

No response

Version

r170

Device

Desktop

Browser

Chrome

OS

Windows

@vlucendo vlucendo changed the title WebGPURenderer: wrong update of compressed texures WebGPURenderer: wrong update of compressed textures Nov 1, 2024
@vlucendo
Copy link
Contributor Author

vlucendo commented Nov 1, 2024

I updated the links to show an example of textures displaying wrong as well.

@GitHubDragonFly
Copy link
Contributor

Just a minor addition, support for one of the texture formats also appears to be missing.

This could be tested by using the fiddle and changing the initial texture from 0 to 1 and then 2 and then 3 (this last one should show the error related to Unsupported format 1023).

@RenaudRohlinger
Copy link
Collaborator

The compression format is properly supported and works as expected when the texture is initialized, for example:

const textureNode = THREE.texture(textures[3]);

However, the issue seems to be that the WebGPURenderer or the NodeBuilder doesn't yet handle dynamic reassignment of the compressedTexture format.

@GitHubDragonFly
Copy link
Contributor

@RenaudRohlinger I am not an expert for these things but that texture seems to be causing some issues.

You and @vlucendo could try the following code in the WebGPU fiddle, which is using a different texture and is also disposing of each texture. It seems to work fine for switching all 4 textures but I am not sure how accurate it is.

import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
import { KTX2Loader } from 'three/addons/loaders/KTX2Loader.js'

let mesh, renderer, scene, camera;

init();

async function init() {

    // renderer
    renderer = new THREE.WebGPURenderer();
    renderer.setSize( window.innerWidth, window.innerHeight );
    renderer.setPixelRatio( window.devicePixelRatio );
    document.body.appendChild( renderer.domElement );
    
    // wait for renderer
    await renderer.init();
   
    // scene and camera
    scene = new THREE.Scene();
    scene.background = new THREE.Color( 0x000000 );
    camera = new THREE.PerspectiveCamera( 40, window.innerWidth / window.innerHeight, 1, 10000 );
    camera.position.set( 0, 0, 20 );

    // mesh
    const geometry = new THREE.SphereGeometry( 5, 64, 32 );
    const material = new THREE.MeshBasicNodeMaterial();
    mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    
    
    // bug report

    
    // create loader
    const loader = new KTX2Loader();
    loader.detectSupport(renderer);
    loader.setTranscoderPath('https://unpkg.com/three/examples/jsm/libs/basis/');
    
    // load texture
    const textures = await Promise.all([
        loader.loadAsync('https://threejs.org/examples/textures/compressed/2d_uastc.ktx2'),
        loader.loadAsync('https://threejs.org/examples/textures/compressed/2d_etc1s.ktx2'),
        loader.loadAsync('https://threejs.org/examples/textures/compressed/sample_uastc_zstd.ktx2'),
        loader.loadAsync('https://threejs.org/examples/textures/compressed/sample_etc1s.ktx2'),
    ]);
    
    const compressedTexture = new THREE.CompressedTexture();
    const textureNode = THREE.texture(compressedTexture);
    
    // create color node
    const getWGSLTextureSample = THREE.wgslFn( `
					fn getWGSLTextureSample( tex: texture_2d<f32>, tex_sampler: sampler, uv:vec2<f32> ) -> vec4<f32> {
						return textureSample( tex, tex_sampler, vec2f(uv.x, 1.0 - uv.y) );
					}
		`);
    material.colorNode = getWGSLTextureSample( { tex: textureNode, tex_sampler: textureNode, uv: THREE.uv() } );

		// assign first texture before render. if the texture on index 2 is selected first, none updates
    compressedTexture.copy(textures[0]);

		// change texture periodically: it should change 4 times but only changes 3 and has visual errors
		let textureIndex = 0;
		setInterval(() => {
    		textureIndex = (textureIndex + 1) % textures.length;
    		compressedTexture.dispose();
    		compressedTexture.copy(textures[textureIndex]);
        compressedTexture.needsUpdate = true;
    }, 500);

    // start animation
     renderer.setAnimationLoop( animate );
}

function animate() {

    renderer.render( scene, camera );

}

@GitHubDragonFly
Copy link
Contributor

GitHubDragonFly commented Nov 12, 2024

Just to mention that, with respect to the current code of the KTX2 Loader, this might not really be an issue but just a wrong approach to treating all returned textures as THREE.CompressedTexture.

Even though the documentation suggests that the load method would return THREE.CompressedTexture, the loader's code suggests that it could also be THREE.DataTexture or THREE.Data3DTexture - logging all loaded textures from the original WebGPU fiddle shows that the 4th texture was loaded with isDataTexture instead of with isCompressedTexture.

The WebGPU fiddle's code seems to specifically treat them all as THREE.CompressedTexture:

    const compressedTexture = new THREE.CompressedTexture();
    const textureNode = THREE.texture(compressedTexture);

As stated before, I am not an expert for these things and will leave it to the experts to figure it out.

@vlucendo
Copy link
Contributor Author

vlucendo commented Nov 13, 2024

KTX2Loader was returning a DataTexture for the last texture, so I replaced it so they are all CompressedTextures on the following updated fiddles:

WebGPU
WebGL

The WebGPU version still seems to have issues, updating 2 times instead of 4.
The WebGL version always updates, but also seems to display an error with the new texture now.

Doing .dispose() on the used compressed texture before new data is being copied to it indeed seems to fix the unexpected behaviour, however one would expect that both WebGL and WebGPU worked similarly by just copying the new data and doing needsUpdate = true in my opinion.

Some loaders in core (TextureLoader or CompressedTextureLoader for example) seem to expect this: they return and empty texture and when the data is loaded, they copy it and flag needsUpdate on the previously returned texture.

@GitHubDragonFly
Copy link
Contributor

As a wild guess, maybe those could be 2 things that require correction:

  • KTX2 Loader to return some sort of unified CompressedTexture for any loaded format
  • CompressedTexture to internally use dispose() within copy() function

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants