WebGLRenderer: Added support for texSubImage2D

[Mugen87]

This PR adds basic support for WebGLRenderingContext.texSubImage2D() which can be used to replace parts of an existing 2D texture with all data of another texture. Instead of enhancing Texture and WebGLTextures, i've just added WebGLRenderer.copyTextureToTexture(). So a simple interface that allows access to texSubImage2D.

Latest feature request in the forum: https://discourse.threejs.org/t/best-way-to-texsubimage2d/2052

[mrdoob]

Nice solution!

[mrdoob]

Thanks!

[pardoman]

Is this file manually updated or is there a script that re-generates this file? @Mugen87

[donmccurdy]

This file is edited manually, and changes will appear on http://threejs.org/docs/ with the next version release.

[mbppower]

Hi, the call to texSubImage2D fails when using WebGL1 renderer, by loading an image rather than a data texture. A check should be made, maybe using _isWebGL2.

Ref: https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/texSubImage2D

[takahirox]

I think #13944 has fixed. Have you tried the latest dev branch?

[andreasplesch]

texSubImage2D seems to be faster than texImage2D for uploading (https://kripken.github.io/emscripten-site/docs/optimizing/Optimizing-WebGL.html#gpu-driver-friendly-memory-access-behavior). It seemed to make a difference for videotextures updates but this will need more research . Perhaps consider using texSubImage2D for videotexture updates until the video texture webgl extension becomes available.

[mrdoob]

@andreasplesch We'd definitely need to test and measure such change in as many platforms as possible. Would you be up for giving it a go?

[andreasplesch]

Hm, let's just keep playing a bit since I cannot dedicate really any time and I feel not too comfortable with the three code flow. x3dom/x3dom#909 was the original reason I looked into this.

Collecting some info first:

three.js/src/renderers/webgl/WebGLTextures.js

Line 605 in 34dc247

state.texImage2D( _gl.TEXTURE_2D, 0, glFormat, glFormat, glType, image );

is where the video update happens ?

going through a state manager

three.js/src/renderers/webgl/WebGLState.js

Line 833 in 34dc247

function texImage2D() {

which has the gl context.

In order to use texSubImage2D, texImage2D had to be used once earlier. I did not see an initialization or a special case for a first upload. It may be necessary to add a texture.initialized property or something like that.

Perhaps a quick approach for testing would be to disable the needsUpdate resetting for video in

three.js/src/textures/VideoTexture.js

Line 27 in 5989b78

update: function () {

and then manage the updates manually in the RAF callback.
First just setting .needsUpdate=true again, to see how that works with regular texImage2D updating.
Then following the example above and using the new copyTexture2Texture method.
But this may not be quite an apples to apples test.

Let's see if there is simple videotexture example:
https://github.com/mrdoob/three.js/blob/master/examples/webgl_video_panorama_equirectangular.html
looks simple enough.

Any pointers much welcome.

[andreasplesch]

Documenting experiments so they do not get lost:
modifying

three.js/src/renderers/webgl/WebGLTextures.js

Line 605 in 34dc247

state.texImage2D( _gl.TEXTURE_2D, 0, glFormat, glFormat, glType, image );

to

else {

	if ( texture.uploadedOnce ) {
		state.texSubImage2D( 3553, 0, 0, 0, glFormat, glType, image );
	}

	else {
		state.texImage2D( 3553, 0, glInternalFormat, glFormat, glType, image );
		texture.uploadedOnce = true;
	}
	textureProperties.__maxMipLevel = 0;
}

seems to work after adding texSubImage2D() to state.
Actually easier than external, manual updates.
Time to try giant textures.

[mrdoob]

Out of curiosity... Are you updating every frame or are you updating 25/30fps (in sync with the video)?

[andreasplesch]

Well, let's see how it matters. I made the changes to use texSubImage2D in
https://github.com/andreasplesch/three.js/blob/texSubImage2D/src/renderers/webgl/WebGLTextures.js
and modified examples to use it.

https://andreasplesch.github.io/three.js/examples/#webgl_video_panorama_equirectangular_sub

uses texSubImage2D and RAF to update. I added three movies with different resolutions and native FPS speeds, and a FPS stats counter.

https://andreasplesch.github.io/three.js/examples/#webgl_video_panorama_equirectangular

uses current dev and has the same options.

For setInterval updates, I modified

https://andreasplesch.github.io/three.js/examples/#webvr_video_sub

to use texSubImage, with needUpdates at 60Hz and four movies.

https://andreasplesch.github.io/three.js/examples/#webvr_video uses current dev,

I am using latest FF (63.03) and Chrome ( 70.0.3538.102) on a lower end laptop with

Intel HD Graphics 4400

On that, texSubImage does not make a large difference, and is one case much slower. Overall FF performs better. So on that configuration texSubImage2D does not make sense.

Added Edge numbers. Edge does 60fps for any combination. Would need larger texture.

Browser	Test	1920x1080_30fps	1920x1080_25fps	3186x1584_30fps	2048x1024_24fps
Chrome	pano	60	25	11
Chrome	pano_sub	18{!)	25	11
Chrome	webvr	62(?)	35	13	30
Chrome	webvr_sub	25	13	6	14(*)
FF	pano	60	60	55(!)
FF	pano_sub	50	45	45
FF	webvr	60	50	48	50
FF	webvr_sub	50	48	50	48
Edge	pano	60	60	60
Edge	pano_sub	60	60	60
Edge	webvr	60	60	60	60
Edge	webvr_sub	60	60	60	60

(*) I discovered that the devtools FPS meter shows double the rates for webvr_sub. So these are suspicious

Let's see how it looks elsewhere.

[andreasplesch]

On cheap Android phone (Moto E 2nd) with Adreno 306, on Chrome, texSubImage also does worse, in some cases (Dolphin, 1920x1080_30fps) significantly.

[andreasplesch]

On Linux, Chrome Version 70.0.3538.77, and Firefox, with Quadro P4000

Browser	Test	1920x1080_30fps	1920x1080_25fps	3186x1584_30fps	2048x1024_24fps
Chrome	pano	30	30	30
Chrome	pano_sub	30	30	30
Chrome	webvr	30	30	30	30
Chrome	webvr_sub	15	15	15	15
FF	pano	60	60	60
FF	pano_sub	60	60	60
FF	webvr	60	60	60	60
FF	webvr_sub	60	60	60	60

Hm, I discovered that the devtools FPS meter shows double the rates for webvr_sub.

So, here subTexImage2D is not rate limiting, something else is.