Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image light performance improvements by implementing Framebuffers instead of Graphics. #6599

Merged
merged 2 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/webgl/light.js
Original file line number Diff line number Diff line change
Expand Up @@ -1144,6 +1144,7 @@ p5.prototype.noLights = function(...args) {
this._assert3d('noLights');
p5._validateParameters('noLights', args);

this._renderer.activeImageLight = null;
this._renderer._enableLighting = false;

this._renderer.ambientLightColors.length = 0;
Expand Down
82 changes: 46 additions & 36 deletions src/webgl/p5.RendererGL.js
Original file line number Diff line number Diff line change
Expand Up @@ -480,10 +480,10 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
this.activeImageLight = null;
// If activeImageLight property is Null, diffusedTextures,
// specularTextures are Empty.
// Else, it maps a p5.Image used by imageLight() to a p5.Graphics.
// p5.Graphics for this are calculated in getDiffusedTexture function
// Else, it maps a p5.Image used by imageLight() to a p5.framebuffer.
// p5.framebuffer for this are calculated in getDiffusedTexture function
this.diffusedTextures = new Map();
// p5.Graphics for this are calculated in getSpecularTexture function
// p5.framebuffer for this are calculated in getSpecularTexture function
this.specularTextures = new Map();

this.drawMode = constants.FILL;
Expand Down Expand Up @@ -553,6 +553,7 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
this.executeZoom = false;
this.executeRotateAndMove = false;

this.specularShader = undefined;
this._defaultLightShader = undefined;
this._defaultImmediateModeShader = undefined;
this._defaultNormalShader = undefined;
Expand Down Expand Up @@ -1914,33 +1915,36 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
* To create a blurry image from the input non blurry img, if it doesn't already exist
* Add it to the diffusedTexture map,
* Returns the blurry image
* maps a p5.Image used by imageLight() to a p5.Graphics
* maps a p5.Image used by imageLight() to a p5.Framebuffer
*/
getDiffusedTexture(input) {
// if one already exists for a given input image
if (this.diffusedTextures.get(input) != null) {
return this.diffusedTextures.get(input);
}
// if not, only then create one
let newGraphic; // maybe switch to framebuffer
let newFramebuffer;
// hardcoded to 200px, because it's going to be blurry and smooth
let smallWidth = 200;
let width = smallWidth;
let height = Math.floor(smallWidth * (input.height / input.width));
newGraphic = this._pInst.createGraphics(width, height, constants.WEBGL);
// create graphics is like making a new sketch, all functions on main
// sketch it would be available on graphics
let irradiance = newGraphic.createShader(
defaultShaders.imageLightVert,
defaultShaders.imageLightDiffusedFrag
);
newGraphic.shader(irradiance);
irradiance.setUniform('environmentMap', input);
newGraphic.noStroke();
newGraphic.rectMode(newGraphic.CENTER);
newGraphic.rect(0, 0, newGraphic.width, newGraphic.height);
this.diffusedTextures.set(input, newGraphic);
return newGraphic;
newFramebuffer = this._pInst.createFramebuffer();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes the framebuffer match the size of the main canvas, but we calculate a different size for it above. We can use that size if we do:

Suggested change
newFramebuffer = this._pInst.createFramebuffer();
newFramebuffer = this._pInst.createFramebuffer({
width,
height,
density: 1,
});

Copy link
Contributor Author

@perminder-17 perminder-17 Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great suggestion, it was an oversight by me. I have changed it in specular framebuffer too as we were matching the size of the main canvas before the loop. Although, when we are inside the loop, we are resizing it again.

// create framebuffer is like making a new sketch, all functions on main
// sketch it would be available on framebuffer
newFramebuffer.draw(() => {
let irradiance = this._pInst.createShader(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we can do the same thing we do for this.specularShader, and make it reuse a single shader for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a look now, I have tested it btw and it's totally working.

defaultShaders.imageLightVert,
defaultShaders.imageLightDiffusedFrag
);
this._pInst.shader(irradiance);
irradiance.setUniform('environmentMap', input);
this._pInst.noStroke();
this._pInst.rectMode(constants.CENTER);
this._pInst.noLights();
this._pInst.rect(0, 0, width, height);
});
this.diffusedTextures.set(input, newFramebuffer);
return newFramebuffer;
}

/*
Expand All @@ -1962,31 +1966,37 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
const size = 512;
let tex;
const levels = [];
const graphic = this._pInst.createGraphics(size, size, constants.WEBGL);
const framebuffer = this._pInst.createFramebuffer();
let count = Math.log(size) / Math.log(2);
graphic.pixelDensity(1);
framebuffer.pixelDensity(1);
if (!this.specularShader) {
this.specularShader = this._pInst.createShader(
defaultShaders.imageLightVert,
defaultShaders.imageLightSpecularFrag
);
}
// currently only 8 levels
// This loop calculates 8 graphics of varying size of canvas
// This loop calculates 8 framebuffers of varying size of canvas
// and corresponding different roughness levels.
// Roughness increases with the decrease in canvas size,
// because rougher surfaces have less detailed/more blurry reflections.
for (let w = size; w >= 1; w /= 2) {
graphic.resizeCanvas(w, w);
framebuffer.resize(w, w);
let currCount = Math.log(w) / Math.log(2);
let roughness = 1 - currCount / count;
let myShader = graphic.createShader(
defaultShaders.imageLightVert,
defaultShaders.imageLightSpecularFrag
);
graphic.shader(myShader);
graphic.clear();
myShader.setUniform('environmentMap', input);
myShader.setUniform('roughness', roughness);
graphic.noStroke();
graphic.plane(w, w);
levels.push(graphic.get().drawingContext.getImageData(0, 0, w, w));
}
graphic.remove();
framebuffer.draw(() => {
this._pInst.shader(this.specularShader);
this._pInst.clear();
this.specularShader.setUniform('environmentMap', input);
this.specularShader.setUniform('roughness', roughness);
this._pInst.noStroke();
this._pInst.noLights();
this._pInst.plane(w, w);
});
levels.push(framebuffer.get().drawingContext.getImageData(0, 0, w, w));
}
// Free the Framebuffer
framebuffer.remove();
tex = new MipmapTexture(this, levels, {});
this.specularTextures.set(input, tex);
return tex;
Expand Down
Loading