Refining the number of samples used when creating diffuse and specular textures.

[Garima3110]

Increasing Access

The goal is to find an optimal configuration that ensures smooth execution on various devices while maintaining an acceptable level of visual fidelity. This optimization process is essential for enhancing the overall performance of the feature as ideated by @davepagurek in issue #6531

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build Process
• Unit Testing
• Internalization
• Friendly Errors
• Other (specify if possible)

Feature enhancement details

Refine into the number of samples used when creating diffuse/specular textures. It may still look OK if we use fewer samples but apply dithering to the sample locations, which is a strategy we use in our blur shader.

[sudhanshuv1]

I want to work on this issue.

[davepagurek]

Thanks @sudhanshuv1! The way to test this is to try measuring the time it takes to call imageLight() for the first time using different numbers of samples in these spots, and how smooth the results look:

p5.js/src/webgl/shaders/imageLightDiffused.frag

Lines 51 to 56 in 773eb0d

	const float sampleDelta = 0.025;
	float nrSamples = 0.0;
	for(float phi = 0.0; phi < 2.0 * PI; phi += sampleDelta)
	{
	for(float theta = 0.0; theta < ( 0.5 ) * PI; theta += sampleDelta)

p5.js/src/webgl/shaders/imageLightSpecular.frag

Line 49 in 773eb0d

for (int i = 0; i < SAMPLE_COUNT; ++i)

One other thing that can improve performance for specular textures in WebGL2 is here:

p5.js/src/webgl/shaders/imageLightSpecular.frag

Line 89 in 773eb0d

float VanDerCorput(int n, int base)

This is working around having no bitwise operations in WebGL1. We can probably replace its implementation with a more efficient version if we're using WebGL2:

float VanDerCorput(int n, int base)
{
#ifdef WEBGL2

    uint bits = uint(n);
    bits = (bits << 16u) | (bits >> 16u);
    bits = ((bits & 0x55555555u) << 1u) | ((bits & 0xAAAAAAAAu) >> 1u);
    bits = ((bits & 0x33333333u) << 2u) | ((bits & 0xCCCCCCCCu) >> 2u);
    bits = ((bits & 0x0F0F0F0Fu) << 4u) | ((bits & 0xF0F0F0F0u) >> 4u);
    bits = ((bits & 0x00FF00FFu) << 8u) | ((bits & 0xFF00FF00u) >> 8u);
    return float(bits) * 2.3283064365386963e-10; // / 0x100000000

#else

  float invBase = 1.0 / float(base);
  float denom = 1.0;
  float result = 0.0;


  for (int i = 0; i < 32; ++i)
  {
        if (n > 0)
        {
        denom = mod(float(n), 2.0);
        result += denom * invBase;
        invBase = invBase / 2.0;
        n = int(float(n) / 2.0);
        }
  }


  return result;

#endif
}

[sudhanshuv1]

Thanks @davepagurek for the guidance. I tried using different numbers of samples in both diffused and specular terms of imageLight, and as expected the results became less smooth when the number of samples were decreased:

Currently the values are set to sampleDelta=0.025; SAMPLE_COUNT=1024, but the level of visual fidelity was still acceptable with the values sampleDelta=0.100; SAMPLE_COUNT=16. Upon decreasing the number of samples further, the results became noticeably worse. To measure the time it took for imageLight() to execute I used performance.now(), but the code I used logged the same time duration in each case:

let img;
let slider;
function preload() {
  img = loadImage('assets/outdoor_spheremap.jpg');
}
function setup() {
  createCanvas(400, 400, WEBGL);
  slider = createSlider(0, 400, 100, 1);
  slider.position(0, height);
}
function draw() {
  background(220);
  imageMode(CENTER);
  push();
  translate(0, 0, -200);
  scale(2);
  image(img, 0, 0, width, height);
  pop();
  ambientLight(50);
  start=performance.now();
  imageLight(img);
  end=performance.now();
  console.log(end-start);
  specularMaterial(20);
  shininess(slider.value());
  noStroke();
  scale(2);
  sphere(15);
}

If we use the values sampleDelta=0.100; SAMPLE_COUNT=16 instead of the current ones, it might result in smoother execution because the number of samples are reduced. But I couldn't measure it. Also, it would be really helpful if you could provide any hints on how to apply dithering to the sample locations. Thanks!

[davepagurek]

oh I think you might have to take your end time measurement after you draw the sphere, since that's when it evaluates all the image light.

Thanks for your tests so far, this looks great! I'm guessing we'll end up going to the lower sample counts that you recommend, but we can measure the time again first just to see how much of a gain we're getting.

We do some sample dithering in the blur shader here, maybe you can try doing something like this in the image light shaders too?

p5.js/src/webgl/shaders/filters/blur.frag

Line 50 in 37d3324

float sample = i * spacing - (numSamples - 1.0) * 0.5 * spacing + randomOffset;

[sudhanshuv1]

Hey @davepagurek, as you suggested, I took the end time measurement after the sphere was drawn. When I used the values sampleDelta=0.100; SAMPLE_COUNT=16, I could achieve a gain of about 2 sec. Further decrement in number of samples didn't result in significant time gain, as is evident from this data I gathered - I had taken the time measurements with and without dither:

I applied dithering using the same random function that is used in the blur filter here, and tested the results using this code in imageLightDiffused.frag:

float random(vec2 p) {
  vec3 p3  = fract(vec3(p.xyx) * .1031);
  p3 += dot(p3, p3.yzx + 33.33);
  return fract((p3.x + p3.y) * p3.z);
}

void main()
{   	 
  float phi = vTexCoord.x * 2.0 * PI;
  float theta = vTexCoord.y * PI;
  float x = sin(theta) * cos(phi);
  float y = sin(theta) * sin(phi);
  float z = cos(theta);
  vec3 normal = vec3( x, y, z);

  vec3 irradiance = vec3(0.0);  
  vec3 up	= vec3(0.0, 1.0, 0.0);
  vec3 right = normalize(cross(up, normal));
  up = normalize(cross(normal, right));
  
  const float sampleDelta = 0.100;
  float nrSamples = 0.0;
  
  for(float phi = 0.0; phi < 2.0 * PI; phi += sampleDelta)
  {
    for(float theta = 0.0; theta < ( 0.5 ) * PI; theta += sampleDelta)
    {
      float x = sin(theta) * cos(phi);
      float y = sin(theta) * sin(phi);
      float z = cos(theta);
      vec3 tangentSample = vec3( x, y, z);

      float randomOffset = mix(-0.1, 0.1, random(gl_FragCoord.xy));                          // dither

      vec3 sampleVec = tangentSample.x * right + tangentSample.y * up + tangentSample.z * normal + randomOffset;
      irradiance += (texture2D(environmentMap, nTOE(sampleVec)).xyz) * cos(theta) * sin(theta);
      nrSamples++;
    }
	}
  irradiance = PI * irradiance * (1.0 / float(nrSamples )) ;
  
  
  gl_FragColor = vec4(irradiance, 1.0);
}

Dithering made the image lighten up a bit, but the difference wasn't noteworthy until the number of samples were reduced significantly, e.g., it's clear that the dithered image became much smoother when the values were sampleDelta=0.250; SAMPLE_COUNT=16, as compared to the one without dither :

Please let me know if the results look satisfactory, or any further improvements that can be made!

[davepagurek]

Sorry for the delay, nice work! I think the dithering looks much better than with no dithering, and is worth the extra time it takes. Would you be interested in making a PR with your changes?

[sudhanshuv1]

I am interested in making a PR, I will update the sample numbers to sampleDelta=0.100; SAMPLE_COUNT=16 and also replace the implementation of vanDerCorput in p5.js/src/webgl/shaders/imageLightSpecular.frag with the more efficient version in case of WEBGL2. Thanks!