Incorrect norm in spin density for finite sampled regions

[aannabe]

Describe the bug
When using the SpinDensity observable coupled with cell and corner keywords for sampling a finite region (not the whole simulation cell), the obtained norm from qdens is incorrect. For example, for an atom, if the whole simulation cell is sampled, the norm should be the total number of electrons in each spin channel (and this is currently correct). However, if one samples a [L/2, L/2, L/2] finite cell in the corner, the norm should be 1/8 = 0.125 of the original. This is currently incorrect, and the norm is still the number of electrons. So the density is being renormalized somewhere. Consider an Mg atom in a box and sampling a corner region of [L/2, L/2, L/2] (attached). With QE I can get the correct norm (0.12370), while the obtained QMC norm is 0.99716.

I tested both batched and legacy codes, and in both cases I get the incorrect norms, so the issue is likely in qdens. The bug can go easily unnoticed if the finite region is centered, then one expects a value close to the original norm, but this would still be incorrect.

To Reproduce

1. Use a cornered cell of 1/8 volume (7.55890453^3 in this case):

<parameter name="cell">
7.55890453        0.00000000        0.00000000
0.00000000        7.55890453        0.00000000
0.00000000        0.00000000        7.55890453
</parameter>
<parameter name="corner">
0 0 0
</parameter>

2. Generate the xsf file:
   qdens -f xsf -e 0 -r 1 -i vmc_J123.in.xml --noplot vmc_J123.s000.stat.h5
3. The norm can be printed using:
   qdens-radial -p -s Mg --radii=1 -a "qmcpack" vmc_J123.s000.SpinDensity_u.xsf

Expected behavior
The norm should be 1/8 of the whole cell.

System:

• NERSC. Mg atom in a box of L = 15.11780906 bohr.

Additional context
Previously, qdens ignored the cell and corner regions, resulting in incorrect densities. This was fixed in #4638, but evidently not everything is resolved yet. From my brief look, I am not sure where this normalization happens, but I will link a PR that will block the use of these keywords in qdens for now. The workaround for this bug is to sample the whole simulation space for now.

test_corner.zip

[prckent]

This is an interesting observation and good to note. Presumably no one has hit it in practice otherwise they would have commented, so it is happily not a major blocker. For periodic cases I think 99.99% of the time people use the full simulation cell. Where getting the normalization correct would be particular helpful is cases where the system is finite (atom, molecule) and the user specifies too small a region for the density to capture all the electron positions. For the finite (fully non-periodic) case, a sensible quick solution would be to print a warning and warn about the current bug. No need to spend any time until someone hits this.

[aannabe]

Per @jtkrogel 's suggestion, I looked at the above stat.h5 file "by hand" and summed up the density for one spin channel. The norm I got is 0.999999. So, it is already incorrect in the stat.h5 file and the bug appears to be in qmcpack and not in qdens.

I agree that this is usually not an issue in PBC. But for OBC, the user will not notice the bug for a reasonably sized sampling cell since the norm will be almost correct. However, the resulting density will remain incorrect except for very large cell sizes.