Question about depth map quality

[brian892]

Hello,
I bought OAK-D as part of the Kickstarter campaign, but the depth quality wasn't great and it was also unstable back then.
I gave it another try after almost 6 months with the latest library with same stereo settings and the quality of depth map is MUCH better compared to version 2.10 of the depthai library. It's also stable, ran it for 2 days with subpixel mode, previously it was failing after a few seconds.
Can you describe the changes? Do you have a changelog ?

[Luxonis-Brandon]

Hi @brian892 ,

Thanks for circling back and for the question. So actually we re-wrote the entire depth pipelining from the ground up, throwing away everything that was there previously. So the changelog unfortunately for that is: Everything. As it's not really a change, but a throw away and write from first principles.

So instead of focusing on changes it's likely best to focus on the functionality as below:
https://docs.luxonis.com/projects/api/en/latest/components/nodes/stereo_depth/#stereodepth

The aggregate changelogs are here however:
https://github.com/luxonis/depthai-core/releases

Thanks,
Brandon

[brian892]

Thanks @Luxonis-Brandon .
Checked the changelog, and looks like the stability issues were resolved in release 2.11.0.0
Checked it, but the depth quality was the same/similar as before.
2.12.0.0 looked much better. I was able to narrow it down to these changes:
https://github.com/luxonis/depthai-shared/pull/62/files#diff-60a341052c5390c70640bc615df9779705e0b205b2f6cb94e0942a84729e0410R125
I tested with same threshold and depth mode, so I assume that enableMeanMode and alpha beta changes made the difference.
I will read through the documentation about the stereo node, it's quite long :)
Thanks for your reply.

[SzabolcsGergely]

Hello.
The parameters alpha beta are affecting how the cost value is calculated, e.g. a value of how similar are two pixels. Previously the absolute difference between 2-pixel values was taken into account alongside census descriptors, which is more susceptible to noise. After the modifications, only the descriptor value from census transform is taken into account.
Census transform was modified to compare each pixel in the window with the mean window value instead of the central pixel, aka. modified census transform, so it doesn't rely as much on the center pixel. It did result in better matching accuracy/less holes in the depth map.

The parameters were changed based on tests from different scenes with both OAK-D and OAK-D-PRO.