How well does it generalize?

[Jaykob]

I'm currently trying out your method with scans of a foot.
Unfortunately it's not really working yet and I played around with different voxel sizes and different scans.

In terms of dimension and surface characteristics it's quite different compared to the office datasets used for training. So I'm not completely surprised that it doesn't work out of the box.

Do you think that this is to be expected? Would training be the only way to solve my problem here or are there maybe other things that I didn't consider yet? Would you expect an improvement with the multiview part? I'm currently considering 3-4 scans of a foot from different perspectives, so overlap is not huge here...

Please let me know it you need some scans or if this question is not appropriate here!

[ouceduxzk]

I also did similar experiments. my observations are:

1. low overlapping is problematic and performance can be worse than RANSAC
2. our object, like a foot or in my case a teddy bear, have general a closed surface , different from a indoor scan (open), i found that the registration is not robust enough to flip the match, rather just move the corresponding points closer to each other.

I am also visualizing the correspondences, how confident is the weights of the confidences is unclear, theoritically, you can use 3 pair of points to do the transformation if they are real correspondences, but you are not sure here.

i am going to train a FCGH descriptor network on my own dataset with foot, teddy bear, those kind of objects and then evaluate how much better it can possibly improve the pairwise matching.

[Jaykob]

Thanks for sharing your experience!
It‘s similar to what I‘m seeing.

That sounds great. Can you elaborate a bit on your training dataset? How many scans do you use?
It’d be great if you could keep me posted on your progress!

[Grungeby52]

This may not be enough for an accurate registration. After what you have to do, applying the icp algorithm(gpu icp to be faster) might fix the problem. I am currently working with 20 micron precision and it works. Pay attention to Voxel Size...