Questions about homography

[Freja1122]

Hi, Thanks for your amazing job. I have two questions about homography.

1. Plane constraints
   Homography needs to meet the plane constraints, or suppose the camera only rotates. Why can the background with the foreground removed can meet the requirements.

2. Close target tracking
   In the scene I am dealing with, the target will be very close (0.5m). In this case, the background cannot be seen, so the homography cannot be calculated.
   But I have a depth map and camera pose（rpy,xyz) . Can I use these informations to calculate homography? If the homography obtained in this way can be used to achieve good results for close target tracking task?

[GeekAlexis]

Plane constraints are good approximations, which do well in most scenarios except when there is parallax.

Is your camera moving? If not just disable camera motion estimation.

[Freja1122]

Question1:
My camera is moving, and the person will appear far away (5m) or very close (0.5m). The height of the camera is about 50cm from the ground, not at a very high place.
Do you have any suggestions to be able to handle such a scene?

Question2:
At the same time, I also encountered an error, someone occur -->no person-->someone occur again. it will no longer recognize, and output "camera motion estimation failed" all the time.
command line output after 00:32:

Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Detected:     person 141 at ( 993, 407), active: True confirm: False
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Detected:     person 142 at ( 993, 407), active: True confirm: False
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Detected:     person 143 at ( 993, 407), active: True confirm: False
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Camera motion estimation failed, len(keypoints) == 0
Detected:     person 144 at ( 994, 406), active: True confirm: False
Camera motion estimation failed, len(keypoints) == 0

[Update] One solution for Question2:
After I move lines 149&150 to 129&130 in flow.py
it might be a bug?

[GeekAlexis]

Q1: Since you have an indoor scene and targets are close, I suggest you disable camera motion compensation #95 and rely on feature matching. You can lower motion_weight and set detector_frame_skip to 1.

If you know both camera extrinsics and intrinsics at each frame, and the 3D coordinates from depth map, you can sample a few 3D points in the bounding box and project them to the two frames to calculate homography for each track. But this is way more complicated.

Q2: I can't access your output link. You are not supposed to move these lines.

[Freja1122]

Since moving those two lines is not correct solution, output link is not important anymore.

Can u access my input link?
In the end of this video, target is not close. But it won’t continue to track, and output “camera motion estimation failed” repeatedly.
Can u reproduce this problem? Could u please tell me how to solve it correctly?

[GeekAlexis]

I still can’t. You need to change access to public.

…

On Mon, Jul 5, 2021 at 6:37 PM Ann ***@***.***> wrote: Since move those two lines is not correct solution, output link is not important anymore. Can u access my input link? https://drive.google.com/file/d/1kxSSk3TRDysV2lXzytBulqKlPSna5_CK/view?usp=sharing In the end of this video, target is not close. But it won’t continue to tracking, and output “camera motion estimation failed”. Can u reproduce this problem? Could u please tell me how to solve it correctly? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#144 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGNJO5TLL56ODHIVRVYBA4TTWJM7FANCNFSM472HRQEQ> .

[Freja1122]

It gets keypoints from prev_frame_small_bg in this line. When len(keypoints) == 0, if u don't update self.prev_frame_gray, len(keypoints) will always be 0 after that.

[GeekAlexis]

@sunniee Thanks, the bug should be fixed by commit 7a0c7d2.

[Freja1122]

BTW, I meet duplicate bboxes, and id switch.

So i change these params for close target tracking:

-            "iou_thresh": 0.4,
+            "iou_thresh": 0.2,

-                "std_factor_acc": 2.25,
-                "std_offset_acc": 78.5,
+                "std_factor_acc": 20.25,
+                "std_offset_acc": 200.5,

I get better results (less duplicate bboxes, and less id switch)

Will there be any hidden problems after adjusting these parameters? For example, tracking will fail when there are many people, people are close, etc.

[GeekAlexis]

I believe these will result in more switches for crowded small targets. But the parameter changes should be good for large fast moving objects, which seems to be your case.

[Freja1122]

Thank you for your prompt communication. All my problems have been resolved, and this issue can be closed.

[krishnaadithya]

@sunniee is your close target tracking working?