We, the Gradient Spaces group at Stanford University, together with collaborators from Oregon State University and ETH Zurich are hosting the 4th International Scan-to-BIM challenge. Our challenge is hosted at the workshop on Computer Vision In The Built Environment For The Design, Construction and Operation of Buildings in conjunction with The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024. To this end, the challenge will include the following tasks:
- 2D Floorplan Reconstruction
- 3D Building Model Reconstruction
Links to Download Data:
- Training Data: Point Cloud + Ground Truth
- Validation Data: Point Cloud + Ground Truth
- Testing Data: Point Cloud ONLY
The submission should be made in the same JSON format as in the provided ground truth. Codalab challenge links:
We would like to note that ALL the submissions need to be constructed automatically . Manual reconstructions are against the spirit of this challenge and will not be allowed.
The 2D Floorplan Reconstruction challenge contains a total of 31 buildings with multiple floors each and dozens of rooms on each floor. Of which, 20 buildings are designated as the training set, with a total of 49 point clouds. The validation and testing sets contain 5.5 buildings with 21 point clouds each. For each model, there is an aligned point cloud in LAZ format. For the training and validation sets, a corresponding floorplan aligned with the coordinate system of the point cloud is also provided.
The training data consists of 16 floors from 8 buildings. For each model, there is an aligned point cloud in LAZ format. The 3D building coordinates for walls, columns and doors are presented in 3 separate JSON files. We focus on the reconstruction of walls, columns, and doors.
We include metrics to evaluate the reconstruction of the walls, doors, and columns, as well as floor area in 2D :
-
Geometric Metrics
a. IoU of each room (a room is defined as a completely separated area with walls and doors).
b. Accuracy of endpoints : Precision/Recall at 3 different thresholds: 5cm, 10cm and 20cm, as well as the F-measure at each threshold will be evaluated in the coordinate system of the point cloud. The provided endpoints will be matched with the Hungarian algorithm to the point cloud, and every point that is within a certain threshold will be determined as a match.
c. Orientation For each matched line between the ground truth, we will compute the cosine similarity metric between them as the normalized dot product. If a line is not matched with ground truth, the cosine metric will be zero. Finally, the metric will be averaged over all the ground truth lines. -
Topological Metrics
a. Warping error : The warping error will first warp the predicted floorplan to the ground truth with a homotopic deformation, and then compute the pixels that cannot match after the deformation.
b. Betti number error : The Betti number error will compare the Betti numbers between the prediction and the ground truth and output the absolute value of the difference.
For exact details and code regarding evaluation metrics, please look here: 2d_floorplan_eval
We evaluate the submissions on a variety of metrics :
- 3D IoU of the 3D bounding box of each wall
- Accuracy of the endpoints : Precision/Recall at 3 different thresholds: 5cm, 10cm and 20cm, as well as F-measure will be evaluated in the coordinate system of the point cloud. The provided endpoints will be matched with the Hungarian algorithm to the point cloud, and every point that is within a certain threshold will be determined as a match. We evaluate per each of the three semantic types (i.e., wall, column, door).
For exact details and code regarding evaluation metrics, please look here: 3d_matching_eval
For more details, please refer to the challenge website: https://cv4aec.github.io/