Basic Information
In image-driven 3D building reconstruction, instance segmentation is fundamental to pixel-wise building component detection, which can be fused with 3D data like point clouds and meshes via camera projection for semantic reconstruction. While deep learning-based segmentation has obtained promising results, it relies heavily on large-scale datasets for training. Unfortunately, existing large-scale image datasets often include irrelevant objects that obstruct building components, making them unsuitable for 3D building reconstruction. This paper addresses this gap by introducing a large-scale building image dataset to facilitate building component segmentation for 3D reconstruction. The dataset comprises 3378 images captured from both interiors and exteriors of 36 university buildings, annotated with 49,380 object instances across 11 classes. Rigorous quality control measures were employed during data collection and annotation. Evaluation of five typical deep learning-based instance segmentation models demonstrates the dataset’s suitability for training and its value as a benchmark dataset for building component segmentation.
Below is the link to the paper:
Mun On Wong, Huaquan Ying, Mengtian Yin, Xiaoyue Yi, Lizhao Xiao, Weilun Duan, Chenchen He, Llewellyn Tang,
Semantic 3D reconstruction-oriented image dataset for building component segmentation,Automation in Construction,Volume 165,2024,105558,ISSN 0926-5805,
https://doi.org/10.1016/j.autcon.2024.105558.
https://www.sciencedirect.com/science/article/pii/S0926580524002942
My Work
I need to convert this dataset into the YOLOV8 format and then train it for architectural component segmentation. This segmentation can be used in point cloud projection to project individual components, such as walls, windows, or ceilings, where I only need specific elements like walls, windows, or ceilings.
Download Hong Kong University Architectural Dataset in YOLOV8 Format
Now, I will share how to train the dataset into a YOLOV8-supported model file.
Training Process
I used the free computational power provided by Kaggle for training, and I trained for a total of 330 epochs.
Install YOLOv8
INPUT:
OUTPUT:
Test YOLOv8
Now we need to test whether YOLOv8 is installed successfully. We will test it with an official image.
INPUT:
If the installation is successful, you will find the following image in the runs/detect/predict
folder. At this point, you can start using YOLOv8.
Begin Training
You can start training the dataset using the following program:
The content of data.yaml
is as follows:
If the training is successful, you will be able to see the training curves in the results.
Below are some details about the scope and labels of architectural components:
Test
Let’s write a piece of code to test the segmentation effect:
The output of this code:
Great,👌 we have obtained a very good result.
Summary
By training the dataset, we obtained a fairly good result for architectural component segmentation, which plays a significant role in subsequent 3D reconstruction and point cloud modeling. Accurate segmentation of architectural components also makes subsequent work much smoother.
Enjoy your usage!