Introduction
Building digitalization is an important topic for the modernization of the construction industry, opening up many possibilities for monitoring and automation of tasks and processes. Under this field falls the gathering of information from different sources, such as floor plans or building pictures, and the creation of added value from its aggregation. Traditionally, this digitalization process had to be manually performed by experts, consuming time and valuable resources. However, recent advances in AI offer new possibilities of automation of such tasks, particularly through the use of deep neural networks. In our group, we tackle this topic through the lines of research described in the following sections
Automatic analysis of Scanned Floor Plans
Floor plans contain information necessary for many different processes, such as construction, refurbishing or maintenance of buildings. They are, however, not always available in digital form, especially so for older buildings that might have been built before the use of CAD tools was as generalized as it is today. To address this problem, we propose a pipeline that takes a floor plan as input and generates a digitalized, vectorial representation of it, containing the semantic information necessary to potentially use it with computer design software. Additionally, we generate a 3D model of the results. The steps of the pipeline can be visualized in the following diagram.
As depicted in the diagram above:
- The scan of a floor plan is taken as an input.
- This floor plan image is fed to a symbol detection module that uses a deep neural network to find relevant construction elements, such as doors and windows.
- The image is also fed in parallel to a segmentation module that generates a binary mask representing the structural elements present in the plan, such as walls and columns.
- A vectorization algorithm is applied to the binary mask that divides it into individual wall segments and columns.
- A post-processing step is applied to merge the vectorial structural information with the results of the symbol detection module and apply necessary corrections.
- This merged representation is used to generate a 3D model of the scanned plan.
Registration of Building Images within the Corresponding Semantic 3D Model
We create algorithms to achieve precise photo registration within a building through AI-based 6D camera pose estimation. In this process, the exact orientation and position of the camera in a building floor is determined.
A semantic 3D model previously created by analyzing plans or point clouds serves as reference for the registration. Only basic semantic features such as ceiling, floor, wall, door and window are used to enable precise mapping of the photo to the 3D model. A detailed BIM model is not required.
If needed to resolve ambiguities, the method provides the top-n most likely camera poses for a photo. Filters and prior knowledge, such as the assumption that all images are from the same room, can further refine the result.
Precise overlay of the photo onto the model yields semantic information for the image. Objects identified in the picture by other AI services, such as fire safety equipment, can then be seamlessly transferred to the model.
Combination of Modules
The modules described in the previous sections have a strong overlapping in their areas of applicability: the image registration module expects a building 3D model as input to use as a reference, which is exactly the output of the floor plan analysis module. This means, they can be seamlessly combined to aggregate existing information from different sources. While it would not originally be possible to use the image registration module if no digital model of the building is available, the combination with the floor plan analysis pipeline allows us to use it as long as at least a printed floor plan exists. Moreover, these previously unrelated pieces of information, i.e. the plan and the photos of the building, are now coupled into a single representation, which makes them more valuable than they were separately.
Publications
A. Cambeiro Barreiro, M. Trzeciakiewicz, A. Hilsmann, P. Eisert, Automatic Reconstruction of Semantic 3D Models from 2D Floor Plans, Accepted at 18th International Conference on Machine Vision Applications (MVA) 2023. [PDF]
N. Gard, A. Cambeiro Barreiro, Towards Automated Digital Building Model Generation from Floorplans and On-Site Images, Forum Bauinformatik 2023. [PDF]